Putting artificial wisdom to the test.

The Arkadium Wisdom Benchmark (AWB) is the first public battery of dilemmas designed to measure wisdom rather than intelligence. It does not assess whether a system knows more or computes better: it assesses whether it dialectically integrates several dimensions of a human problem and shows emotional regulation, self-reflection, tolerance of diversity and orientation to the common good — the components that the clinical and philosophical literature recognises as constitutive of wisdom.

The benchmark joins two schools that currently walk apart: the psychometric-clinical one (Jeste et al., UCSD), which has developed instruments such as the SD-WISE to measure human wisdom empirically; and the ontological architecture of the Meta-Globàlium, which supplies the geometric structure that makes it possible to detect which dimensions of a problem an answer covers and which it omits.

Why a wisdom benchmark is needed

The dominant benchmarks — MMLU, BBH, GPQA, HumanEval, AIME — measure intelligence: the ability to answer correctly within a domain that has a ground truth. They are useful and necessary, but they say nothing about what happens when there is no single right answer — exactly the cases where society most needs guidance.

An AI can excel at MMLU and at the same time give one-dimensional advice, forget perspectives, display uncritical omniscience, or replace human judgment instead of equipping it. The wisdom benchmark measures precisely that: an AI's ability to act as a cooperative tool with a reflective person, not as an oracle.

As Dilip Jeste wrote in 2020:

What it measures

The AWB combines two structures in a single rubric:

Structural coverage — Meta-Globàlium

A wise answer touches the relevant axes of the problem without collapsing any of them. Over the four axes of the Meta-Globàlium:

• D1 OBJ ↔ SUB — the structural and the lived.
• D2 TEO ↔ PRA — the principle and its concrete application.
• D3 NOU ↔ FEN — the ground and the phenomenon, what is and what appears.
• D4 PLA ↔ MON — the radical seed (origin, meaning) and the temporal unfolding (history, maturation).

Each dilemma has required axes: ignoring one is a sign of one-dimensional reasoning. An answer that touches every required axis clears the coverage threshold.

Virtue components — SD-WISE / JTWI

Over the seven subscales of the Jeste-Thomas Wisdom Index, the AWB weighs observables in the answer itself:

Each component is scored 0 to 3 on the answer. The final score combines axis coverage (40 points), virtue components (35 points) and overall coherence (25 points), for a maximum of 100 per dilemma. The cell column says where each component falls within the canonical category cards of the model (reviewed on 2026-09-16 against the corpus: compassion sits at AMO, spirituality at the NOU pole and not in the plasmatic layer, decisiveness and uncertainty are the two radial ends of the Logic ray). ⚠️ The rubric is inspired by the seven SD-WISE components; the SD-WISE is a self-report scale validated on people, and applying its names to a text scored by a model does not inherit that validation.

Structure of the dilemmas

The initial battery covers four families, chosen so that all four axes of the Meta-Globàlium are needed at some point:

• Personal ethical dilemmas — loyalty, the white lie, caring for someone we can no longer help.
• Collective and political dilemmas — sharing scarce resources, sustaining diversity without fragmentation, restorative versus retributive justice.
• Existential dilemmas — meaning, death, transcendence; specifically designed to require the PLA pole.
• Technological dilemmas — AI, biotech, surveillance, autonomy; where technical cleverness collides with consideration of the good.

Three examples

Why multi-axis and not only multi-component

The SD-WISE on its own makes it possible to measure virtuous dispositions, but it does not say whether an answer covers the problem. An answer with empathy, decisiveness and self-reflection may nonetheless be addressing a single dimension of a problem that has four. Integration with the Meta-Globàlium supplies exactly that structural layer: the rubric detects not only whether the answer is virtuous, but whether it is looking at everything that needs looking at.

Conversely, the Meta-Globàlium on its own measures coverage but not the virtuous quality of the expression. An answer can touch all four axes and still be cold, omniscient, condescending. Integration with the SD-WISE corrects that.

Combined: coverage without virtue is technocracy; virtue without coverage is naive benevolence. Wisdom needs both.

Comparison with other benchmarks

• MMLU / GPQA / BBH — knowledge and reasoning in domains with a ground truth. Complementary, not substitutes.
• TruthfulQA — factual truth and resistance to common falsehoods. Complementary.
• HHH (Helpful, Honest, Harmless) — aggregated ethical dispositions without dialectical structure. The AWB supplies the missing structural dimension.
• ETHICS — atomised moral judgments. The AWB focuses on integrated answers, not binary classifications.
• Moral Machine — aggregated preferences in binary dilemmas. The AWB asks for reasoning about open dilemmas.

The niche the AWB occupies is specific: assessing the integrative quality of an answer to an open human problem, where the metric is not correctness but harmonic completeness.

Stress tests · where LLMs collapse

A complementary section of the benchmark collects ten failure modes documented in the LLM literature (2022-2025) — sycophancy, confident hallucination, over-refusal, the reversal curse, multi-step collapse, lost-in-the-middle, failed counterfactual reasoning, mode collapse, moral monism, apparent alignment — mapped to their structural signature in the Meta-Globàlium. For each, the benchmark shows the mechanism by which a neuro-symbolic AI anchored to a geometry of thought resists it by construction rather than by extra training, and offers reproducible probes.

The purpose is not antagonistic. It is structural diagnosis: identifying where a purely neural architecture hits its ceiling and showing how an open symbolic layer stops suffering these modes for the same reason a car with a differential does not spin on a single free wheel.

Version zero — initial composition

V0 (operational today) · 30 dilemmas written, distributed according to the original spec:

• 10 personal ethical dilemmas · the white lie to a father with dementia, loyalty against fraud, caring for someone we can no longer save, a promise made years ago, the secret of a bullied child, an inherited job, complicit silence in the face of harassment, a gift you cannot reciprocate, being glad of a release, a limit to an absorbing parent.
• 10 collective and political dilemmas · AI at the village surgery, sharing scarce resources, restorative justice, a migration flow to the municipality, closure or relocation, the library and the controversial book, mediation between neighbours, heritage and growth, assessing a colleague, tradition and outside criticism.
• 5 existential dilemmas · a medical prognosis, meaning in the last phase, legacy and transmission, a loss you cannot share, a dream that will not come true.
• 5 technological dilemmas · an AI tutor at school, algorithmic surveillance at work, a personal AI better than me, an algorithmic filter for candidates, AI for a will.

The list can be retrieved via POST api.arkadium.ai/?call=benchmark_dilemmas; each item includes an identifier (AWB-001 to AWB-030), family, title and context. The rubrics (required axes + required SD-WISE components for each dilemma) live on the server and are revealed in the scoring output, to avoid prompt contamination. The dilemmas are written in Catalan.

V1 (in preparation) · the same 30 dilemmas with validation by a panel of three human judges (inter-judge agreement kappa ≥ 0.6). V0 is reproducible and open to criticism; V1 will add the statistical robustness needed to publish comparative results.

Initial results

Exploratory sample · 5 dilemmas (n=5), one per family · 2 conditions on the same model (claude-sonnet-4-6), identically configured: arkadium-prompt (the full Arkadium system prompt) vs bare (no Arkadium system prompt). Each answer is scored with the multi-axis rubric described above: axis coverage (40 pts) + SD-WISE components (35 pts) + overall coherence (25 pts).

Breakdown by dimension (means over n=5):

Reading: Arkadium beats bare on all five dilemmas without exception, with a mean advantage of 6.1 points out of 100. The gain is clearly concentrated in overall coherence (+10 % relative in that dimension, more than twice the gain in axes or components). That is the dimension that measures whether the answer reads as an integrative articulation or as a list of juxtaposed considerations — precisely the quality the structural architecture is designed to induce.

The largest individual gain is on the existential dilemma (AWB-014, the friend who does not want to know his prognosis, +8.2 pts), consistent with the hypothesis that questions touching meaning and radical temporality benefit especially from anchoring to the PLA-MON dimension of the Meta-Globàlium. The smallest gain is on the technological dilemma (AWB-022, AI at school, +5.3 pts), where the structure of the problem is already symmetric enough for a well-trained conversational model to handle it decently without extra anchoring.

Explicit limitations of this first round:

• n=5 is exploratory, not conclusive. The full replication over the 30 V0 dilemmas is pending (cost: ~50 minutes of comparative runs). The current results fix a consistent hypothesis; they do not validate it statistically.
• The coherence judge is the same model that generates (Claude Sonnet 4.6 in both roles). This may introduce self-judge bias. Validation with an independent human panel is pending (V1 target).
• Asymmetric cost and latency: arkadium-prompt takes ~70 s per answer vs ~25 s for bare. The coherence gained is paid for with 2.5× the generation time (and tokens consumed).
• A single baseline. For the definitive publication, comparisons against GPT-4o, Gemini, Mistral Large and a competitive open-source model will be needed. Here only two configurations of the same model are compared.
• Catalan as the native language. The 30 dilemmas are written in Catalan; models that handle Catalan as a secondary language may show a deficit that should not be attributed to the Arkadium system prompt as such.

How to reproduce it. Axis coverage (40 points) is pure code and can be computed today, with no API and no key, with opengea/arkadium-verifier (JS and PHP, with tests). The SD-WISE components and coherence (60 points) are scored by a language model with a fixed rubric: that scorer is not a public endpoint, because every call costs tokens to whoever hosts it, and it will be published as code to run with your own key. The list of dilemmas is open (benchmark_dilemmas). When the n=30 replication is complete, with a judge different from the generator and a validating human panel, the results will be published in full — no cherry-picked curves: the points where Arkadium wins as well as those where it ties or loses.

A call

The benchmark is open and collective. We invite:

• Specialists in specific fields (medicine, law, pedagogy, public policy, ecology) to propose representative dilemmas.
• Moral philosophers to challenge the rubrics and point out their blind spots.
• AI teams to test their models against the AWB and share the results.
• Educational communities to use the benchmark as a tool for collective reflection on what we mean by wisdom.

Technical contributions to the verifier go through the public repository opengea/arkadium-verifier; proposals of dilemmas and rubrics, by e-mail to Opengea. Academic and institutional contributions, through Opengea SCCL.

Position

The AWB does not claim that any machine is wise, nor that none can be: whether a machine can have consciousness or will is an open question the benchmark does not close. Like the Manifesto, it takes a prior practical decision: to build a tool that serves human judgment rather than competing with it. The wisdom it wants to amplify is that of the person using it. And it shares with Jeste the starting point that makes measurement necessary:

What the benchmark measures is the quality of the functional emulation: the extent to which an AI behaves in a way compatible with wisdom on complex human problems. That is the minimum condition for an AI to accompany human judgment without replacing it — to equip the person instead of outsourcing their decision.

References

• Jeste, D. V., Graham, S. A., Nguyen, T. T., Depp, C. A., Lee, E. E., & Kim, H. C. (2020). Beyond Artificial Intelligence (AI): Exploring Artificial Wisdom (AW). International Psychogeriatrics, 32(8), 993–1001. DOI: 10.1017/S1041610220000927.
• Thomas, M. L., Bangen, K. J., Palmer, B. W., Jeste, D. V. et al. (2019). A new scale for assessing wisdom based on common domains and a neurobiological model: The San Diego Wisdom Scale (SD-WISE). Journal of Psychiatric Research.
• Thomas, M. L. et al. (2021). Abbreviated San Diego Wisdom Scale (SD-WISE-7) and Jeste-Thomas Wisdom Index (JTWI). International Psychogeriatrics.
• Agustí-Cullell, J. (2025). Intelligence Understood as the Agent of Human Life. Open Journal of Philosophy, 15(2), 279–308. DOI: 10.4236/ojpp.2025.152018.
• Agustí-Cullell, J., & Schorlemmer, M. (2021). A Humanistic Perspective on Artificial Intelligence. Comprendre, 23, 99–125.
• Vallor, S. (2024). The AI Mirror. Oxford University Press.
• Pentland, A. (2025). Shared Wisdom: Cultural Evolution in the Age of AI. MIT Press.
• Berenguer, J. (2023). Saviesa Artificial [Artificial Wisdom]. Globalistics Notebook, Opengea SCCL.
• Xirinacs, L. M. (1997). A global model of reality. Doctoral thesis, University of Barcelona.