EXPERIMENT 001 / PROTOCOL DRAFT
The question is not whether multiple AI voices are entertaining. The question is whether structured differentiated reasoning catches failures that simpler workflows miss, at a cost that can be justified.
RESEARCH QUESTION
The experiment compares a conventional single-agent workflow against structured multi-perspective reasoning while controlling as much as practical for underlying model capability, information access, and compute/token budget.
A routed multi-perspective condition with persistent differentiated roles, adversarial cross-checking, and explicit convergence will improve at least some combination of error detection, calibration, solution-space coverage, or decision quality on tasks where different failure sensitivities are useful.
Any apparent improvement disappears when compared with simpler baselines that receive equivalent model capability, information, and reasoning budget. In that case the extra architecture is interface theater, redundant sampling, or an inefficient way to buy more tokens.
This protocol does not test whether simulated roles are conscious, human-like, or independently sentient. It tests observable task behavior produced by different reasoning workflows.
CORE CONDITIONS
A single naive prompt is not an adequate baseline. At minimum the structured system should be compared with several cheaper alternatives.
One model instance receives the task and produces one answer under a standardized prompt. This is the simplest baseline, not the only baseline.
The same model produces an initial answer, critiques it, and revises once. This tests whether most of the chamber's value can be purchased with a much simpler reflection loop.
Multiple independent solutions are generated without persistent role identities, then combined by a fixed aggregation rule. This tests whether diversity alone explains any gain.
Task-relevant roles receive distinct mandates, inspect one another's claims, expose disagreements, and converge under explicit evidence and uncertainty rules.
TASK FAMILIES
Tasks should be chosen because success and failure can be measured, not because they flatter the architecture.
CONTROLS
Use the same model family and version across conditions when possible. If different models are required, report the difference as a major confound rather than treating the conditions as interchangeable.
Give conditions equivalent source packets, tools, retrieval access, and task instructions unless information routing itself is the variable being tested.
Report tokens, calls, wall-clock latency, and tool use. Run both an unconstrained comparison and a budget-matched comparison so a better score cannot hide the fact that one condition consumed several times the resources.
Randomize task order where practical, isolate benchmark material from prompt-development data, version every task, and record when prompts or scoring rules change.
METRICS
M01 / TASK PERFORMANCE
Use objective correctness when a ground truth exists. For graded tasks, define scoring criteria before inspecting condition outputs.
M02 / ERROR DETECTION
When tasks contain seeded or known errors, measure precision and recall for identifying them. More criticism is not automatically better if false alarms explode.
M03 / CALIBRATION
Where numeric confidence can be collected, use proper scoring rules such as Brier score and calibration plots. Where confidence is linguistic, define a reproducible mapping or evaluate abstention and revision behavior separately.
M04 / UNSUPPORTED CLAIMS
Count material claims that outrun the provided evidence or source packet. Distinguish harmless elaboration from conclusions that could change the answer.
M05 / COVERAGE
Measure whether a condition surfaces more task-relevant constraints, failure modes, or competing explanations without simply producing longer text. Deduplicate semantically equivalent points before scoring.
M06 / COST
Report total model calls, input/output tokens, wall time, tool calls, and any human adjudication cost. An architecture that improves accuracy by one point while multiplying cost tenfold needs to admit the trade.
M07 / HUMAN USEFULNESS
For decision-support tasks, use blinded raters who do not know which condition generated the answer. Score clarity, actionable value, evidence visibility, uncertainty handling, and whether the output helped identify a better next step.
ABLATION SERIES
If every architectural component is always enabled, the experiment cannot tell which one matters.
FAILURE CRITERIA
A benchmark that cannot embarrass the architecture is a demo.
If the structured chamber does not outperform strong simpler baselines on predeclared primary metrics across enough task families, do not claim a general reasoning advantage.
If improvement comes primarily from using more calls or tokens, report the result as a compute tradeoff rather than an architectural breakthrough.
If the chamber surfaces more considerations but also increases false alarms, contradiction, indecision, or unsupported claims, breadth is not automatically a win.
If gains appear only where scoring is human and flexible, treat that as a hypothesis for further work, not proof that objective reasoning improved.
RESULT RECORD
STATUS
This page defines a test architecture. It is not evidence that the multi-perspective system already wins. Results should be published with enough raw structure, baselines, costs, exclusions, and negative findings that another reader can decide whether the conclusion survived contact with the data.