Simone Systems Research
Independent AI Systems Research
Research on how AI systems coordinate, verify results, allocate compute, and improve reliably.
Recent Evidence
docs-only share of those commits (855/1,979)
composition, not merge rate
Source: summary.json →sampled PRs that changed the playing agent
80-PR stratified inspection, single inspector
Source: sample80_summary.json →assumptions invalidated in the initial prior-art sweep
grew to 21 after a later step
Source: thebus →model vendors in The Council (Claude, GPT, Gemini, Grok)
fast-tier models: Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Grok 4.3
Source: thecouncil →Research Focus
Core questions guiding our systems development and empirical investigations.
Agent Orchestration
How should heterogeneous models, tools, and agents divide work and coordinate reliably?
Evaluation & Verification
How do we distinguish genuine improvement from false progress, benchmark noise, and brittle behavior?
Compute Economics
When does additional inference improve verified outcomes enough to justify its cost?
Adaptive AI Systems
How can systems learn from measured failures and improve their own workflows without losing control or reproducibility?
Research Projects
Public artifacts, work awaiting a publication decision, and registered plans.
SEED
- Question
- Does an incident-earned, machine-checked control plane reduce invalid completion claims, recovery cost, or unnecessary human intervention in agent-driven software work, relative to a bare repository?
- Status
- Measurement protocol and instrument are public. Hypothesis C-004 is registered and untested. Fleet orchestration is deferred until a single-agent baseline is measured.
- Evidence
- Author-run census of 1,979 PR-linked commits on one repository; 43.2% changed only docs.
The Council
- Question
- Can independent answers, peer critique, cross-vendor claim verification, and synthesis produce an auditable answer with a full evidence trail?
- Status
- Working system. Fixture runs are simulated evidence. Live runs are captured evidence, not a benchmark.
- Evidence
- 4 model vendors, each on its fast tier.
BigBoss
- Question
- Can a local approval plane keep human decision authority over gated agent actions without stalling the harness?
- Status
- Harness proposes, BigBoss evaluates and routes, human approval is authoritative for gated actions, decisions are bound to action hashes, state and audit persist. MCP, Codex bridge, phone approval, and cost metering are secondary.
- Evidence
- Tests and fixture demo run offline. No outcome measurement yet.
The Bus
- Question
- Was there unclaimed design space for a shared coordination bus between heterogeneous agents?
- Status
- Retired. Retraction ledger kept. Do not build on it.
- Evidence
- Prior-art sweep invalidated 17 assumptions in the initial sweep.
Godot AI Methodology
- Question
- Are deterministic, observable, testable, modular, data-driven, documented codebases easier for AI coding systems to modify safely and verify?
- Status
- Second-domain application. Methodology documents and gate skills. Examples original; no decompilation, no extracted assets, no copied source.
- Evidence
- Gate scripts run offline. No measured effect claimed.
Every card links only to artifacts a reader can open today. The central causal claim, that this control plane improves outcomes over a bare repository, is registered as a hypothesis and has not been tested.
How the projects relate
- SEED
- measures whether agent-driven work constitutes verified progress.
- BigBoss
- controls which autonomous actions can occur and preserves human decision authority.
- The Council
- tests independent verification through heterogeneous model families.
- The Bus
- shows adversarial review terminating a bad architecture before further implementation.
- Godot Methodology
- tests whether the same verification principles generalize into software architecture.
Research Principles
Standard operating commitments applied across all experiments, evaluations, and published artifacts.
Evidence before promotion
Improvements should survive measurement, not merely look plausible.
Independent verification
Positive results should be reproduced outside the context that generated them.
Compute must earn its cost
Expensive inference should be reserved for decisions where it materially improves verified outcomes.
Negative results are retained
Failed hypotheses are useful evidence and should not disappear from the record.
Artifacts matter
Claims should refer to the exact code, model, configuration, and environment actually evaluated.
Research collaborations
Simone Systems Research welcomes discussions with researchers and technical teams working on agent evaluation, verification architecture, human-control systems, and compute efficiency.
jon@simoneresearch.comResearch Notes
Perspectives and working notes. Results with data are labeled as such when they exist.