Home 00 CLI 01 Neuralese 02 MultiThink 03 Research 04 Models 05
Home / MultiThink

MultiThink

One brain, two hemispheres, thinking together — two independently trained models working at the same time rather than in turn. An architecture we are building, and actively researching.

The idea

Agents take turns. That's the bottleneck.

Every multi-agent system in production today has its models take turns. One writes, the other reads, and nothing happens at the same time — so most of what the first model knew never survives the compression into words, and nothing can intervene until it has finished speaking.

What runs today couples two independently pretrained models concurrently rather than in turn. One generates. The other works alongside it on separate hardware — reading the stream as it appears, dispatching focused sub-agents, consolidating what they find, and splicing a correction into the context so the generator carries on from where it was instead of re-deriving from scratch. The second model works while the first is actively emitting tokens for a substantial share of the time; a turn-based relay scores zero by construction. We are not printing a precise figure here yet. Our first one read 100% until we found the metric was counting time spent waiting at a barrier as parallel work, and the corrected numbers we have are not yet consistent enough to publish. We would rather re-measure than restate.

The two models still talk in text. Coupling them at the vector level — passing state instead of sentences — is what we are building toward, and it is not what runs today. We would rather write that here than let the rest of this page imply otherwise.

Two hemispheres of one brain: the left dense with an active network, the right sparse and watching alongside it.
How it works

Four parts. One never stops.

This is the target architecture. Two of these four run today: the generator and the orchestrator, communicating in text. The guardian and the latent bus both depend on the vector channel described above, which we have not built — they are design, not software, and are described here as what we are working toward.

Generator

Writes, and never pauses

The model the user is actually waiting on. It streams continuously, and every other component has to work around a stream already in motion. When a correction comes in the generation call is restarted, but the accumulated context is kept — so it carries on from where it was rather than re-deriving the answer from scratch.

Orchestrator

Thinks before it speaks

Dispatches focused sub-agents, then consolidates what they return through its own forward pass before anything reaches the generator. This ordering isn't cosmetic: transfer between models is directional, and raw sub-agent output fed straight to the generator is the weak-to-strong case, which measurably degrades the stronger model. Routing it through the orchestrator first keeps the channel strong-to-strong.

Guardian

Watches the trajectory, not the text

Runs cheap arithmetic on the internal state the generator is already producing — drift, uncertainty, and distance from the receiving model's own familiar territory. All of it costs microseconds. The expensive model is woken on an alarm, never on a schedule; a guardian that costs a full forward pass per token costs more than the work it guards.

Latent bus

Event-driven, and forgets fast

Messages are sent on events — a detected gap, a finished sub-task, a guardian alarm — not on every token. Streaming continuously would serialize two models that should be running in parallel, which is the opposite of the point. Packets carry an expiry, and a stale one is dropped rather than delivered, because a correction describing a state the generator has already moved past is worse than no correction at all.

HYPER-X1
HYPER-X2
+
MultiThink
IN PRACTICE

What a dispatch looks like

The channel between the two models really is English today — the vector-level version is still ahead of us. What this illustrates is the timing: the generator keeps working while the orchestrator dispatches sub-agents in parallel, consolidates what they find, and hands back one result instead of raw fragments.

Conceptual illustration

A correction is applied only to what the generator is producing — never to the prompt it is reacting to. In the vector-level version we are working toward, it would land at the next decode step and be bounded in magnitude; today it arrives as a note spliced into the context, and the generator resumes with what it had already worked out.

How we work

Cheap gates kill expensive ones.

The pipeline is ordered so the tests that cost an hour run before the tests that cost a week. Each gate can end the project on its own. None of the early ones is evidence of anything — they're permission to keep going.

  1. G0

    Are these two models even mappable?

    The cheapest and most predictive test we have. Across 23 published model pairs, token-level vocabulary agreement predicts downstream transfer quality at r = 0.898. Every documented cross-family success sat well above the line; every failure sat at or below a quarter.

  2. G1

    Build the paired corpus

    Collect matched internal states from both models on the same inputs.

  3. G2

    What is the most constrained map that fits?

    Deliberately not the most powerful one. A translation layer with more capacity than the task can't distinguish "these two models share a geometry" from "the translator learned to do the job itself" — a distinction that has been shown to collapse entirely when the map is allowed to get expressive enough. Keeping it constrained is what makes a positive result mean anything.

  4. G3

    Does the channel carry content? the gate

    A causal audit that swaps the real message for a wrong one and measures what survives. Everything before this point is necessary and proves nothing. A translation layer can fit the geometry almost perfectly and still communicate nothing — and because geometric fit and behavioral effect are essentially uncorrelated, that's the expected outcome, not an edge case.

  5. G4

    If it failed, which end broke?

    Always-on diagnostics, cheap enough that nobody is ever tempted to switch them off.

  6. G5

    The architecture — only once G3 passes

    The full system is the last step, not the first.

Status

Where this actually stands. Currently in active research.

Done

  • A test harness with positive and negative controls — it verifies the fitting math recovers a known answer exactly, and that it correctly fails to fit unrelated data. A harness that only tests the happy path measures nothing.
  • An earlier 61-layer merged model produced pure garbage output for months. That failure is traced end to end to a definitive root cause, and the fix is now applied, not just specified — the rebuilt splice has been run end to end and the merged model speaks correct, fluent English with no post-merge training at all. The fix went into a rebuilt pipeline on a new pair; the original broken artifact was never repaired, because its dense parent no longer survives as a standalone checkpoint.
  • Mechanism tests green, and no longer only on small models: two halves run in parallel, exchange information through a gated connection, and save and reload as a single artifact — now demonstrated end to end at full scale as well.
  • The re-merge above, actually built: a single small model, one set of weights standing in for both roles, now lands close to a separately-trained two-model baseline on the same measured tasks — within about a point on deploy accuracy, matching or slightly ahead on the harder judgment calls. Early, small-scale, and not a clean win against the bar we set going in — but a working merge, not just a specified one.
  • A cross-architecture merge at 8.1B, completed end to end: a 1.8B dense chat model and a 60-expert mixture-of-experts chat model — genuinely different architectures — spliced into one standard-format checkpoint that loads with stock transformers and no custom code. With no post-merge training at all it answers correctly and fluently, agrees with its dense parent on 63.7% of next tokens over held-out text, and hands off cleanly where the two halves meet: layer RMS 10.04 going in, 10.05 coming out.
  • Why that one worked where the 61-layer attempt produced garbage: parent choice, not architecture. The mixture-of-experts parent had been upcycled from the dense parent's own lineage — relatives, not strangers. The earlier stranger pairing needed a measured 294× scale correction at the seam and still produced nothing usable.
  • On the same 389-row deploy slice, the merged model scores 87.1% against 83.3% for the dedicated 3B orchestrator it replaces. Read that with the obvious caveat: 7B versus 3B confounds size with merging. We never trained a 7B on the orchestrator role alone — deliberately, because the comparison we care about is between the two artifacts you would actually ship.
  • Spurious rewrites — the model "correcting" an answer that was already right — fell from 17 to 9 between the two-model setup and the merged one. We are deliberately not publishing the matching miss-rate figures alongside it yet: the numbers we had did not survive a check against the run artifacts, and half a trade-off is not a result.
  • Several honest negative results, written up in full in the research log — including a seam-healing run we threw away. Distillation improved the metric it was optimising, KL 0.660 down to 0.384, while actual generation collapsed into a repetition loop. The metric said better, the model said worse, and the model wins. We shipped the raw merge and kept the healed weights only as a tagged record.

Not done

  • The target-scale model pair has not been run through the pipeline, and the causal audit of the latent bridge has never been attempted at scale.
  • The 8.1B merge is built and measured, not released. It sits in a private repository and inherits a research licence from its parents — research and personal use, no commercial use — so there is nothing to download today.
  • That merge is also not what runs the demo above. The two-role duplex and the dense-plus-MoE merge are separate tracks that our own notes have both called "one brain"; only the smaller duplex runs the exchange shown on this page.
  • The smaller merge above runs on ordinary chain-of-thought models and passes its correction through a system prompt, not the vector-level Bridge. The Bridge is still ahead.
  • Nothing here is deployed, and nothing here is in the CLI.

The things we learned getting here — including the results that argue against our own approach — are written up separately.

Read the research log
Upstream of this work

A model that can't think in latent space can't meaningfully send latent thought.

MultiThink is the between-model track. Neuralese is the inside-model track — whether a single model can carry reasoning forward as continuous internal state at all. That question gates this one in practice, which is why we're training it first.

Read about Neuralese
References

The published work this rests on.