Developers and a project lead review system states across several screens

Technology for production

Technology

The stack follows the system.

LatentDrift designs AI architectures for agent systems, direct and latent communication, data and evidence layers, and real-time interaction. We turn current research into practical decisions: how agents coordinate, what they may retain, how outputs are checked, and when a human decision is required. Models, frameworks and infrastructure follow access rights, failure modes and operating needs. This keeps technology choices subordinate to the system rather than the other way around.

View services View project patterns

Agent coordination

Working memory for agents.

In complex agent systems, the bottleneck is often state transfer rather than the individual model. The important questions are what one agent may share with another, what is needed only during a run, and what must remain available for later review. LatentDrift separates these layers and defines how research, planning and review agents exchange information.

Shared working memory

Agents work on tasks, intermediate results and internal state in separate workspaces. Read and write permissions determine which information may be shared. Conflicting intermediate results are reconciled before handoff. This avoids turning every internal step into a long text message without creating one shared, uncontrolled memory.

Example: in an agent workspace, a research agent collects source passages while a planning agent prepares the next steps. A review agent flags contradictions and presents the responsible team with the relevant results and open questions.

Direct and latent communication

Direct or latent channels can reduce latency and token use, for example when agents exchange embeddings, hidden states or KV caches. These internal representations are not portable across all models and can increase the risk of unintended data exposure. Direct channels therefore need clear boundaries, limited permissions and a readable record for critical handoffs.

Tests determine whether a direct channel provides a measurable benefit and where a readable summary or approval is still required.

Tracing internal state

Internal state that cannot be inspected directly needs explicit checkpoints. Run histories, state diffs, model versions, and token and latency limits show what changed between two steps. Rather than storing every internal state in full, a state diff records only the relevant change.

Run logs connect internal handoffs with short summaries, state changes, model versions, cost and review status.

Runtime and control

Agent systems need explicit runtime rules.

Roles and prompts are not enough. The runtime controls task assignment, system access, budgets, checks, retries and human approval. It defines which agent may act at each point and how a run can be stopped, inspected or repeated. Keeping these rules in versioned configuration makes them easier to change independently of prompt text.

Roles and priorities

Roles and responsibilities define who plans, executes and reviews. Priorities determine which tasks run first and when a run must wait or be handed to a person. These rules remain versioned and can be adjusted without rewriting the entire prompt logic.

Access and budgets

Each agent receives explicit permissions for tools and data, along with cost and runtime limits. It may be restricted to reading, preparing suggestions or writing approved changes. Permissions to change an operational system are more restrictive than permissions to research or prepare a plan.

Example: agents may gather information and prepare options, but changes to operational systems still require approval.

Checks, stop conditions and recovery

Critical intermediate results are checked before the next step runs. Defined reset points limit the effect of errors. Actions that cannot be repeated safely are blocked or sent for approval. A retry must not create duplicate records or repeat an action with unintended consequences.

This defines when a run continues, returns to a known state or is handed to a person.

Model routing without a fixed provider

Routing rules determine which models are suitable for a run. The decision depends on access and usage permissions, context, cost, latency and the required output quality. A task may use an API, a self-hosted open-weight model, batch processing or streaming. Model, prompt and tool versions remain linked to each result, so changes stay visible and reviewable instead of going unnoticed.

Traceability

Evidence belongs in the system.

Traceability should be part of the architecture, not added at the end. The evidence layer connects outputs with data sources, the model run, review status and access rights. It shows how an output was produced and what information is needed for use, approval or review. In public-sector and regulated settings, this helps distinguish plausible answers from results that can be checked against their sources and system history.

Data provenance and evidence graph

For relevant outputs, the system records the request, data version, source passage, model version, prompt and tool configuration, important intermediate states and approvals. The evidence graph links this information to the result and records its provenance. That supports corrections by subject-matter experts, technical review and later model or data changes.

Example: in a complex document review, each recommendation links to its source passage, data version, applied rule, model run and review status. Subject-matter and technical teams use the same evidence for different review tasks.

Role-specific explanations

End users, subject-matter experts and technical teams need different explanations. For each role, we define which sources, uncertainties, limits and system steps must be visible. The interface shows the information relevant to the decision or review rather than exposing every technical detail.

Evaluation and approval

Test cases, error classes and approval rules determine how far the system may act on its own. In RAG systems, retrieval, generated answers, source grounding, safety, cost and latency are evaluated separately. A plausible explanation also needs review because it may misrepresent the actual system run.

The evaluation shows which errors occur, whether the sources support the results and where human approval remains necessary.

Real-time interaction

Real-time systems need explicit states.

The critical logic of a live system sits below the interface: streaming, turn-taking, interruptions, latency, context and approvals. LatentDrift models these interactions as explicit states: listening, waiting, clarifying, speaking, interruption and handoff. An avatar is only one possible interface. The same logic can support a dashboard, a voice interface or a review workflow. States, latency limits and human handoff are defined and tested independently of the interface.

Example: a training or service interface responds to speech, interruptions and pauses. When confidence is low, the system asks for clarification. The output may appear through an avatar, on a dashboard or as voice only, with human handoff available when needed.

Technical reference points

Provenance, security and accessibility.

We use established specifications and guidance as reference points for provenance, application security and accessible interfaces. How they apply in practice depends on the system and its context.

Provenance and evidence

W3C PROV describes models and formats for connecting sources, data versions, processing steps and the people or systems involved in a provenance record.

W3C PROV OverviewExternal source
See related project patterns