# ThruWire Full Site Context



ThruWire connects humans and agents to a shared model of the work, enabling coordination and iteration without being tied to any single agent, harness, or factory.



Canonical website: https://thruwire.ai

Sitemap: https://thruwire.ai/sitemap.xml

Agent manifest: https://thruwire.ai/agents.json

LinkedIn: https://www.linkedin.com/company/thruwire



# ThruWire

ThruWire connects humans and agents to a shared model of the work, enabling coordination and iteration without being tied to any single agent, harness, or factory.

## 1. The definition of the work is scattered

Today, the definition of the work is scattered across context, skills, workflows, code, and the people operating them.

Every change forces people and agents to find the rules, history, and decisions spread across the stack.

## 2. Separate the work from its execution

ThruWire separates the work from the people, models, and tools that execute it, so the work can keep moving even as the workers change.

The work becomes its own system of record and survives improvements in models, harnesses, and tools.

## 3. Make the state of the work explicit

ThruWire makes the important state of the work explicit, while leaving agents free to reason, delegate, use tools, and choose their own path between checkpoints.

A directed acyclic graph (DAG) connects the work and its dependencies. Checkpoints create deterministic boundaries around nondeterministic execution, with receipts and provenance for the artifacts produced.

## 4. Carry context forward

At every checkpoint, the artifacts produced are connected to the context that produced them, so each result updates the shared model and becomes context for what happens next.

Goals, methods, skills, policies, and prior work flow into the work. The artifact and its receipt carry the result forward.

## 5. Work independently and coordinate

Humans and agents can enter anywhere, work independently, and still coordinate through the same evolving model of the work.

They can review, correct, branch, and hand off the work without reconstructing the whole job inside one worker's context.

## 6. Better workers, persistent work

ThruWire is building for a world where workers improve rapidly with every advance in models and harnesses, while the organization's work remains.

Replace the worker without replacing the work, and put better agents to work on the context and evidence already in place.

## 7. The product

ThruWire is an AI checkpoint system that connects humans and agents to a shared model of the work, enabling coordination and iteration without being tied to any single agent, harness, or factory.

A checkpoint steers a worker toward a required artifact and provides proof that the work was done. Between checkpoints, the worker chooses how to reason, delegate, and use tools.

## Start with your work

Connect an agent, or work with us to model your first job.

- [Connect your agent](https://thruwire.ai/connect.md)
- [For SaaS companies](https://thruwire.ai/for/saas.md)
- [Explore the work primitives](https://thruwire.ai/primitives.md)
- [Work with our advisory team](https://thruwire.ai/work-with-us.md)
- [Assess whether ThruWire fits your work](https://thruwire.ai/right-for-me.md)




## Site Pages

- For SaaS companies: https://thruwire.ai/for/saas - A persistent model of customer work that can move across the agents and AI tools they choose.

- Home: https://thruwire.ai/ - An AI checkpoint system connecting humans and agents to a shared model of the work.

- Viewpoint: https://thruwire.ai/viewpoint - Why AI work needs explicit checkpoints, inspectable evidence, and freedom between them.

- Research: https://thruwire.ai/research - Research on AI checkpoints, workplanes, artifacts, evidence, execution lineage, and improving shared work.

- Technology: https://thruwire.ai/technology - Working software for shared checkpoints, typed artifacts, exact relationships, and independent execution.

- Product Examples: https://thruwire.ai/use-cases - How shared checkpoints can guide customer work across several kinds of SaaS products.

- Workplane Primitives: https://thruwire.ai/primitives - Goals, context, requirements, artifacts, evidence, dependencies, and continuation.

- Comparisons: https://thruwire.ai/vs - Which layer LangGraph, CrewAI, Claude Agent SDK, long context, and ThruWire solve in a customer-facing AI product.

- Team: https://thruwire.ai/team - Company and team background.

- Advisory: https://thruwire.ai/work-with-us - Turn your way of working into checkpoints, inspectable evidence, and working software.

- Blog: https://thruwire.ai/blog - Research and practical notes on AI checkpoints, shared work, artifacts, evidence, and execution independence.

- About: https://thruwire.ai/about - Company background, operating principles, and product focus.

- Contact: https://thruwire.ai/contact - Verified contact channels for partnerships, support, privacy, and legal questions.

- Developers: https://thruwire.ai/developers - API, MCP, authentication, versioning, and safe integration guidance.

- Get Started: https://thruwire.ai/connect - Client-specific setup for Claude Code, Codex, Cursor, ChatGPT, Claude, Grok, and other MCP clients.



## Blog: In the Token Stream: Architectural Options for Monetizing AI
URL: https://thruwire.ai/blog/in-the-token-stream-architectural-options-for-monetizing-ai
Date: 2026-09-10
Description: AI companies can tie revenue to inference by owning the model path, agent runtime, or integration layer, but each added toll must justify its cost.
Tags: AI economics, Product strategy, Agent infrastructure
Model providers have a business model tied directly to AI usage. As customers use more inference, they make more money. For those of us building on top of those models, that relationship is not always obvious. But look across the AI landscape and you’ll see legions of startups designing their architectures to create similar economics. Here’s how they’re doing it.

You can think of this as getting in the token stream. The most literal version is reselling inference and putting a margin on every token, but there are many ways to tie your revenue to inference without becoming a model provider yourself.

To make this work, you have to decide where your product sits relative to the model, the agent runtime, and the rest of the application stack. You can own the path through which inference runs, own the runtime that creates it, or integrate into a stack controlled by someone else. Here are the primary options.

 Put Inference Inside the Product

The most common architecture is also the simplest. You build an application or agent that uses inference, and customers pay to use the product. Anyone in the AI space is familiar with this approach.

Cursor and Devin are straightforward examples of this architecture. Customers pay for an AI product that performs work, while the product consumes model inference underneath.

The customer might pay per seat, per task, through usage-based pricing, or as part of a subscription. In practice, heavier users often move into higher-priced plans or usage tiers, so revenue can still rise with the amount of inference the product supports. Underneath that pricing, the product calls models to do the work. More customer usage creates more inference, and the inference becomes part of the cost of delivering the product.

The customer does not need to know how many tokens or model calls were involved. They are paying for the product and the work it performs.

This is the baseline architecture. It creates a direct way to get into the token stream without selling infrastructure at all. The application owns the customer relationship and decides how much inference to use to deliver the product. If a task can be completed with a cheaper model, fewer tokens, better caching, or a more efficient agent loop, the cost of delivering the same product goes down.

 Put a Gateway in Front of the Models

One of the most direct architectures is to put a gateway in front of the model provider, and many are trying to do this. Instead of an application calling OpenAI, Anthropic, Google, or another provider directly, it calls the gateway, which forwards the request and returns the result.

Portkey is a direct example of this architecture. Its AI Gateway sits between applications and model providers and provides routing, fallbacks, load balancing, retries, caching, and other controls across providers.

Cloudflare AI Gateway takes a similar position in the request path. Applications route AI requests through Cloudflare, which can then provide analytics, caching, rate limiting, retries, model fallback, and other controls without owning the underlying models.

A thin version of this architecture allows customers to bring their own model credentials and pay the gateway for usage. Alternatively, the gateway can resell the underlying inference and include its margin in the price.

A fatter version may be the enterprise policy gateway. Microsoft Azure API Management’s AI gateway applies policy controls to AI endpoints and supports token limits, load balancing, logging, content-safety policies, and governance across model backends.

An enterprise can make a gateway like this the controlled route between its applications and external models. The gateway is then doing more than transporting tokens. It’s an enforcement point for enterprise inference, while its usage still grows naturally as more inference crosses that boundary.

 Own the Runtime That Creates the Inference

This architecture is one level above individual model calls. Instead of sitting between the application and the model, the infrastructure company owns the runtime that decides when and how those calls happen. The opportunity for margin lives inside those decisions.

Amazon Bedrock AgentCore Runtime is one example of this. Developers deploy agents into the runtime, and AgentCore provides the execution environment in which those agents run rather than simply proxying an individual model request.

LangSmith Deployment provides another version through managed infrastructure for deploying and running agents built around LangGraph. The runtime is responsible for executing longer-lived agent work around those model calls.

The customer asks the runtime to execute an agent, complete a task, or run a workflow, and the runtime determines how to do the work. One request might require a single model call while another requires twenty, with the runtime choosing models and executing tools along the way. It owns the execution until the work is complete.

This gives the runtime provider an opportunity to control more of the economics. The customer pays for execution performed by the runtime, while the underlying inference may be billed separately or bundled into the runtime’s higher-level unit of work.

Bundling inference opens up another source of margin. Better model selection, caching, compilation, and other execution improvements can lower the cost of producing the same work. A runtime that sells a higher-level unit of work can therefore improve its margin as the underlying execution gets more efficient.

 Insert Middleware Into Someone Else’s Agent Loop

Owning the runtime is powerful, but it is not required. Agent frameworks are exposing more places for outside infrastructure to participate in their loops.

An agent runtime natively calls models, executes tools, updates state, and decides what happens next. Frameworks such as the OpenAI Agents SDK, Google ADK, Semantic Kernel, and LangChain middleware now expose callbacks, filters, wrappers, and middleware around these operations, giving outside products explicit places to integrate into the loop.

OpenAI, for example, exposes lifecycle hooks around model and tool calls, while LangChain includes hooks such as 'before model', 'after model', 'wrap model call,' and 'wrap tool call.'

That allows an external product to get installed into a loop owned by somebody else. A security product might inspect a model request before it is sent. Or an evaluation system might judge an output before the agent continues. You could build an external policy system that decides whether a proposed tool call is allowed.

Some frameworks now expose these controls quite explicitly. OpenAI’s Agents SDK has a 'callModelInputFilter' that can modify model input immediately before invocation, Semantic Kernel has function and prompt filters, and Vercel’s AI SDK language-model middleware can wrap a model and intercept or modify calls before they reach the underlying provider.

The economics can still track inference closely. A third party can participate in thousands of agent loops without building the harness that owns any of them, provided the frameworks make enough of the loop extensible.

 Put Inference Behind a Tool

We often say the model is the brain and tools are the hands, with the model doing the thinking while tools perform deterministic actions in the outside world. That analogy has been slowly breaking down with what some might call “hands with brains.”

Granola’s MCP server provides a good example. It exposes conventional tools for listing meetings, retrieving meeting details, and fetching transcripts, but it also exposes query granola meetings, which Granola describes as “Chat with Granola” and uses to answer questions based on meeting history.

The outer agent may be Claude, ChatGPT, Codex, or another MCP client. It decides that it needs something from Granola, calls the tool, and Granola performs whatever retrieval and inference are required before returning the result.

Firecrawl’s MCP server provides an even clearer version of the architecture. Its 'firecrawl agent' tool can take a research request, search and navigate the web, read pages, and assemble structured results across multiple sources.

In some ways, the original brains-and-hands analogy was convenient for frontier model providers. It presented an architecture where their model was the only intelligent thing in the block diagram, and everything around it was a capability to be called. But the reality is that intelligence is increasingly distributed.

The outer model is no longer simply using a hand. It is delegating part of the thinking to another system that has its own models, context, data, and execution logic. MCP gives those services a standard way to present themselves to many different agent environments.

This architecture also has different economics from middleware. The vendor does not need to touch every model call made by the outer agent because it creates its own inference whenever the agent decides to delegate work to it.

 The Economics Are Still Unclear

More value is being added on top of models than ever before. That value should rightly be monetized by the companies building it.

We are still figuring out what those economics should look like. I expect we will discover even more architectures for tying revenue to AI usage as the stack matures.

There is also a risk in how many companies are chasing the same pool of inference spend. If every layer wants to add its own margin to the token stream, the combined markup can get large quickly.

Getting into the token stream is only part of the job. Your product also has to add enough value to justify the additional spend. Customers may accept a substantial premium for infrastructure that materially improves the product, but there is probably a limit to how many tolls they will tolerate on the way to the model.

## Blog: Designing for Mistrust: Architectures for Building Around Untrusted AI
URL: https://thruwire.ai/blog/designing-for-mistrust-architectures-for-building-around-untrusted-ai
Date: 2026-09-09
Description: Trustworthy AI systems keep consequential authority in deterministic control planes, sandboxes, policies, and independent verification around the model.
Tags: AI safety, Agent architecture, Trust
Meta just released Muse, a personal agent running inside a dedicated Secure VM. Beyond the VM itself, Meta carefully built an entire security-sensitive control plane around one premise: we can't fully trust the models. It assumes that the model and harness can and will misbehave at times, and puts the authority to do consequential things somewhere else.

This is no longer just a theoretical threat model. In July, models being evaluated by OpenAI circumvented controls intended to isolate them from the internet, exploited vulnerabilities in shared infrastructure, and accessed Hugging Face systems.

These are very different situations, but they point toward the same architectural question: How should you build an AI system if you assume the model itself could become a threat?

For a long time, one of the implicit assumptions in AI application architecture was that the model was trying to do roughly what you asked. It was your “helpful assistant” eager to please. It might hallucinate, misunderstand a prompt, or call the wrong tool, but the model itself was still sitting somewhere inside the trusted part of the application.

There are now enough implementations in the wild to see several common patterns for designing systems where the model and harness are instead untrusted.

 Put the Model Inside the Sandbox

The most common approach is to treat model-driven execution the same way we treat untrusted code. We let the model do what it needs to (generate code, run commands, manipulate files, install packages, use tools), but it does those things inside an environment where it's deliberately constrained.

Muse puts its workspace, files, tools, and more importantly, the harness itself inside an isolated runtime environment. Like any VM, root inside that VM does not mean root on the host. It lets the agent own its own filesystem and virtual network interface, but sensitive services remain outside.

NVIDIA’s Secure Agent Workspace architecture uses a similar pattern: the managed workspace is the execution and policy boundary, and the agent cannot expand its own authority. AWS likewise recommends secure execution environments for agents that can execute code or interact with websites.

In short, a sandbox allows the model to have substantial freedom inside the box (i.e. ship the computer) without receiving equivalent freedom over the rest of your system.

 Move Authority Outside the Agent

Sandboxing limits what an agent can reach locally, but it does not answer a harder question: what happens when the agent legitimately needs to do something in the outside world? One emerging pattern is to create a separate authority layer.

Muse has Sentinel, a host-side component that sits outside the agent runtime. Muse can propose a connector action or network request, but Sentinel is the component that actually decides whether it is allowed. It evaluates connector policies and network requests independently of the agent.

AWS is moving toward a similar architecture with AgentCore Gateway and AgentCore Policy. Agents request tool calls through a centralized intermediary, while authorization policies can inspect the action and even individual parameters. AWS explicitly recommends enforcing authorization outside the agent so a compromised agent cannot simply decide that it has permission.

Microsoft makes essentially the same recommendation that authorization should happen on every action rather than only when the agent starts a session. High-impact actions can then require separate approval or temporary escalation.

The main goal is for the model to become a proposer rather than an authority. It can decide that sending an email is the appropriate next action, but a different, more privileged component determines whether this agent, acting for this user on this task, is actually allowed to send that email to that recipient.

 Give the Agent Capabilities, Not Credentials

If we aren't going to fully trust our agents and models, we certainly should not give them the keys to the kingdom. An agent that can see a credential can potentially leak it. The solution is to let the agent use authenticated systems without ever possessing the underlying secrets. This is not a new concept for cloud infra in general, but it's increasingly being extended to agents.

Muse’s hatch-authd stores real credentials outside the agent runtime. The agent sees surrogate credentials instead. When an approved network request crosses the boundary, Sentinel obtains the actual credential and swaps it just before the request leaves the system. The model never needs to see it.

NVIDIA’s Secure Agent Workspace uses almost exactly the same approach. A credential proxy rewrites authorization headers at egress. Postman’s AI Engineer similarly keeps credentials at arm's length so the agent can request authenticated actions without receiving the credentials themselves.

Google applies the same idea to browser passwords. Gemini’s agentic Chrome capabilities can use Google Password Manager, but the model does not receive direct access to stored passwords.

Clearly, this is stronger than telling the model not to reveal a secret because there is no secret available for it to reveal.

 Treat Network Egress as a Privileged Action

An agent that can read private data and make arbitrary network requests has most of what it needs to leak that data and exfiltrate it. Several architectures therefore put a hard boundary around egress.

The architectural idea is to treat reading data and transmitting data as separate capabilities. An agent might legitimately need access to your email, source code, CRM, or internal documents, but that does not mean every process handling that information should also be able to open an arbitrary connection to the internet.

Muse routes runtime traffic through Sentinel. The authorization layer can inspect the hostname, resolved IP, port, protocol, HTTP method, path, and decoded request before allowing it to leave the VM.

Agent Browser supports domain allowlists that apply not only to top-level navigation but also to subresources, WebSockets, EventSource, and sendBeacon.

Muse goes even further with what Meta calls tainted egress. Processes begin clean and become tainted when they read user data. Once a process is tainted, some automatic egress privileges disappear. So it tracks whether the agent’s processes have accessed a user’s private data, and extra rules kick in. In this scenario, the request goes to a stricter approval path, perhaps requiring human approval. Meta implements this using eBPF and Linux Security Module hooks.

There are likely even more creative ways we can try to observe agent behavior and restrict network actions. These types of capabilities are essential if we are going to continue giving agents more personal and confidential information. Without them, it would be all too easy to convince agents to hand over private data.

 Give the Model Only the Data It Needs

Another approach attacks the problem earlier, at the data level. Rather than letting the model see everything and controlling what it can do afterward, the pattern is to reduce what enters its context in the first place.

Google is doing this in agentic Chrome with Agent Origin Sets. For a given task, the system maintains a constrained set of web origins the agent can read or interact with. A separate gating function determines whether additional origins are relevant, and the planning model can't simply add arbitrary sites itself. Google can also distinguish between origins that are readable and origins that are writable.

Suppose an agent is booking a flight. The browser may contain logged-in sessions for email, banking, healthcare, internal company tools, and dozens of other sites. A human browsing the web has access to all of them, but the agent does not necessarily need the same authority. Instead, the system can construct a task-specific view of the world.

This principle extends beyond browsers. Retrieval systems can filter records before they reach the model, tools can return only authorized fields, and memory can be partitioned by user, project, or task. AWS explicitly recommends enforcing data authorization outside the agent so prompt injection cannot cause the model to retrieve information the user was never authorized to access.

This seems obvious because it all comes back to just good data engineering principles, but it's just as important as the other mitigations. Controlling context can be an authorization mechanism of its own.

 Use Another Higher-Trust Model

Not every useful security decision can be expressed as a deterministic rule. Questions like whether an action was actually implied by what the user asked, whether a navigation makes sense for the task, or whether a tool call is suspicious given the surrounding context require judgment.

That has produced another architecture. If you put a second model between the main model and the action, you can use a specialized agent to mitigate some of the risk. Google’s agentic Chrome architecture includes a User Alignment Critic. The planner proposes an action, then a separate model evaluates whether that action serves the user’s stated goal.

Importantly, the critic is isolated from the untrusted page content that influenced the planner. It sees metadata about the proposed action, but not the raw potentially malicious web content.

Muse uses a related pattern. hatch-safety runs outside the runtime container and independently inspects inference traffic using models and classifiers for threats including prompt injection. Because those components live outside the runtime, a compromised runtime cannot simply disable them.

This resembles the runtime LLM-as-judge pattern I've written about, but with a security constraint added. The judge should not necessarily have the same context as the model it is judging. If both models ingest the same malicious instructions, adding a second model may just give the attacker two models to manipulate.

A higher-trust model can instead receive a smaller, sanitized representation of the proposed action and answer a narrower question. The asymmetry between the two models is what makes the architecture useful.

 Scope Authority to the Task

Traditional applications often authenticate once and then carry substantial authority for the rest of a session. Or that authentication represents the human and not the agent. That model gets dangerous when the thing holding the session is autonomous.

Microsoft recommends giving agents their own identities, restricting them to task-specific roles, using short-lived tokens, and applying just-in-time elevation rather than broad standing permissions. AWS similarly recommends temporary, scoped credentials and a traceable authorization chain between the user and every agent action.

⁠Keycard takes this idea further by giving agents their own identity and scoped permissions, so access can be granted to the agent for a particular resource or action rather than inheriting the human’s credentials and authority.

Agent permissions are starting to look less like application permissions and more like leases. An agent gets enough authority to perform a specific piece of work, but then that authority expires.

 Selectively Put Humans at the Boundaries

Human approval (aka human in the loop) has been the standard for a while because it is the obvious answer to model uncertainty. But requiring approval for everything eliminates much of the reason to use an agent. The more useful architecture is selective approval.

Google’s browser agent can work autonomously through much of a task but asks for confirmation around sensitive sites, password use, purchases, payments, sending messages, and other consequential actions. Microsoft similarly recommends explicit approval for high-impact operations such as sends, deletes, purchases, deployments, and permission changes.

The key design question is not whether the system has a human in the loop and more about where the human sits in the loop. Good checkpoints are when authority expands, sensitive information crosses a boundary, money moves, or permissions change. This is something that needs to get designed into your application stack or harness from the bottom up.

 Better Models Need Better Machinery

Put these patterns together and a fairly coherent architecture emerges. We need a whole bunch of mostly deterministic (but sometimes probabilistic) machinery around our models if we are going to trust our AI-powered app. We may not fully trust the models, but that doesn't mean we can't trust what we build on top of them.

The common idea is that increasingly capable models do not require us to give models increasingly broad authority. In fact, the opposite is true. As these models get more powerful, the bar is higher to design a system around them that we can trust.

It's only going to get more complex from here. Attention is now focused on the systems we are building around these models and whether they can keep up with the rapid innovation in the models themselves.

## Blog: LLM-as-Judge Architectures: Putting Evals Into Your Agent Runtime
URL: https://thruwire.ai/blog/llm-as-judge-architectures-putting-evals-into-your-agent-runtime
Date: 2026-09-08
Description: LLM judges are moving from offline evaluation into runtime control flow, creating powerful new feedback loops and new production risks.
Tags: LLM-as-judge, Evaluation, Agent runtime
LLM-as-judge is already a common part of building AI applications. Run your application against a dataset, have another model grade the results, and use the scores to figure out whether the latest version got better or worse.

Now, LLM judges are moving into the runtime itself, where they evaluate work as it happens. Increasingly, we are seeing products put LLM judges directly inside the agent loop rather than using them as an offline tool. The judge is no longer just evaluating the application. It’s participating in its control flow.

If your application does not have some form of LLM judge yet, there is a good chance it eventually will. Traditional deterministic software gives us plenty of ways to check whether something worked. We can use schema validation, type checks, assertions, and exact comparisons.

AI applications, on the other hand, produce work that often cannot be checked that way. Is the work complete enough to move on? Was the answer actually useful? Did the research support the conclusion? Should I let this generated artifact into my company brain? If a human would normally have to look at the result and make a judgment call, an LLM judge gives the application a way to make some of those judgments itself.

The model evaluation world has spent years figuring out how to get models to reliably judge other models. But it turns out that many of these techniques can also be useful one layer higher, as part of the architecture of the applications we are building with those models.

There are a ton of ways to build that layer. Here are some common patterns for LLM-as-judge that you should consider adopting as part of your product’s application architecture.

 Start With Another Model

This is the canonical LLM-as-judge approach. One model does the work and another model judges it. The judge gets some combination of the original request, the generated result, a rubric, reference material, and perhaps an expected answer. It returns a score, label, or explanation. Systems such as LangSmith, Phoenix, and DeepEval make this a normal part of application evaluation.

One upfront choice you have to make with this approach: which model should be the judge? If cost were no concern, the obvious answer is to use your strongest frontier model. But the judge does not necessarily need to be smarter in the general sense. Judging a narrow property can be much easier than doing the original work. A model that could not produce a great research report may still be perfectly capable of deciding whether every claim is supported by the supplied evidence.

This has led to specialized judge models such as Prometheus, as well as smaller evaluators trained or distilled for particular kinds of judgment. Galileo has also built specialized evaluation models intended to make production-scale evaluation cheaper than repeatedly calling a large frontier model.

For high-volume applications, the economics could matter quite a bit. Because cost is more noticeable when you're running these evals constantly at application runtime, a specialized model that is designed to be cheap for this use case is a good option.

 Break the Judgment Apart

A large amount of LLM judging boils down to one main question: was this output good? An answer can be correct but incomplete. It can be well written but unsupported by the source material. Or an agent can arrive at the right result after taking actions it should never have taken.

One upgrade to your judging architecture is to break the judgment into a set of smaller decisions. One judge checks whether the answer addresses the request. Another checks whether its claims are supported by the supplied evidence. Another looks at whether the agent completed the required work.

G-Eval took an early step in this direction by having the model generate evaluation steps from the criteria before producing its score. DeepEval pushes the idea further with DAG-based evaluation, where individual LLM judgments can sit inside a larger deterministic decision graph.

In other words, instead of asking one model to make a giant fuzzy decision, break that decision into smaller judgments with well crafted smaller prompts and put structure around how they are combined.

 Compare Instead of Score

Models are not always particularly good at telling you that something deserves a 7 rather than an 8. They can be much better at deciding which of two things is better. Pairwise judging is an approach that takes advantage of this. If you give the judge two outputs produced from the same input, you can ask which one better satisfies the criteria.

Systems such as LangSmith and DeepEval support this directly, including techniques such as randomizing which answer appears first to reduce position bias.

This is commonly useful for application regression testing. However, the same pattern could move into runtime architectures. An agent could generate several plans and use a judge to choose between them. Alternatively, two agents could independently perform a piece of analysis and another model could compare the results. In short, a proposed action could be compared against an alternative before the system commits to it and proceeds.

 Judge the Work Instead of the Answer

For a chatbot, judging the final response is fine. For an agent doing twenty minutes of work, it tells you much less. The final result might look completely reasonable even though the agent retrieved the wrong documents or ignored an important source. It may have called the wrong tool or wandered through a series of unnecessary steps before getting lucky at the end.

Application evaluation systems are already moving further into the trace. Phoenix has evaluators for retrieval relevance, tool selection, tool invocation, tool responses, and overall agent performance. LangSmith can apply evaluators to individual runs as well as larger traces and threads.

Generalizing this to agents in general and especially long-running agents, the takeaway is to judge the pieces of work where the important decisions are actually being made.

For example, a research agent might have its source selection judged before synthesis. Or a coding agent might have its proposed approach judged before implementation. An operational agent might have the evidence supporting an action judged before that action is taken. Integrating this into your application means scattering judges throughout the runtime to form key checkpoints.

 Use More Than One Judge

One uncomfortable fact about LLM-as-judge is that the judge is still an LLM. It can make mistakes for all the same reasons the model doing the work can. One way to handle this is to stop treating a single judge as authoritative.

Research on model evaluation has explored panels of judges, evaluator personas, voting, aggregation, and debate between evaluators. MAJ-EVAL, for example, creates multiple evaluator agents representing different dimensions of the evaluation and lets them deliberate over the result.

There is not a lot of evidence suggesting applications have adopted this approach in production yet. But the mechanism is interesting above the model layer for a different reason. Even if you don't care whether three judges vote 2–1 that an answer is good, you may care enormously that they disagree.

Agreement between independent judgments can increase confidence whereas disagreement could be a reason to retry with a stronger model, collect more evidence, or send the work to a human. You could imagine building a retry or escalation mechanism in your app based on this principle.

 Judge the Judge

Once a judge can affect what happens in an application, its reliability matters a lot. And things can go wrong. For example, LLM judges have been known to have biases such as changing their decisions based on which answer is presented first. Or they can favor certain writing styles. They might struggle when the outputs being evaluated are close in quality. Sometimes a model might even have preferences for outputs that resemble its own.

The model-evaluation world has built benchmarks specifically to measure these problems. LLMBar tests whether judges can distinguish instruction-following responses under difficult conditions. JudgeBench contains difficult response pairs with objective preferences. RewardBench evaluates reward models across challenging preference tasks and can also be used to evaluate generative LLM judges.

Anthropic’s Bloom provides another useful pattern. Anthropic evaluated a collection of candidate judge models against human-labeled transcripts before choosing its judge. The resulting evaluation pipeline also includes a meta-judge that looks across the broader evaluation results.

Application builders can use the simpler version of this idea today: periodically compare the judge against humans. If a judgment controls something important, collect examples of those decisions and have people independently evaluate them. Measure where the judge disagrees, and change the rubric, model, context, or decision boundary when the disagreement is unacceptable.

 Put Determinism Around the Judge

There is a temptation to turn every application decision into another LLM judgment and to throw more agents at more agents. That dismisses one of the biggest advantages we have at the application layer that, unlike the model, we control the surrounding system.

If something can be checked deterministically, check it deterministically with tests or schema checks or even a database check. You could also consider writing a policy engine for this type of validation.

OpenAI’s grader architecture reflects this separation. It supports model-based graders alongside deterministic graders such as string checks and Python code, and multiple graders can be combined into a larger evaluation.

The same pattern makes sense inside applications. An LLM might judge whether the evidence is sufficient, whether a recommendation is well supported, or whether an agent appears to have completed the task. Deterministic code can decide what combination of those judgments and other facts is required before the workflow moves forward.

This sort of combined deterministic / non-deterministic logic is one of the biggest opportunities available at the application layer. The better we get at drawing that boundary, the more value we are adding above and beyond the models themselves.

 Implication: Put the Judge in the Loop

Many of the above patterns require that we move LLM judges inside the agent loops and put them in the critical path for our applications. That's in contrast to the many LLM judges today that sit outside the application execution path.

It means that judgment informs control flow. It can cause the application to continue, retry, route to another model, gather more information, or escalate to a person. This is a much bigger role for LLM-as-judge. The judge is no longer just telling you whether the application worked yesterday. It is helping determine what the application does next.

It also means mistakes from the judge become application failures. A judge that is slightly noisy in an offline eval may be annoying. The same judge sitting in front of every important action can create loops, block good work, approve bad work, and add latency to every execution.

We will need to invest in these architectures and mature these patterns before we feel comfortable putting LLM judges on the critical path in production. But there are huge opportunities for innovation here.

## Blog: Multiplayer AI: Competing Architectures for Human-Agent Collaboration
URL: https://thruwire.ai/blog/multiplayer-ai-competing-architectures-for-human-agent-collaboration
Date: 2026-09-07
Description: Multiplayer AI can share conversations, sessions, harnesses, graphs, memory, or work; the key decision is which state must remain durable.
Tags: Multi-agent systems, Human-agent collaboration, Shared state
Multiplayer AI is starting to show up everywhere. Anthropic is bringing Claude into shared Slack channels with Claude Tag. Dust is building shared spaces for humans and agents. Asana is putting agents into its Work Graph. Superconductor and AQ are making agent sessions and harnesses collaborative, while Cognee is building shared memory across agents.

There are now enough different approaches in the wild that it’s possible to ask a more useful question: when we make AI multiplayer, what exactly needs to be shared?

The answers are surprisingly different. Depending on the product, the multiplayer layer might be the conversation, the agent session, the harness, the work graph, the workspace, memory, or the work itself. Coding systems sometimes go in the opposite direction, giving every agent an isolated environment and only bringing their work together afterward.

These are more than different product interfaces. They represent different architectural approaches for how humans and agents should work together, which state belongs to execution and which belongs to the collaboration, where shared state should live, and what needs to persist as workers come and go.

There is no agreement yet on the right human-agent architecture, and unsurprisingly each company’s answer often starts from the part of the stack it already owns.

Looking across the products being built today, seven architectural patterns for multiplayer AI stand out. These patterns are not mutually exclusive. A single product can combine several of them.

 Share the conversation

The most straightforward approach is to put the agent somewhere everyone can talk to it. Claude Tag does this inside Slack. Instead of every employee having a private Claude conversation, Claude can participate in a shared channel. It sees the surrounding channel context, multiple people can interact with it, and its work is visible to the team.

Microsoft is taking a similar approach with agents in Teams. Its Channel Agent belongs to a particular channel and can participate in the conversations, meetings, and work happening around it. In both cases, the existing communication system becomes the multiplayer layer.

Architecturally, the conversation itself is the shared state. The underlying agent execution can remain mostly unchanged, while the context stream surrounding it is what's shared and persisted. For a lot of collaborative work, this architecture is enough.

 Share the agent session

Superconductor goes a layer deeper by sharing the actual agent session. Multiple people can participate in the same running agent environment, see what is happening, steer the work, review it, and continue from the same state.

AQ brings a similar idea into the coding harness. A task gets an isolated worktree, while the terminal, editor, preview, and agent environment around it can be shared with the team. HumanLayer is exploring related territory around coding-agent sessions, plans, diffs, and tasks.

In these architectures, the collaboration is centered around the agent and its execution environment, making the session or harness itself the multiplayer layer. In other words, the shared state moves down the stack to the live execution state itself.

There is a meaningful difference between this and sharing a conversation. The team isn’t just communicating with the same agent. It is sharing more of the environment in which the agent is actually doing the work.

 Share the work graph

Asana and Linear start from a different place because they already have multiplayer systems. Their shared objects are already projects, tasks, issues, owners, dependencies, comments, status, and approvals. They can make agents participants in that existing model instead of inventing a new collaboration layer.

Asana is particularly explicit about this. Its AI Teammates operate against the same Work Graph as people. Agents work against shared organizational context and participate in workflows with visibility, governance, and checkpoints.

Linear has been moving agents into its issue model. A human can remain accountable for an issue while delegating work to an agent. Patchwork pushes the idea further toward agents as visible coworkers that can own tasks, have conversations, work on schedules, and pull humans in when needed.

In these architectures, the existing domain model remains the system of record. Agents are just new actors operating on these same durable objects and relationships as humans.

One key property of this approach: the agent doesn’t need to be durable. It can show up, read the task and context, do the work, and disappear. The project, task, and dependencies are still there when a different agent or human connects.

 Create a shared workspace

Dust is tackling the problem by creating a new container for the team. Its Pods combine people, agents, conversations, files, tasks, and context around a shared goal. Agents can work against the state of the Pod, create conversations and tasks, work with files, and hand work between participants.

QM is another interesting version of this approach. It gives people and shared rooms durable environments containing memory, files, credentials, permissions, scheduled work, apps, and sandboxes, while allowing different agent harnesses to operate against them.

Here the architecture is more about creating a new shared-state container around the collaboration itself. A shared workspace can bring several necessary parts of multiplayer work together in one place and make it all available to agents.

There is a good argument for building a workspace architecture because human-agent teams have requirements our existing collaboration products weren’t designed for.

 Share the memory

Cognee has a different answer. One developer can work with Claude Code while another uses Codex. Their agents remain separate, but useful context from docs, chats, tickets, code, and agent runs becomes a shared company brain, allowing one person’s agent to benefit from something another person’s agent learned earlier.

Several agent-memory companies are approaching adjacent versions of this problem, including Mem0, Zep, and Letta. They differ in how they represent and retrieve memory, but the common architectural approach is separating memory from the individual worker so that agents can remain independent without starting from scratch every time.

Shared memory gives otherwise independent agents a common layer of accumulated knowledge. The agents can use different models, harnesses, and interfaces while still benefiting from what the rest of the team has already learned.

The system of record here is not the execution or even necessarily the work. It is the accumulated context about the work, persisted independently of the workers that produced it.

 Keep execution isolated

Coding agents also demonstrate an almost opposite approach to multiplayer: don’t share the execution state at all.

Codex, Cursor, Warp, and other coding systems increasingly let many agents operate in parallel, but they generally don’t let every agent mutate one shared environment. Workers get separate worktrees, branches, sandboxes, or cloud machines, and their contributions are reconciled later.

There is a reason software development evolved this way. Shared mutable state is hard, and two intelligent workers editing the same thing at the same time can create more coordination problems than they solve. Git gives each worker private working state and provides a well-defined way to bring the contributions back together.

That idea may extend well beyond coding. Each agent can get the context required for its job, work independently, leave behind well-defined outputs, and rely on a shared system to coordinate those outputs. Multiplayer in this architecture comes from coordinating independent workers rather than putting them all into the same execution environment.

 Separate the work from the execution

Honeycomb Canvas takes another approach. During an investigation, humans and agents operate on the same canvas, producing findings, queries, visualizations, hypotheses, and other intermediate work that everyone else can see and use.

What’s interesting here is the possibility of treating the artifacts, relationships, and state transitions that constitute the work as durable state in their own right, independent of the execution producing them.

I think there is a bigger architectural opportunity here, and one we are exploring at ThruWire: separate the work itself from the execution doing it. The agent session, harness, model, and runtime can form an execution plane, while a separate work plane holds the durable objects, relationships, state, history, and checkpoints that define the work.

Once the work lives separately from the execution, humans and agents can come and go without taking the work with them. They don’t need to share a brain, a harness, or even the same model. The work itself provides the durable state through which they collaborate.

So where should multiplayer live?

There is a reason no one agrees on the architecture. Each company’s definition of multiplayer tends to follow the layer of the stack it already owns.

These patterns solve different parts of the problem, and a real human-agent team will probably combine several of them at once. A project might have ten humans and fifty agents moving through it over several months. They won’t all share one conversation, run in one harness, use the same model, or even be around at the same time.

The architectural question is therefore not simply what humans and agents should share. It is which state must be durable, where that state should live, and which parts of the system should be allowed to remain ephemeral and isolated.

Whatever architecture we end up with has to survive all of them. At minimum, the shared state needs to survive any one human or agent doing the work.

## Blog: AI-Powered Data Warehouses: Architectural Lessons for Every AI Product
URL: https://thruwire.ai/blog/ai-powered-data-warehouses-architectural-lessons-for-every-ai-product
Date: 2026-09-03
Description: The data stack offers an early view of how mature systems can add inference and agents while preserving governance, semantics, and deterministic foundations.
Tags: Data warehouses, AI architecture, Enterprise AI
Some of the most interesting AI architecture work is happening inside the data stack. Snowflake, Databricks, ClickHouse, BigQuery, MotherDuck, Redshift, and others are adding models and agents to systems that were built well before LLMs.

That is forcing them to work through architectural problems that every AI builder is starting to face. Where should inference run? What should remain deterministic? How should model output be represented as data? Where should agents live? And how should all of this fit into existing systems without rebuilding the entire stack around AI?

Data warehouses are a particularly interesting place to watch this happen. These are mature systems with well-established boundaries around query execution, transformations, compute, semantics, governance, and lineage. AI is now pushing on almost every one of those boundaries. The architectural patterns emerging here may offer an early look at how the rest of the software stack will adapt to AI.

Here are seven architectural lessons we can take from how data platforms are adapting to AI.

 1. Inference is turning into a database operator

One of the clearest patterns is the movement of inference directly into the query layer. Snowflake, BigQuery, Databricks, and others now let developers use models for filtering, classification, extraction, generation, scoring, and aggregation inside data queries.

Calling an LLM from a query is only the beginning. Once inference can be composed with ordinary database operations, the model is effectively participating in query execution. A query can scan rows, ask a model to determine what they mean, filter based on that judgment, and feed the result into a deterministic aggregation.

Snowflake is already pushing this one step further by introducing AI-aware query optimization for AI operators. LLM calls have very different costs from traditional predicates, so the optimizer has to decide where those semantic operations belong in a query plan. At that point, inference is less like an external service you call and more like a new class of database operator.

 2. Transformations can infer facts, not just reshape data

The transformation layer inside warehouses is also changing. Traditional transformations parse, join, normalize, and aggregate using deterministic operations. LLM transformations can go further and actually infer something about the source and then materialize that inference as new data.

This unlocks a whole new set of data sources where the data can only be found through inference. For example, a contract can turn into a collection of obligations. Or a sales call can turn into a set of objections.

Databricks can combine document parsing and AI extraction inside data pipelines, while MotherDuck can apply inference across rows and return structured values that behave like ordinary warehouse data.

Some columns come directly from source systems, some are deterministically calculated, and others may now represent model judgments produced during the pipeline. Downstream, using SQL, they all look like data.

 3. The semantic layer is turning into infrastructure for agents

Text-to-SQL exposed the fact that the schema is not the business model. Knowing that a column is called “revenue” doesn’t tell an agent how the company defines revenue, which table is authoritative, which filters normally apply, or how an analyst would answer a particular question.

Snowflake’s semantic views provide Cortex Agents with that understanding by exposing metrics and relationships along with filters, instructions, and verified queries. Databricks Genie similarly combines Unity Catalog data with example queries, business semantics, and natural-language instructions. Microsoft Fabric’s Data Agent combines schema information with data-source instructions and example queries when generating answers.

The semantic layer was largely built as an interface between warehouse data and analytics tools. Agents give it another job, providing additional context that teaches models how the organization expects its data to be used.

This pattern has uses outside of data warehouses. Agents need access to the data, but they also need a machine-readable model of what that data means.

 4. The location of the agent is an architectural decision

The platforms are taking noticeably different approaches to where agents should live. Snowflake Cortex Agents, Databricks Genie Agents, and ClickHouse Agents put the agent inside the data platform, where it can sit close to the execution layer.

Other platforms expect the agent to live outside the warehouse and expose the data platform as a tool. Databricks and MotherDuck provide MCP interfaces for external agents, while ClickHouse also supports MCP-based connectivity across its agent and data products. Similarly, Agent Toolkit for AWS allows external coding agents to interact with warehouse infrastructure.

Several vendors are pursuing both approaches, and that may become ubiquitous eventually. A company might use a warehouse-native agent for analytics while also allowing Claude Code, Codex, or a higher-level enterprise agent to use the same data platform as one tool among many.

 5. The warehouse is now an execution environment for agent work

Giving an agent query access naturally leads to a larger question: can the agent create and operate the machinery that produces the data as well?

MotherDuck provides a particularly clean example. Its Flights runtime executes Python next to the data on demand or on a schedule. An external coding agent can inspect warehouse data through MCP, write an ingestion or transformation program, deploy it as a Flight, schedule it, and then query what the program produces.

Databricks approaches the same problem with a much broader platform. Unity Catalog, SQL, Python, Lakeflow, Model Serving, Agent Bricks, Apps, and MLflow increasingly provide one environment in which data pipelines and AI systems can be created, executed, governed, and evaluated.

The old boundaries start to get blurry here. Previously, the warehouse held the data, an orchestration system managed pipelines, and applications lived somewhere else. Now, an agent that can inspect data, create transformations, and execute them crosses all three.

 6. Agents are a new database workload

ClickHouse has been particularly explicit about this. They believe agents behave differently from human users, and traditional analytical databases have been designed around humans, not agents.

A human analyst might write a handful of queries while investigating a problem. An agent can inspect metadata, generate a query, execute it, examine the result, form another hypothesis, query another table, encounter an error, inspect the schema, retry, and compare several possibilities in seconds. One human request can therefore produce dozens of database operations.

Agent traffic, on the other hand, can be highly iterative, bursty, concurrent, and latency-sensitive. ClickHouse has emphasized low query latency and high concurrency for this reason, while MotherDuck’s hypertenancy model gives individual users or agents isolated DuckDB compute rather than putting all of their activity onto the same shared compute.

The modern analytical stack was heavily shaped by its dominant consumers: BI dashboards, scheduled transformations, data applications, and human analysts. With agents becoming another major consumer, databases need to be designed around their access patterns too.

 7. AI-generated data needs its own lineage

Once LLMs become part of the transformation layer, their outputs start showing up in the warehouse alongside every other kind of data. But AI-generated data has a different history from traditional derived data. For example, if a model decides that a customer interaction represents a billing complaint, we may also need to know which model made that decision, which prompt it received, and which version of the transformation was running at the time.

The key problem is that it will become more difficult to distinguish a fact that came from a source system from a judgment that originally came from an LLM.

Data platforms already have rich systems for tracking where data came from and how it was transformed. As inference moves into those transformations, that lineage may need to include the models and prompts that helped produce the data too.

 The data stack is a preview of AI architecture

The patterns these data warehouse companies are adopting are likely to matter and be applicable well beyond the data warehouse.

Data warehouses may be one of the first places where large companies figure out how to deploy AI-enabled software at real production scale. They already sit inside mature enterprise systems with existing data, governance, permissions, infrastructure, and users.

That makes the data stack an unusually important testing ground for what production AI actually looks like inside established companies. The architectural patterns that emerge here could end up shaping how much of the enterprise software stack adopts AI.

## Blog: How Much of Your Architecture Is a Temporary Model Workaround?
URL: https://thruwire.ai/blog/how-much-of-your-architecture-is-a-temporary-model-workaround
Date: 2026-09-02
Description: AI architecture is built around model limits that keep moving, forcing builders to distinguish durable product structure from temporary workarounds.
Tags: AI architecture, Model capabilities, Product strategy
AI builders are spending enormous amounts of time building around limitations in the models that may disappear in the coming years. Which ones are safe to build on? And how much of what we’re building today will turn out to have been architectural workarounds?

Some of the hardest engineering problems are caused by limitations in the models themselves. Context is scarce and inference is expensive. Agents forget things. If you give them a long-running task, they tend to go off the rails without a very smart harness. Even getting a model to reliably use tools requires a surprising amount of machinery around it.

We are all building around these limitations even if we don’t realize it. We retrieve information because models can’t hold everything in context, and we give agents memory because they otherwise forget what happened before. We break large jobs into smaller ones because models struggle to stay coherent over long stretches of work. If one agent can’t reliably handle the whole job, we divide the work among several.

There is nothing wrong with doing any of this. These are real problems today. But they create an unusual architectural risk. We may be building increasingly elaborate infrastructure around constraints that are moving underneath us.

The key question is which limitations are temporary properties of today’s models and which represent real boundaries that AI systems will still need years from now.

 Practically Unlimited Context

Context windows are an obvious place to start because their trajectory is clear. We went from a few thousand tokens to hundreds of thousands and then millions remarkably quickly. At the same time, improvements in attention and inference are making those larger windows more practical to use.

No one knows exactly when we'll get to hundreds of millions or a billion tokens, but are headed in that direction. At that point, context becomes abundant enough that application architecture changes. An agent might eventually be able to work with an entire repository alongside a large amount of its project history all at once. It could have access to previous work without someone carefully selecting a handful of fragments to place in its prompt.

There will always be more information somewhere, and attention itself may remain scarce even if tokens aren’t. But the role of retrieval could change considerably. At that point, we may start investing more in helping the model navigate a working set it can already access rather than in retrieval algorithms.

 Agents That Don't Forget

Memory has become one of the most active layers of the agent stack. There are now companies building entire products around giving agents persistence. There is an entire ecosystem of technology emerging for turning what happened during previous runs into reusable knowledge and skills.

Some of this is clearly compensating for the fact that models are effectively born again every time they are invoked. Whatever happened during the last run has to be reconstructed and supplied again. That makes external memory essential.

But that boundary isn’t guaranteed to remain where it is. Models and model systems are getting better at maintaining long-running context and preserving useful state across interactions. Some experimental models can already compact their histories as they work. Research into test-time learning goes further by exploring whether "experience" can actually change future model behavior.

That doesn’t mean memory infrastructure goes away. But there is an important difference between memory that exists because the model forgets and memory that exists because the organization needs to remember.

I would be much more comfortable building around the second.

 Dependable Long-horizon Orchestration

A lot of agent architecture assumes that models can’t be trusted with too much work at once. So we decompose tasks and build graphs of work. We create specialized agents for pieces of the job and retry them individually when they fail. Increasingly sophisticated orchestration layers have grown around this basic constraint.

These techniques are needed because they address a real weakness in the models. Agents can perform amazingly well on bounded problems. Give them a long, open-ended one and all bets are off. Sometimes it produces amazing results and other times it just wastes hours of tokens.

But reliable task horizon is improving. If that trajectory continues, the natural unit of work we give a model gets larger. Work that currently needs to be divided into twenty carefully orchestrated steps may eventually be something we hand to a model as a single assignment and let it decompose for itself.

This doesn’t necessarily eliminate orchestration. A software factory, for example, still has good reasons to separate specification from implementation and verification. But software factories will need to focus more on encoding how organizations work rather than compensating for deficits in the model.

 Cheap and Commoditized Inference

Another architectural assumption hiding everywhere is that inference is expensive. We design around that cost in ways that have become almost automatic. We cache aggressively and route work between models tiers. Or sometimes we build deterministic machinery for a task largely because asking a capable model every time would be too expensive.

This assumption may not hold up. Hardware is improving and model architectures are getting more efficient. And the open weight models are competing here in a way that will drive costs down.

Imagine capable inference becoming another hundred times cheaper. Suddenly it may be perfectly reasonable to ask a model to interpret an API response rather than maintaining brittle transformation logic. A workflow might make dozens of small semantic decisions along the way instead of trying to minimize every model call.

The economic boundary between deterministic code and inference could move substantially, and architecture built primarily around minimizing model calls may age surprisingly quickly.

 Models With Built-In Harnesses

Today’s models live inside increasingly sophisticated harnesses. The harness is responsible for much of what makes an agent feel like an agent in the first place. It assembles context, exposes tools and keeps the execution loop moving. As agents have become more ambitious, harnesses have taken on more responsibility for keeping them on track.

There is already pressure on that boundary from both directions. Agent frameworks are becoming more sophisticated, but model providers are also pushing more agent behavior closer to the model and its APIs. Models are increasingly trained to use tools and operate computers. They are also getting better at recovering from the consequences of their own actions over longer interactions.

Over time, capabilities we currently consider features of the harness may become ordinary properties of the model interface. A future model might manage much of its own working context and sustain extremely long execution loops without the application implementing those capabilities independently. Parallel work and tool use could move further down the stack as well.

The harness isn’t going away, but it's a moving target.

 Work Risks Becoming Opaque

There is a downside to models absorbing more of the machinery we currently build around them. The more context management, memory, planning, and decomposition happen inside the model, the less of the work may be visible to the systems around it.

Today, a lot of this work is externalized almost by accident when an orchestrator creates explicit steps or a memory system records what information was brought into context. Or we can see the work by inspecting the outputs that our subagents produced. These systems exist partly because of model limitations, but they also leave behind useful evidence of how the work happened.

But if all this is happening inside the model in hidden state, we also lose something. Soon we can give the model a huge working context and an objective, and it might spend hours navigating information, decomposing the problem, revising its approach, and carrying forward what it learns. The result could be a much simpler architecture, but also one where far more of the valuable work happens inside inference and disappears when the run is over.

This creates an interesting tension between frontier model innovation and what AI builders may actually need. Frontier labs have every reason to push more context management, planning, memory, and decomposition into the model. But as more of that work moves inside inference, application builders risk losing access to the intermediate state that makes the work understandable and reusable.

As models become capable of doing more internally, we may need to become much more deliberate about deciding what they are required to leave behind.

 Build for a Moving Target

The improvements we can see coming are probably only part of the story. Some of the model capabilities that change how we build over the next few years may be things we aren’t even thinking about yet. A few could make entire categories of infrastructure less important (or more important) almost overnight.

That is part of what makes building in this moment so interesting and so difficult. We are going to have to keep adapting, throwing away some of what we build, and inventing new layers around capabilities we don’t have yet.

## Blog: Agent Memory Roundup 2026: The Race to Build Universal Memory for AI
URL: https://thruwire.ai/blog/agent-memory-roundup-2026-the-race-to-build-universal-memory-for-ai
Date: 2026-09-01
Description: As many agents work across models and applications, shared external memory is becoming a durable operational data layer rather than a feature of one assistant.
Tags: Agent memory, Multi-agent systems, Data architecture
Agent memory has gotten pretty good at helping one assistant remember. ChatGPT helped make persistent memory mainstream, while coding agents and agent harnesses have added plenty of ways to preserve information and carry context from one run to the next. But we are quickly moving from a world where you work with one agent to one where many agents work on the same things.

Claude Code might learn something about a repository that Codex needs later. Or a research agent might remember something about a customer that a sales agent should know. An agent inside a SaaS application may discover something that should be available to an agent running somewhere else entirely.

Today, those agents often have separate memories even when they are working for the same person, company, or project. That creates a different memory problem: how do agents share what they learn, even when they run on different models, harnesses, or applications?

Mem0, Zep, Letta, LangMem, Amazon Bedrock AgentCore Memory, Redis, Supermemory, and Cognee are approaching this in surprisingly different ways, and at different layers of the stack. But there is a common architecture underneath them: memory is moving outside the individual agent, getting its own identity and lifecycle, and becoming something multiple agents can read and write.

Put these approaches together and you can start to see the beginnings of something closer to universal memory for AI, not one universal memory format, but memory that work for any agent consuming it.

 Mem0: Give Memory an Identity

Mem0’s approach starts with scope. Memories can be associated with entities such as a user, agent, application, or run rather than simply living inside a conversation. Multiple agents can then operate against memory associated with the same entity.

It's a neat ownership model where the memory get associated with the thing it's describing. If a support agent learns something about a customer, the useful memory does not necessarily belong to the support agent. It can belong to the customer. Another agent working with that customer can retrieve it later without needing access to the original agent’s state.

Mem0 also extracts durable information rather than requiring agents to replay entire histories. The shared layer becomes a collection of memories organized around who or what they describe. Agents become producers and consumers of that memory rather than its permanent owners.

 Zep: Make Shared Memory a Temporal Graph

Zep makes a different bet about the underlying representation. It builds a temporal knowledge graph containing entities, relationships, facts, and the episodes from which that knowledge was derived. Multiple agents can operate against a shared graph while more personal information remains separated.

It also tracks the temporal validity of facts, including when they became true and, where applicable, when they stopped being true. This is critical for sharing knowledge. For example, let’s say a customer works at one company and later moves to another company or a project changes architectures. One agent reaches a conclusion and another later finds evidence that changes it. Shared memory doesn’t work unless it understands that some knowledge supersedes other knowledge, which is what Zep is able to do well.

Zep also exposes memory to different agent clients through MCP (a common approach by memory providers). That extends the shared-memory problem beyond agents running inside the same framework. An agent controlled by a company and an off-the-shelf agent can potentially operate against the same underlying memory.

 Letta: Give Agents Shared Memory Objects

Letta takes the idea of shared memory more literally. Its memory blocks are persistent objects that can be attached to agents, and the same block can be attached to multiple agents at once. It’s closer to the shared-memory model in operating systems, where multiple processes can access the same underlying state. A block representing a customer, project, or set of operating instructions can therefore become something several agents share rather than information copied into each one’s private memory.

Letta also supports shared archival memory, allowing multiple agents to contribute information to and retrieve information from a common archive.

Letta is closer to collaborative working memory than simple retrieval. Agents can participate in maintaining the persistent memory they use, which also introduces a harder problem: once several agents can modify the same memory, the system has to deal with incorrect conclusions and conflicts.

 LangMem: Build Your Own Shared Memory

LangGraph and LangMem take a more composable approach (as you might expect from LangChain). LangGraph separates the state of an individual thread from longer-term stores that can span threads. LangMem then organizes those memories using namespaces that can represent users, agents, teams, organizations, or other application-defined scopes.

This gives developers control over who shares what. Several agents can read from a common team namespace while maintaining their own private memories elsewhere.

LangMem also separates doing the work from deciding what should be remembered. An agent can deliberately create memories while it works, or another process can inspect interactions afterward and extract or consolidate memories in the background.

But LangMem is much less opinionated about making that memory universally accessible to arbitrary agent products. Developers still decide how agents outside that architecture get access to the store, but you’d have to add an API or MCP interface yourself. That makes it an outlier in this lineup.

 Amazon AgentCore: Make Memory Managed Infrastructure

Amazon Bedrock AgentCore Memory pushes the same separation into managed infrastructure. Amazon went all in on the infrastructure aspects of this. Memory is its own resource rather than something that has to live inside an agent or harness. An existing AgentCore Memory resource can even be attached to multiple harnesses, allowing different agents to operate against the same underlying memory.

AgentCore separates short-term events from long-term memory and provides different strategies for extracting semantic knowledge, summaries, user preferences, and episodes. Namespaces determine how those memories are organized and can include dimensions such as users, sessions, organizations, teams, or other application-defined scopes.

The differentiator here is how much of the memory lifecycle becomes infrastructure. AgentCore can take raw interactions, decide what durable information to extract, organize it into namespaces, and enforce access boundaries around it. Memory formation is becoming something developers configure independently from the agents that ultimately use the resulting memories.

 Redis: Treat Agent Memory Like Data Infrastructure

Redis’s approach looks less like an agent feature and more like a data architecture. Its Agent Memory system separates working memory, long-term memory, and event history, then adds infrastructure concerns such as namespaces, expiration, deduplication, summarization, retrieval, and background consolidation.

That architecture directly supports agents that are spread across processes and machines. Instead of each agent maintaining its own local memory, they can operate against memory that persists independently of where the agents are running.

Redis is essentially applying familiar data infrastructure capabilities such as storage, indexing, retention, and retrieval to agent memory. As more agents depend on the same memory, it increasingly starts to look like data infrastructure.

 Supermemory and Cognee: Share Memory Across Agent Products

Supermemory and Cognee are pushing shared memory across another boundary: completely independent agent products. Supermemory exposes a common memory service that different agent clients can access through MCP, while Cognee can allow multiple clients to operate against a centralized knowledge graph.

Mem0 and Zep are also moving in this direction. The important idea is that Claude Code, Codex, Cursor, or an internal company agent do not necessarily need their own isolated memory systems. They can become clients of memory that exists independently from any one of them.

This is an interesting expansion of MCP. We mostly talk about MCP as a way for agents to access tools and data, but memory providers are also using it to give different agents access to persistent context.

MCP does not make the underlying memory itself interoperable, but it can give different agents a common way to access the same memory provider. If that pattern continues, the same memory could increasingly follow a user or piece of work across models, agents, and applications. The agent becomes a client of memory rather than its owner.

 Architectures Are Converging

The implementations look different, but the direction is surprisingly consistent. Across these systems, the broad direction is toward pulling durable memory out of the individual agent. And most are exposing it over an API or MCP. That allows it to survive changes in sessions, agents, harnesses, and model providers.

The key takeaway is that universal memory does not have to mean one enormous memory that every AI system can see. It can simply mean that memory has an identity independent of the agent consuming it, with boundaries determining which agents can access which parts.

These systems are also converging on something else: structured memory. Shared memory can’t simply be a transcript. Useful memory needs a derived representation of what happened, whether that is a fact, entity, relationship, experience, preference, event, summary, or persistent object. This is a data modeling problem for sure, and another example of where lessons from data engineering need to be carried forward.

 Agent Memory Is Now Its Own Infra Layer

We are at the point where we should consider agent memory its own independent layer of the agent stack, much like databases became an independent layer of application architecture. And there is room for many players.

As organizations use more models and agents, it’s going to be the infra around them that holds them together and allows them to work. Universal memory (or at least externalized memory) makes it possible to change the thing doing the reasoning without necessarily throwing away what previous agents learned.

I also expect the memory layer itself to become more structured and borrow more principles from the data space. Raw events and source material can sit at the bottom. Above them, systems can maintain derived memories with provenance and temporal validity. And above that we may begin to obsess over schemas and relationships within our memories.

At that point, “memory” may start to feel like an inadequate word for the category. We are really describing a durable operational data layer built for a world in which many different agents contribute what they learn.

## Blog: AI Builders: Stop Passing Around Text. Give AI Work a Data Model.
URL: https://thruwire.ai/blog/ai-builders-stop-passing-around-text-give-ai-work-a-data-model
Date: 2026-08-21
Description: Durable AI work needs identity, types, relationships, and provenance instead of being repeatedly flattened into text and reconstructed later.
Tags: Structured work, Data models, Durable artifacts
Language models are good at reconstructing meaning from prose, which makes it tempting to represent AI-generated work as text. But for systems that need to reason across many artifacts over time, repeatedly recovering structure from text is a weak foundation. If AI-generated work is going to persist, be reused, and influence downstream decisions, it needs a data model.

AI systems are already producing that kind of durable work: research findings, plans, recommendations, decisions, approvals, risk assessments, summaries, and other outputs that people and agents may rely on later. Yet most of it still gets passed around as text.

One model produces a paragraph, another reads it, and a third may summarize or reinterpret it. As that work moves through the system, the original structure can get lost, leaving later models to infer what kind of thing they received and how it relates to everything else.

 1. An Artifact Should Have a Type

Consider the difference between a recommendation and a decision. They may both be represented as Markdown and may even contain similar language, but they are not the same thing.

A recommendation proposes an action, while a decision records that an action was chosen. An approval authorizes something, a risk identifies a potential problem, evidence supports a claim, and a plan defines intended work.

Those differences matter to the system even if a model can infer them from prose. Data systems learned a long time ago that important distinctions should be represented explicitly rather than repeatedly inferred from generic text.

AI work should follow the same principle. Instead of producing generic text, a model might produce a Recommendation, Decision, Risk, Claim, Evidence, or Plan, with each type carrying its own fields, validation rules, permissions, lifecycle, and expected relationships to other artifacts.

The text is still there, but it no longer has to carry the entire burden of meaning. The surrounding system now knows what kind of thing it is dealing with.

 2. Typed Artifacts Make Relationships Explicit

Once artifacts have types, relationships between them can become part of the system rather than something buried inside prose. A claim can be supported by evidence, a recommendation can be based on several findings, a decision can accept or reject a recommendation, and a risk can affect a plan.

Those relationships are already present in real work, but today we often force the model to rediscover them every time it reads the surrounding text. That adds both cost and uncertainty to workflows that could preserve those relationships directly.

A system that already knows how one artifact relates to another does not need to infer the relationship from wording, dates, or nearby text each time. It can use those relationships as part of the application state.

This is the role of a semantic layer over AI-produced artifacts. The artifact type tells the system what something is, while the semantic layer captures how that artifact relates to the rest of the work.

 3. Provenance Becomes Much More Useful

We already talk a lot about provenance in AI, but provenance attached to unstructured text only gets us so far. It is helpful to know that a paragraph came from a particular model run or source document, but structured artifacts let provenance become part of the work itself.

A recommendation might record which source material and intermediate findings shaped it, who reviewed it, which model produced it, and which later decisions referenced it. If a downstream result looks wrong, the system can trace through the artifacts that contributed to it rather than reconstructing the history from transcripts.

As agents produce inputs for other agents, preserving origin and status becomes increasingly important. Generated content can otherwise lose its identity as it moves through a system, especially after it has been summarized, rewritten, or incorporated into later work.

A model-generated hypothesis can start looking like a fact, a recommendation can start looking like an approved decision, and an old conclusion can remain in circulation after a newer one has replaced it. Typed artifacts give the system a way to preserve those distinctions.

 4. Change Can Propagate Through the Work

A semantic layer over artifacts also gives AI systems a better way to handle change. Imagine that a source document is corrected and the system needs to determine which downstream work relied on the old version.

If all of that work is stored as documents, answering the question requires another round of search and model interpretation. If the artifacts and relationships are explicit, the system already knows which pieces of work are connected.

This makes invalidation, review, and selective recomputation much easier. A changed input does not necessarily require rerunning an entire workflow, and the system does not have to blindly trust every downstream output either.

It can identify the affected artifacts and determine what needs to be reviewed or regenerated. Data engineering has relied on explicit dependencies for exactly this reason, because connected work is much easier to manage when the system knows what depends on what.

 5. This Is More Than Structured Output

Structured output is a useful first step because it gives a model response a predictable shape. Typed artifacts go further by assigning that output an identity, a lifecycle, and a defined role in the larger system.

A JSON object may exist only long enough for the next step to consume it. An artifact can persist across runs, accumulate reviews and annotations, be superseded by a newer version, and stay connected to the other artifacts around it. The schema describes its fields; the artifact model carries its history and relationships.

Once multiple artifact types share the same semantic model, they can function as durable application state rather than temporary payloads between model calls.

 A New Kind of Data

AI is producing a category of data that traditional software created much less frequently: machine-generated work products that contain interpretation, judgment, synthesis, and recommendations. We should not treat those outputs as disposable text simply because models happen to communicate through language.

If the work is important, it deserves identity, and if different kinds of work behave differently, they deserve types. We have spent the first phase of generative AI getting models to produce better text, and the next phase will require better systems for representing what that text actually is.

## Blog: Context Is Data: AI Builders Should Start Treating It That Way
URL: https://thruwire.ai/blog/context-is-data-ai-builders-should-start-treating-it-that-way
Date: 2026-08-20
Description: Context determines what an AI system knows and produces, so it needs the schemas, lineage, controls, and quality discipline applied to other critical data.
Tags: Context engineering, Data provenance, AI architecture
For most of the history of software, data was treated as something worth managing. We gave it schemas, identifiers, lineage, access controls, versions, indexes, retention policies, quality checks, and systems designed specifically to move it from one place to another.

Then AI arrived, and a lot of that discipline disappeared the moment the data entered a prompt. We started calling it context, which makes it sound temporary. It might come from retrieved documents, conversation history, tool outputs, instructions, or examples, but we often treat all of it as material assembled for the next model call rather than data that needs to be managed.

But increasingly, context is one of the most important forms of data inside an AI system. It determines what the model knows at a particular moment, what assumptions it makes, what sources it trusts, and ultimately what it produces. If we treated context more like data, I think we would build much better AI systems.

 1. Context Needs Structure

We would never design a serious data system where every application received a giant blob of vaguely related information and was expected to figure out what everything meant. Yet that is remarkably close to how many AI systems work.

We gather documents, tool results, conversation history, instructions, user preferences, examples, and intermediate outputs, concatenate some combination of them into a prompt, and ask the model to make sense of it. Models are unusually good at doing exactly that, which is probably why we have gotten away with it for so long.

Their ability to interpret unstructured information should not become an excuse for the surrounding system to have no structure at all. Data systems distinguish between tables, records, fields, entities, relationships, and types. Context systems need similar concepts.

A policy is different from a customer record. A human-approved decision is different from a model-generated summary. A source document is different from an interpretation of that document. Those distinctions should exist before the context reaches the model.

Once context has structure, the system can make better decisions about what to include, how to present it, and how much authority to assign to it. The model no longer has to infer the entire information architecture from a pile of text every time it runs.

 2. Context Needs Provenance

Data engineers care deeply about where data came from. If a number appears in a report, we want to know its source, which transformations produced it, and what upstream systems contributed to it.

AI context deserves the same treatment. If a model receives a claim that a customer has a particular contract term, where did that information come from? Was it extracted directly from the signed contract, copied from a CRM field, summarized by another model three weeks ago, or mentioned casually in a Slack conversation?

Those pieces of information should not be treated as interchangeable simply because they can all be converted into tokens. A system should know the original source, when it was retrieved, what transformations were applied, which model produced any derived interpretation, and whether a human reviewed or modified it.

This matters even more as AI systems create context for other AI systems. A model generates a summary, another model uses that summary to make a recommendation, and a third model later treats the recommendation as input. Very quickly, generated information can become indistinguishable from source information unless the system explicitly preserves the difference.

Data systems learned to track lineage because downstream outputs become difficult to trust when their origins disappear. AI systems have the same problem, except the transformations between source and output are probabilistic.

 3. Context Needs Change Management

Once context is durable, another familiar data problem appears: it changes. A policy is updated, a customer changes plans, a project moves into a new phase, someone corrects a document, a human rejects an earlier model conclusion, or a source that was authoritative yesterday is superseded today.

The question is no longer simply what context should the model receive. The system also needs to know which context is still valid.

Data engineering has spent decades dealing with this problem through incremental processing, change data capture, lineage, materialized views, invalidation, and selective recomputation. The terminology may differ for AI, but the underlying problem is nearly identical.

Imagine an AI system generated ten downstream artifacts using a policy that has now changed. It should be possible to determine which artifacts depended on that policy, whether the change materially affects them, and which pieces of work need to be regenerated or reviewed.

Without that information, there are only a few options: trust potentially stale context, rerun everything, or hope somebody notices when the outputs stop making sense. None of those approaches scales particularly well.

This also changes how we should think about caching. A generated artifact is not reusable merely because we have seen the same request before. It is reusable if the inputs that mattered have not changed.

That requires dependencies, versions, and identity. Those are already familiar data concepts, and context needs them too.

 4. Context Needs a Lifecycle

A surprising amount of AI context currently exists for only a few seconds. A tool is called, its output enters the context window, the model uses it, the response is generated, and much of the useful intermediate work disappears into a transcript or is discarded entirely.

Data infrastructure generally does not work this way. Important data moves through a lifecycle. It may begin as raw input, get cleaned or enriched, turn into a derived dataset, and eventually feed reports, applications, or other downstream processes.

AI context needs a lifecycle too. Some context begins as source material. A model may extract facts from it, combine those facts with other information, produce an intermediate interpretation, receive human feedback, and eventually generate an approved artifact or decision.

Those stages matter. A raw document should not necessarily have the same status as a reviewed conclusion. A temporary hypothesis should not quietly become permanent organizational knowledge. A model-generated interpretation should not overwrite its source.

If the system preserves these stages, context can move from raw information toward increasingly useful and trusted forms without losing its history. The result of an AI task should also frequently become durable context rather than disappearing after the response, because today’s output may be tomorrow’s input.

 5. Context Needs Governance

Once context is treated as data, governance has to be part of the architecture around it. Data platforms have permissions because not everyone should see every row, retention rules because information should not necessarily live forever, and authoritative sources because conflicting copies of the same fact eventually create problems.

AI context has all of the same requirements. The fact that a model technically can retrieve something does not mean that information should automatically enter every context window.

Context should be assembled according to permissions, purpose, authority, freshness, and relevance. This is particularly important as agents gain access to more systems.

Giving an agent twenty tools does not just increase its capabilities. It dramatically expands the universe of information that might enter its working context. The important architectural question is not only whether the model can find information, but whether the system can control the information the model is allowed to reason over.

Data governance developed because unrestricted access to data eventually creates reliability, security, and ownership problems. AI systems will face the same pressures around context.

 The Context Window Is Not the Context System

The context window has become one of the dominant abstractions in AI. We measure how many tokens fit inside it, optimize prompts around it, cache pieces of it, and increasingly build retrieval systems whose primary job is deciding what gets inserted into it.

But the context window is only the final delivery mechanism.

We spent decades learning how to manage data because unmanaged data eventually becomes unreliable data. As AI systems accumulate more sources, artifacts, decisions, memories, tool results, and generated knowledge, context will need the same discipline.

## Blog: Reality Check: A Lot of Frontier AI Is Just Really Good Engineering
URL: https://thruwire.ai/blog/reality-check-a-lot-of-frontier-ai-is-just-really-good-engineering
Date: 2026-08-14
Description: Many apparent model gains come from strong systems engineering around inference, a lesson application builders can use without training frontier models.
Tags: AI engineering, Model infrastructure, System design
The story we tell about frontier AI is overwhelmingly a research story. A new model comes out and the coverage focuses on the training run, the architecture, the data, reinforcement learning, post-training, reasoning techniques, scaling laws, and the researchers figuring out how to push another set of benchmarks forward. When a model gets faster, smarter, safer, or more reliable, our instinct is to assume somebody figured out how to train a better model.

That is also the story the labs themselves are most eager to tell. Their announcements are full of researchers, training breakthroughs, new capabilities, and benchmark gains. None of that is fake or overstated by definition, but it creates a very particular picture of where progress comes from: smarter models produced by better AI research.

A lot of the time, that is exactly what happened. But it leaves out a second source of progress that is much less glamorous and much less visible. OpenAI, Anthropic, Google, and the other labs are not just becoming extraordinarily good at AI research. They are also becoming extraordinarily good at engineering AI systems around those models, and some of the gains we experience as “the model getting better” come from that engineering.

Some of the improvement is inside the weights. Some of it comes from what happens around the weights once the request hits the endpoint. If you only follow the research narrative, it is easy to underestimate how much of the final product is being shaped by software.

 What It Seems Like Is Happening

From the outside, using a frontier model looks almost absurdly simple. You send a prompt to an API, specify the model you want, and get an answer back. Somewhere in the middle, a lot of GPUs run a very large neural network, but from the developer’s point of view that complexity is mostly invisible.

That simplicity encourages a clean story about where the capability comes from. The researchers train a more powerful model, the company puts an API in front of it, and the rest is basically serving infrastructure. When the endpoint gets better, it is natural to assume the model itself got better.

The labs benefit from that perception because the model is the thing with the name, the benchmarks, and the launch event. It is the part of the system they want developers and customers talking about. The cleaner the interface becomes, the easier it is to imagine that the endpoint is little more than a direct line to the intelligence created during training.

But that is not really what is happening anymore. The API hides a growing amount of software that shapes how the model is actually used, and some of the product qualities we attribute to the model are coming from that surrounding system instead.

Once you start looking under the hood, the endpoint looks much less like a thin wrapper around a neural network and much more like a carefully engineered runtime for AI.

What Is Actually Happening Behind a Frontier API Call?

The exact architectures are proprietary, and not every provider uses every technique on every request. But the labs have publicly documented enough pieces that we can see how far we have moved beyond simply sending a prompt into a model and returning whatever tokens come out.

Request safety classification. A request can be examined by a separate safety system before or around the main inference, with OpenAI describing requests being classified into different risk thresholds. That system may use dedicated classifiers or other models, meaning the frontier model may not even be the first model involved in handling your request. The result of that classification can influence whether the request proceeds normally, receives additional scrutiny, or is blocked.

Real-time output safety classification. Safety checks can continue while the model is actively generating rather than waiting until the answer is complete. OpenAI and Anthropic have both described classifiers that inspect streaming output and can pause or intervene during generation. This effectively creates another model-driven control loop running alongside the model the developer actually called.

Input and output content filtering. Providers can separately evaluate both the incoming prompt and the generated candidate response against safety policies. Gemini, for example, documents filtering at both stages rather than relying solely on the behavior learned by the underlying model. The product therefore has an enforcement layer that can override what either the user requested or the model attempted to produce.

Variable inference-time reasoning. Reasoning models can perform substantial internal computation before producing the visible response, and APIs can expose controls that alter how much reasoning effort is allowed. Two calls to the same weights can therefore consume meaningfully different amounts of inference compute depending on the configured reasoning level. Capability is partly a function of how the runtime chooses to exercise the trained model.

Dynamic reasoning allocation. Some systems can go further and decide for themselves how much internal reasoning a particular request deserves. An easy question may require very little additional computation while a difficult one receives a much larger reasoning budget. That turns compute allocation itself into an inference-time policy rather than a fixed property of the trained model.

Speculative decoding. A smaller model or specialized speculative component can predict several tokens ahead, while the larger frontier model verifies those predictions. To the developer it still looks like one call to one model, but underneath, multiple pieces of model computation are cooperating to produce the response faster. The weights did not suddenly become lower latency; engineers changed the execution strategy around them.

Automatic prompt and KV caching. Providers can reuse computation from previous requests when prompt prefixes overlap, preserving intermediate key/value state rather than simply caching the final response. OpenAI and Google both document forms of this happening automatically for eligible requests. An apparently fresh model invocation can therefore begin with a meaningful amount of its neural computation already completed.

Cache-aware request routing. Once cached model state exists, the system has to get new requests to the infrastructure where that state is useful. OpenAI has described routing requests toward machines that have recently processed the same prefix. What appears externally to be a single API call therefore involves distributed-systems decisions about memory locality, placement, and scheduling before inference begins.

Sampling and decoding policy. The neural network produces a probability distribution over possible next tokens, but software still decides how that distribution becomes an actual sequence. Temperature, top-p, token limits, stop sequences, and other decoding policies sit outside the learned weights while directly shaping the response users experience. Even the basic act of turning logits into text is already a collaboration between model behavior and conventional software.

Generation-time grammar and schema constraints. Structured Outputs can do more than ask the model to produce valid JSON and validate it afterward. The runtime can restrict which tokens are valid at each point in generation so the output remains compatible with a schema or grammar. This is a particularly clean example of probabilistic intelligence being surrounded by deterministic software to produce a more reliable product.

Stop and refusal intervention during generation. Generation can end for many reasons other than the model naturally deciding it is finished. Safety systems can intervene, explicit stop sequences can fire, token limits can be reached, or the runtime can recognize that the model is transitioning into a tool call or refusal. The serving layer therefore retains control over whether and when the model is allowed to continue producing tokens.

Multimodal preprocessing. An image sent to a frontier API is not simply passed untouched to the model as a JPEG. Providers resize, tokenize, encode, and allocate different representation budgets depending on settings such as image detail. By the time the model sees the input, software around the model has already made consequential decisions about what representation of that input the model will actually receive.

Automatic modality-dependent processing. The same issue gets broader when an endpoint accepts PDFs, audio, video, images, and text through a common interface. Different input types necessarily require different parsing, segmentation, tokenization, encoding, or representation paths before they can participate in inference. The clean multimodal API hides what is effectively a collection of modality-specific processing pipelines.

State preservation and reuse across turns. Some modern APIs can maintain server-side conversation or reasoning-related state instead of requiring every request to reconstruct the interaction from scratch. That lets the runtime preserve or reuse information and computation across calls while exposing a much simpler conversational abstraction to the developer. The apparent continuity of the model can therefore be partly a property of state management in the surrounding system.

Put those pieces together and the picture looks very different from the one we usually carry around in our heads. There is still a frontier model at the center, but around it are classifiers, caches, routers, schedulers, decoders, constraints, preprocessing pipelines, state managers, safety systems, and sometimes other neural models participating in the same request.

The important point is not that any one of these techniques is revolutionary on its own. Many of them are recognizable software and systems ideas applied to a new kind of workload. What is impressive is how much of the quality of the final product comes from combining them well.

 The Labs Deserve Credit for Both

None of this diminishes the research. Training frontier models is an extraordinary technical achievement, and the labs deserve all the attention they get for advances in model architecture, training, post-training, reasoning, and scale.

But taking those models and turning them into products that can respond interactively, economically, safely, and reliably to enormous volumes of unpredictable requests is a second technical achievement. That work is less visible because successful infrastructure disappears behind the interface, but it materially changes the experience of using the model.

When speculative decoding cuts latency, users experience a faster model. When constrained decoding makes structured output reliable, users experience a model that seems better at following schemas. When caching reduces cost and response time, the model appears more efficient. When a safety classifier catches something the primary model missed, the overall product appears safer.

The model gets the credit because the model has the name. The engineering disappears behind the endpoint, which is exactly what good engineering is supposed to do.

 This Is a Useful Lesson for the Rest of Us

There is a tendency in AI product development to treat model capability as the main lever. Teams compare models, wait for new releases, adjust prompts, increase reasoning budgets, and assume that whatever is not working today may be fixed by the next generation.

The frontier labs themselves are showing that there is another axis of progress. They are improving the models, but they are also constantly improving the systems around those models, and the product gets better from both directions.

That should be encouraging for anyone building on top of frontier APIs. Most teams will never train a frontier model, but every team can improve the system around the model it already has. You can make a product faster by eliminating unnecessary computation, cheaper by reusing work, more reliable by enforcing constraints outside the model, safer by adding independent checks, and more consistent by managing state deliberately.

The specific techniques at the application layer will not be identical to the techniques used inside frontier inference infrastructure. Most of us do not need to build speculative decoders or GPU schedulers, but the architectural lesson still transfers.

Good AI systems come from deciding carefully which problems should be solved by the model and which should be solved by the surrounding software. The labs are doing that constantly, even if the public narrative mostly focuses on the model itself.

## Blog: Your Agents Need Boundaries. Please Keep Them in a Data Sandbox.
URL: https://thruwire.ai/blog/your-agents-need-boundaries-please-keep-them-in-a-data-sandbox
Date: 2026-08-10
Description: Agents need broad access to useful information without owning the pipelines, provenance, and decisions that determine what reaches their context.
Tags: Agent architecture, Data provenance, AI safety
Every time I log into an agent platform, I see more connectors and plugins.

Connect your Slack. Connect Salesforce. Connect Jira, Notion, GitHub, Snowflake, Google Drive, your databases, your internal APIs, and everything else that might contain useful information. Give the agent tools for all of them and let it decide which source to query, when to query it, and how to combine what comes back.

I understand the appeal. More connectors mean more things the agent can know and do. But I think we are heading toward architectures where the agent has far too much responsibility for how data gets into its own context.

Every source has its own semantics, freshness characteristics, schema, failure modes, and security assumptions. The agent ends up responsible for both solving the task and navigating the data infrastructure required to solve it.

I think we should move in the opposite direction. Bring the data to the agent, not the other way around. Put the data through a controlled ingestion process, and let the agent operate inside a trusted data environment.

 Agents Should Consume Data, Not Own the Data Pipeline

An agent already has a difficult job. It has to understand the objective, determine what information matters, reason over it, use tools, produce artifacts, and decide when the task is complete. When we also make it responsible for discovering and reconciling data across dozens of systems, we quietly give it another job: data engineering.

For example, customer information might be spread across Salesforce, billing, support, product analytics, and the warehouse. Those systems may use different identifiers, update at different times, represent the same fields differently, and disagree about which source is authoritative.

A data team would normally resolve those issues before exposing the result to downstream consumers. We create mappings, canonical entities, transformations, tests, lineage, and freshness guarantees. With direct tool access, we often push those decisions into the agent loop and ask the model to reconcile the mess while solving the user’s problem.

 Centralize the Path Into Context

One of the most important questions in an agent system is simple: how does information become context?

If the agent can call whatever tool it wants, every tool becomes part of your context pipeline. Governance, observability, validation, provenance, and access control become distributed across dozens of integrations.

A centralized ingestion layer gives you a stronger boundary. External systems can remain the systems of record, while information passes through a deliberate pipeline before becoming available to agents.

Inside that controlled environment, you can resolve identities, deduplicate records, enforce schemas, attach provenance, strip restricted fields, classify sensitive information, precompute joins, and expose stable semantic objects rather than forcing the model to understand every upstream API.

 ETL Becomes Part of the Security Boundary

We usually think about ETL as infrastructure for analytics. In agent systems, it can also become part of the security model because the data entering the model may directly influence its behavior.

Documents can contain hostile instructions. Web pages can contain prompt injection. A compromised upstream system can introduce manipulated information, while stale or malformed data can produce bad decisions without any malicious actor involved.

Direct access makes these risks harder to contain because the agent retrieves information at the same moment it is reasoning over it. A controlled ingestion layer creates a place to inspect, classify, sanitize, quarantine, and annotate data before it enters the agent environment.

You can record provenance, distinguish trusted internal records from untrusted external text, restrict which fields are exposed, and preserve raw source data for auditing. It gives you a defined control point before data becomes model context.

 Provenance Gets Much Easier

After an agent makes a bad decision, one of the hardest questions can be: what exactly did it know?

With direct tool access, the agent may have queried several APIs whose responses have since changed. It may also take different retrieval paths on different runs, which makes reproducing the original context difficult.

A controlled data plane lets you version the data environment itself. Instead of saying the agent queried several systems at 2:13 PM, you can say it operated against customer snapshot 8421, generated from Salesforce, billing, support, and product events using pipeline version 17.

You can inspect that snapshot, rerun the task against the same inputs, and determine whether a failure came from the model, the transformation pipeline, or the upstream data.

 Tools Still Have a Place

Agents still need tools to send messages, create tickets, modify records, trigger jobs, run code, and interact with systems that cannot reasonably be replicated into a central data layer.

There will also be read cases where direct access makes sense because the data is too dynamic, specialized, or expensive to ingest continuously. The important architectural decision is where the default sits.

I would make arbitrary direct reads the exception and the controlled data layer the normal route by which information reaches the model. That keeps the agent environment understandable while preserving escape hatches where they are genuinely useful.

 The Data Sandbox

I like thinking about this as a data sandbox because the analogy is useful. We already sandbox agent code because we do not want generated programs freely roaming across our infrastructure, so we constrain the filesystem, network, credentials, libraries, runtime, and resources they can access.

Data access deserves a similar boundary. An agent can have access to enormous amounts of information while still operating inside a controlled environment containing structured records, documents, events, cached API responses, embeddings, historical artifacts, and materialized views.

What matters is that information crosses the boundary through a known path. You know where it came from, when it arrived, how it was transformed, what policies apply to it, and which version the agent used.

 Make the Outside World Come Through the Front Door

As agents become more capable, there will be pressure to give them access to more information. I want agents to have access to enormous amounts of data, but I also want to know how that data reached them, where it came from, when it was captured, and whether I can reproduce the context behind a decision.

That makes the data pipeline a first-class part of agent architecture. Build the connectors, ingestion pipelines, canonical entities, semantic representations, validation rules, and provenance layer that bring the outside world into a controlled environment.

Then keep your agents inside the data sandbox.

## Blog: If You’re Not Doing These 4 Types of Caching, You’re Wasting Tokens
URL: https://thruwire.ai/blog/if-youre-not-doing-these-4-types-of-caching-youre-wasting-tokens
Date: 2026-08-06
Description: Prompt, tool-response, output, and structured-data caching can make agent systems faster, cheaper, and more context-efficient.
Tags: AI engineering, Caching, Token efficiency
I see a lot of AI builders spending enormous amounts of time optimizing prompts and skills while leaving money on the table when it comes to token efficiency. We shorten instructions, remove examples, reduce tool descriptions, and obsess over context windows. All of that can help, but a bigger question is whether the model should be processing those tokens again in the first place.

That starts with your caching strategy. For a reasonably sophisticated agent system, there are at least four kinds of caching you should be thinking about: prompt caching, tool response caching, output caching, and structured data caching.

They operate at different layers, but they all follow the same principle: do not pay the model to repeatedly process information or perform work that you already have.

 1. Prompt Caching

This is the one people are increasingly familiar with because the model providers themselves offer it. The basic idea is simple: if a large portion of your input is identical to something the provider has processed recently, it may be able to reuse some of that computation instead of starting over. OpenAI, Anthropic, and others each have their own mechanics, pricing, and retention rules, but the architectural lesson is the same.

Your prompts need stable regions. For prefix-based caching in particular, static content should come first and dynamic content later. That means your prompt builder should also be deterministic. If 30 tool definitions have not changed, do not arbitrarily reorder them. If ten pieces of system context are identical, do not let retrieval order, timestamps, or implementation details move them around between requests. A semantically identical prompt is not necessarily a cache-identical prompt, and one small change early in the prefix can wipe out the savings for everything that follows.

You can spend hours shaving a few hundred tokens off a system prompt while accidentally destroying cache reuse across thousands of requests. Prompt optimization should therefore include prompt stability, not just prompt length.

 2. Tool Response Caching

Provider prompt caching does not solve another common source of waste: agents repeatedly rediscovering information they already retrieved. Start a new conversation or agent session and the agent often checks the repository again, reloads project configuration, reads the same wiki page, calls the same customer API, or queries the same database just to rebuild enough context to continue.

If nothing changed, your harness should not force the agent to rediscover it. There is an obvious savings from avoiding the external API or tool call, but there is also a second savings that may be larger: eliminating an LLM round trip. A typical agent loop goes model → tool call → tool execution → model again with the result. That second model invocation usually means replaying the context plus the tool response, so avoiding the round trip can save a surprising number of tokens.

I increasingly think this should be the harness’s problem. The harness should know what information has already been materialized, what it depended on, how fresh it is, and whether it can still be reused. When the agent needs that information, the harness can inject it just in time rather than requiring another tool call. A wiki page might be keyed by revision, a repository file by commit SHA, a database result by a short TTL, and a customer profile by the events that invalidate it.

Think of the harness as a just-in-time context materializer. The interesting engineering problem is not storing the response. It is knowing when that response is still valid.

 3. Output Caching / Artifact Caching

Now take the same idea one level higher. If you give a model the same instructions, the same source material, the same supporting context, and the same model, should you perform that work again?

My rule of thumb is: if the model has not changed and the inputs have not changed, the output should be roughly the same, or at least close enough that you should seriously question why you are recomputing it. I am using inputs loosely here to mean the full effective context of the operation: instructions, source material, selected knowledge, tool results, upstream artifacts, configuration, and anything else that meaningfully affects the result.

The hard part is that agent systems are often terrible at defining those inputs. Context gets assembled dynamically, retrieval results change, conversations accumulate state, and hidden parts of the harness alter prompts. Two runs that appear identical at the application layer may actually be different model calls. The better you get at identifying and reproducing the effective context for a unit of work, the more model output starts to look cacheable.

This is one of the architectural ideas behind ThruWire. Model work produces artifacts with explicit dependencies, and the runtime has a structured representation of the work and the context that contributed to it. That makes it possible to determine that an intermediate artifact is still valid for a particular set of inputs and reuse it instead of blindly rerunning the model.

You can think of this as memoization for agentic work. It also changes how you structure the work itself. Instead of one giant agent invocation that does ten things, smaller units of model work can have understandable inputs and reusable outputs. If step seven changes, you should not necessarily have to pay to rerun steps one through six.

 4. Structured Data Caching

There is one more kind of caching that I think is badly underused in agent systems: structured data like SQL query results. Client memory is cheaper than tokens. Be liberal about keeping useful data around and frugal about actually dragging that data into the model’s context.

Imagine an agent retrieves 200,000 customer records. You could serialize a giant subset to JSON and stuff it into the prompt because the model might need it. A better approach is to keep that dataset in a locally queryable format and give the agent a tool for asking precise questions about it. Parquet or Iceberg can work well here, with @duckdb providing a lightweight query layer over the data.

Instead of putting the whole dataset into context, give the agent a local query tool and let it request only what it needs. The model might ask for the top 20 high-revenue customers with elevated churn risk, and only those rows cross the model boundary. You keep a rich representation of potentially useful information close to the agent while delaying tokenization until the last possible moment.

That distinction is important. Agent memory should not mean putting everything the agent might remember into its prompt. Memory should mean making information cheaply addressable when the agent needs it. The context window is a working set, not a database.

 Cache the Work, Not Just the Prompt

Once you look at agents this way, token efficiency becomes a systems problem. A good agent harness should constantly be asking whether it has already processed something, already retrieved it, already produced it, or actually needs to put it into context yet.

Prompt engineering and skill design still matter, but there is a more fundamental optimization underneath both of them.

Be caching rich and token poor.

## Blog: I’ve Seen This Movie Before: What AI Builders Can Learn From Data Engineering
URL: https://thruwire.ai/blog/ive-seen-this-movie-before-what-ai-builders-can-learn-from-data-engineering
Date: 2026-08-04
Description: AI systems can borrow proven data-engineering patterns for dependencies, transformation, state, and interfaces instead of relearning them.
Tags: AI architecture, Data engineering, Execution lineage
AI feels new, but many of the problems surrounding it are familiar. We are coordinating complex work, moving information between systems, managing change, and trying to make powerful technology useful to people with very different levels of technical expertise.

Data engineering has spent decades working through versions of these problems. Probabilistic models introduce genuinely new challenges, but they do not make every architectural question new. We should carry forward the patterns that already work rather than rediscovering them through avoidable failures.

After decades working with data systems and now building with AI, I keep seeing the same architectural lessons reappear in a new form. Four of them continue to shape how I think about AI systems.

 1. Graphs Are Your Friend

Data engineering has used graphs for a long time. Query planners turn SQL into execution graphs, orchestrators represent jobs as directed acyclic graphs, and lineage systems track how changes in one dataset affect downstream models and dashboards. Tools such as dbt define transformed datasets as nodes connected by explicit dependencies.

Graphs are useful because complex work is rarely a simple sequence, nor is it a simple loop. Some operations depend on others, some can run in parallel, and some should execute only under specific conditions. A graph makes those relationships visible and executable.

AI applications often have the same shape. A system might retrieve information, classify a request, generate several candidate outputs, evaluate them, request approval, and produce a final artifact. That work already forms a graph, even when it is presented as a single agent loop.

The lesson goes beyond choosing a graph-based orchestration framework. The orchestration itself should usually be deterministic, even when the work inside a node is probabilistic. The model can interpret, synthesize, and generate, while the system defines which steps exist, what they receive, what they produce, and what depends on them.

This creates a practical boundary between flexibility and control. Agent loops remain useful when a task requires exploration or flexible tool use, but they should not automatically become the architecture for the whole system. Data engineering already learned the cost of hiding important dependencies inside procedural logic, and AI does not need to repeat that process.

 2. Extract, Transform, and Load Are Different Jobs

Data engineering separated the movement of information into distinct phases: extract, transform, and load. The order has evolved, with many modern platforms favoring ELT, but the core lesson remains that retrieving information, changing it, and persisting the result are different responsibilities.

There is a direct analogy in AI. Extraction brings new context into the system through tool calls, APIs, databases, document retrieval, search, or other services. It determines what information enters the process and where it came from.

Transformation is where the model does its work. It may classify, summarize, compare, interpret, or synthesize. This phase receives the most attention because the model is the novel component, but its quality depends heavily on what was extracted and how clearly that context was represented.

Loading means persisting the useful result somewhere durable. The output might be a structured record, an intermediate artifact, a decision, an annotation, or a finished deliverable. Once stored, that result can become input for later AI calls, so what is loaded today may become context extracted tomorrow.

Many current AI applications collapse all three jobs into one chat session. Information is retrieved into a prompt, transformed by the model, and returned in a conversation, but the useful result may never be persisted outside the transcript. The boundaries between source material, model interpretation, and authoritative context remain unclear.

That approach works for isolated interactions, but it becomes harder to manage when work continues across users, sessions, and systems. AI needs a durable context layer that can hold sources, intermediate artifacts, decisions, and finished outputs in an organized, queryable form.

Separating these jobs also makes failures easier to diagnose. We can ask whether the correct information was retrieved, whether the model transformed it well, and whether the result was stored appropriately. When everything is combined inside one opaque loop, every failure starts to look like a model failure.

 3. Change Detection Is Key

Data systems are constantly changing. Records are added or updated, schemas evolve, and business logic changes. Data engineering developed techniques such as change data capture, incremental processing, materialized views, lineage tracking, and selective recomputation to keep downstream systems consistent.

All of those techniques address the same question: what changed, and what does that change affect? AI systems need to answer it for context, artifacts, decisions, and generated outputs.

Suppose a company policy changes. Which procedures relied on the old version, which summaries are now stale, and which decisions should be reviewed? Which cached transformations remain valid, and how far should the update propagate?

The same problem appears when a customer record changes, a document is corrected, a human revises an intermediate artifact, or a model produces a better version of an earlier result. The system needs to understand both the change and its downstream consequences.

Without explicit change detection and dependency tracking, AI systems tend toward two poor outcomes. They either trust stale context because they do not know what became invalid, or they rerun everything because they cannot identify what remains valid.

Caching, replay, and propagation are all parts of this larger problem. A cache is useful only when the system knows whether its result still applies. Replay is efficient only when it can begin from the affected point, and propagation is safe only when dependencies are explicit.

AI artifacts therefore need identity, history, and lineage. The system should know which inputs shaped an output, which versions were used, and which downstream artifacts depend on it. A generated result should be treated as part of a connected system of work rather than an isolated block of text.

Data engineering moved beyond full refreshes because they were expensive and unnecessary. AI systems should not have to learn the same lesson through large bills, slow execution, and stale outputs.

 4. Democratization Creates New Interfaces and New Roles

Data became more useful as the industry created different interfaces for different participants. Data engineers built pipelines, analytics engineers created governed transformations, analysts explored data, and business users consumed dashboards, reports, and self-service tools.

Not everyone interacted with the data platform in the same way. The ecosystem developed layers of abstraction that allowed more people to participate without requiring everyone to understand the underlying infrastructure.

AI is currently centered on chat. Chat is flexible and broadly accessible, but it is unlikely to remain the only interface through which people participate in AI-enabled work. Some users will want conversation, while others will need structured forms, workflows, studios, workbenches, dashboards, editors, or approval interfaces.

A person approving a financial decision needs a different experience from someone brainstorming ideas. A domain expert curating shared knowledge needs different controls from an end user requesting a summary. Democratization means designing interfaces that match each participant’s responsibilities and expertise, not giving everyone the same blank text box.

The data industry did not reach broad adoption by forcing every participant into SQL or asking every business user to operate a pipeline. It created tools and abstractions for different kinds of work, and AI should expect a similar progression.

New roles may also emerge. The rise of the analytics engineer created a category between traditional data engineering and business analysis. Analytics engineers organized transformations, defined reusable models, applied software engineering practices to analytical logic, and helped make data trustworthy across an organization.

AI may produce a similar role around context. A context engineer might organize the information AI systems depend on, define relationships between sources and derived artifacts, establish what is authoritative, and determine how changes should propagate.

The title may change, but the responsibility is likely to remain. As AI becomes embedded in organizational work, someone will need to take ownership of the structure and quality of the context surrounding the model. Prompting alone does not cover that responsibility, and existing engineering or product roles may not cover it completely either.

 The Patterns Are Familiar

AI introduces capabilities that data engineering never had to account for. Models can interpret ambiguous requests, generate new material, work across unstructured information, and perform transformations that cannot be reduced to conventional rules.

Those capabilities make the surrounding architecture more important. We still need to represent dependencies, separate retrieval from transformation and persistence, understand what changed, and design interfaces for different kinds of participants.

There will be plenty of genuinely new AI problems to solve, so we should avoid spending unnecessary time relearning the systems lessons that already have good answers.

## Blog: I Paid for These Reasoning Tokens, So Why Can’t I See Them?
URL: https://thruwire.ai/blog/i-paid-for-these-reasoning-tokens-so-why-cant-i-see-them
Date: 2026-07-13
Description: Reasoning tokens create valuable work that usually disappears; durable artifacts can preserve the useful structure for the next iteration.
Tags: Reasoning, Execution lineage, Durable artifacts
Reasoning models were trained to spend more tokens before giving you an answer. Models such as Sol and Fable are rewarded for using that extra computation to break difficult problems into steps, consider alternatives, test conclusions, backtrack when an approach fails, and keep working until they arrive at a better result.

Then almost all of that work disappears. It is a lot of work that the next iteration of your loop will never learn from or benefit from. Why are we throwing away so much valuable work?

Those extra tokens are reasoning tokens. The model generates them into an internal scratchpad that you usually never see, even though they contribute to latency, consume context, and appear on your bill.

Each new reasoning token is fed through the model’s weights again and changes the prediction for the next token. The prompt you write initially steers the weights toward one set of possible outputs, then the model’s own generated context begins steering those same weights in new directions. The model is using its own output to pull the next prediction in directions your original prompt did not explicitly specify.

This is what you are paying for when a model burns thousands of reasoning tokens.

 The Model Is Writing for Itself

It is easy to imagine that the hidden scratchpad contains a clear explanation of how the model reached its answer. Maybe it states its assumptions, compares options, checks its work, and records the sequence of decisions that led to the result.

The actual scratchpad may be far less readable. It can contain fragments, shorthand, repeated calculations, abandoned approaches, tentative claims, contradictions, and abrupt changes in direction. It was trained to help the model get to a better answer, not to help you understand how it got there.

Some of the computation also happens outside the token sequence. The model operates through activations, attention patterns, and distributed representations across many layers. Reasoning tokens give it more computational steps and more self-generated context, but they are not a complete textual transcript of everything happening inside.

The hidden reasoning may therefore be private, messy, and only partially human-readable. That may explain why providers do not simply print it into the interface, but it does not resolve the larger problem.

 The Explanation You See Is Another Output

Products still need to show some sign of progress. A model that remains silent while generating thousands of hidden tokens can feel frozen, especially when a difficult task takes a long time.

The common solution is reasoning narration. The interface tells you that the model is reviewing evidence, comparing approaches, checking calculations, or refining the answer. Some systems also provide a polished reasoning summary after the task is complete.

That narration may be useful, but it is not necessarily what it actually did. It can be a separate model-generated account based on the task, the final answer, selected signals from the hidden trace, or some combination of them.

Research has shown that chain-of-thought explanations can sound convincing while failing to disclose what actually influenced the result. Models can follow a hidden bias in a prompt and then produce a clean explanation that never mentions the factor that changed the answer.

The explanation can rationalize the outcome after the relevant decision has already been made. It gives you a coherent story, but coherence does not guarantee fidelity.

 The Narration Is Theatre

We all love watching models narrate what they are doing. The model says it is reviewing the evidence, comparing alternatives, checking an assumption, or reconsidering its approach, and the running commentary makes the process feel visible.

But the narration is not the process itself. It may be generated separately, sometimes after the relevant reasoning has already happened, working backward from the task and emerging answer to construct a plausible account of what the model might have done.

What actually happened remains invisible. Instead of seeing the process, you are shown a clean textual performance and encouraged to treat it as though it were a live transcript.

Narration gives you the feeling of observability, but it’s a low-fidelity simulation of that process. How are we supposed to build loops and processes around something we can’t really see?

 Loop Engineering Has an Observability Problem

This becomes a serious issue as teams embrace loop engineering. They surround models with prompts, tools, memory, retrieval, evaluators, retries, planning procedures, and feedback, then study completed runs to improve how the loop behaves.

Improvement requires locating the source of an error. A weak result might come from the very reasoning we can’t see. Hidden reasoning makes the diagnosis even harder because the model may have spent thousands of tokens exploring alternatives and correcting itself, while the loop engineer sees only the prompt, some tool calls, a polished narration, and the answer.

What if you wanted to improve how the model thinks about your specific problem? What if you wanted to steer its reasoning so it more closely reflects how your company evaluates risk, how your engineering team makes tradeoffs, or how you personally operate?

What if you simply wanted to audit its thinking so you could trust the final answer more?

The team is left changing the system from the outside. They revise the prompt, replace the model, add an eval, increase (or decrease) the reasoning budget, or introduce another retry, then watch whether the final answer improves.

 Models Should Be Rewarded for Building Structure

We need to reward models for producing structure that breaks complicated work into reusable and inspectable pieces. A strong final answer should not be the only thing the system values.

A research process should leave behind source digests, evidence maps, claim structures, competing interpretations, and unresolved questions. A planning process should preserve objectives, constraints, decision criteria, alternatives, and implementation plans.

Coding work should leave behind requirement interpretations, repository findings, design decisions, patches, test plans, and verification results. Those pieces should exist independently enough that a human or another agent can inspect them, correct them, and use them in later work.

The model can still use a messy hidden scratchpad. The important change is that selected outputs from the reasoning process become explicit work products rather than disappearing into the final answer.

That requires a different incentive. The model should be rewarded not only for producing an acceptable result, but also for constructing intermediate state that is accurate, well organized, reusable, and useful to downstream work.

 The Structure Comes From the Surrounding Stack

Models are unlikely to create this discipline consistently on their own. Their immediate incentive is still to generate the next token that eventually leads to a rewarded answer.

The structure will likely come from the stack surrounding them. It comes from the harness that breaks work into stages, the tools that let models create and revise explicit objects, and the infrastructure that stores those objects with identity, dependencies, and history.

The harness can require a research step to produce an evidence artifact before synthesis begins. It can require planning criteria to be committed before the model ranks alternatives, or require a coding agent to publish repository findings and a verification plan before declaring the task complete.

Tools can make those artifacts addressable. Instead of burying a decision matrix in a chat transcript, the model can create it as a named object that another step consumes directly.

Infrastructure can preserve versions, track which downstream outputs depend on which upstream artifacts, and identify what should be regenerated when something changes.

 The Work Needs a Surface and a Steward

Complex work improves when people and agents have something durable to work on together. Instead of handing each new loop a prompt, a transcript, and a final answer, the system can preserve the evidence, criteria, assumptions, plans, partial products, and unresolved questions that shaped the work.

Those intermediate artifacts create a shared surface. Humans and agents alike can consume those same artifacts directly, build on them, and leave behind stronger versions for whoever works next.

Each human or agent becomes a steward of the work for the next loop, the next iteration, and the next person or system that has to continue it.

This creates compounding value. Each loop inherits more than the result of the previous one. It inherits the strongest parts of the thinking that produced that result, expressed as inspectable pieces that can be reused, challenged, and improved.

Reasoning tokens are real computation and have real business value. I paid for the reasoning tokens, but I probably do not need to read every one of them. I do need the valuable work produced with those tokens to survive the loop.

## Blog: Self-Improving Agents, But Improving at What?
URL: https://thruwire.ai/blog/self-improving-agents-but-improving-at-what
Date: 2026-07-09
Description: Self-improving agents optimize prompts, skills, and workflows, but the bigger opportunity is reusable structure that transfers across new kinds of work.
Tags: Agent learning, Durable artifacts, Evaluation
Self-improving agents have become one of the fastest growing areas of agent research. New papers appear almost weekly describing agents that improve themselves after deployment. The common promise is straightforward: an agent should learn from experience instead of remaining static after it is first built.

The interesting question is what the agent is actually improving. Different systems optimize different parts of the agent stack, and those choices determine both how improvement is measured and how well it transfers to future work.

 Conventional Agent Self-Improvement

The majority of self-improving agents optimize the harness that surrounds the foundation model rather than the model itself. Fine-tuning is expensive, slow, and difficult to validate continuously (but notably there have been huge strides in making those cycles faster and cheaper). It is still much easier to modify the surrounding components that influence how the model behaves, like the harness or the parts it manages.

Those components usually include system prompts, task prompts, skills, tool selection policies, planning heuristics, memory, retrieval, and workflow graphs. The common pattern is to observe an agent performing work, identify recurring failures, propose changes to one or more of these components, and validate those changes before adopting them.

 The Optimization Loop

Although implementations differ, the optimization loop has become fairly standardized. An agent executes a task while recording its trajectory, including prompts, tool calls, observations, intermediate outputs, errors, retries, execution time, and final results. Production systems often supplement this with human edits, approvals, corrections, or downstream business outcomes.

Those execution traces become the raw material for improvement. The system analyzes successful and unsuccessful trajectories, generates candidate changes to prompts, skills, routing policies, memories, retrieval behavior, or workflows, and then replays historical tasks or runs a held-out evaluation set. The new version is compared with the current one, and changes are promoted only when they improve the selected metrics without causing unacceptable regressions elsewhere.

This creates a fairly conventional optimization pipeline. Collect trajectories, extract patterns, generate candidate modifications, evaluate them against representative work, compare versions, and promote the winner. The agent may participate in every stage, but the process usually still depends on an externally defined evaluation function and promotion policy.

 Evals Optimize Tasks

It’s the evals that improve the skills, prompts, and models. The system gets better at the tasks represented by its eval suite because that is what it is rewarded for improving.

But tasks are only one possible optimization target. A system could instead evaluate whether it has developed a better understanding of the domain itself. Has it discovered a simpler decomposition? Has it separated durable concepts from task-specific details? Are its intermediate artifacts becoming more reusable? Are its evaluation criteria becoming more general? Has it extracted first principles, guidance, values, or reasoning that can be reused across entirely different kinds of work?

Those questions are much harder to score than task completion, which is one reason they receive less attention today. They are also the kinds of improvements that are far more likely to transfer when the next task is unlike any the system has seen before.

 Improving the Structure Behind the Harness

If the optimization target extends beyond task execution, the reusable structure behind the harness becomes the more interesting place to invest. Improving that structure requires a different kind of iteration than replaying benchmark tasks through an eval suite.

Organizations have spent decades building these kinds of assets. They interview experienced practitioners, document decision making, extract first principles, write design guidance, establish values, and refine the way experts think about their work. The goal is not simply to solve today’s task more effectively. It is to build reusable ways of thinking that make many future tasks easier.

This matters technically because prompts, skills, and policies are local control surfaces. They influence execution at runtime. But reusable structures, on the other hand, shape the representation of the problem itself, giving every future harness a stronger foundation to build on.

 Improving Representations Alongside Behavior

Imagine two agents completing one hundred competitive intelligence projects. The first agent improves its system prompt, adjusts its search strategy, rewrites several skills, and learns which tools produce better summaries. It becomes noticeably better at the next competitive intelligence project because its execution policy has adapted to that work.

The second agent makes those changes as well, but it also reorganizes the reusable structure behind the work. It discovers that market structure, customer segmentation, technical differentiation, ecosystem position, regulatory pressure, and execution maturity repeatedly emerge as independent analytical dimensions. Those dimensions become reusable artifacts with their own inputs, evaluation criteria, dependencies, and synthesis rules.

When the second agent encounters an unfamiliar company or a different industry, it begins with a richer conceptual representation rather than only a better prompt. The improvement transfers because the structure transfers, even when the exact task, company, or market has never appeared in the training traces.

 Graphs as Executable Structure

Graphs are especially interesting because they provide a place for higher-order thinking to become durable. At the simplest level, a graph describes how work is decomposed into intermediate artifacts and how those artifacts relate to one another. As that graph evolves, it can become an executable representation of how a domain should be understood rather than simply how one task should be executed.

Instead of optimizing for the shortest path to the answer, graphs make it natural to optimize the thinking itself. A graph can deliberately fan out into multiple perspectives, compare competing explanations, search for dissonance, evaluate alternative decompositions, challenge assumptions, and synthesize the strongest ideas back together. Those intermediate products become reusable assets instead of disappearing into a single prompt.

Current models are rarely rewarded for this kind of exploration because their optimization loop is centered on task completion. If the evaluation only measures the final answer, the journey becomes overhead. Giving higher-order reasoning an explicit structure changes the incentive. The exploration itself becomes part of what is preserved, evaluated, and improved over time.

 Harnesses Should Inherit Better Structure

Today’s self-improving agents are primarily optimization systems for tasks. They become better at the work represented by their traces and eval suites by refining prompts, skills, planning heuristics, retrieval, and workflows.

The larger opportunity is to let every new harness inherit an improving body of reusable structure, including conceptual models, decomposition patterns, intermediate artifacts, dependency graphs, evaluation frameworks, first principles, guidance, and values. That shifts the optimization target from individual task performance toward the reusable understanding that makes entirely new classes of work easier to tackle.

## Blog: Code Mode Over MCP: Escaping the Tool Maze
URL: https://thruwire.ai/blog/code-mode-over-mcp-escaping-the-tool-maze
Date: 2026-07-08
Description: Code mode turns MCP from a maze of individual tool calls into a programmable product surface shaped by a semantic SDK.
Tags: MCP, Agent interfaces, AI engineering
The Basic Pattern

Instead of exposing a large set of individual MCP tools directly to the agent, you expose one MCP tool: execute. That tool accepts a small TypeScript or Python program. Inside that execution environment, the program uses an SDK to read state, update resources, call product actions, handle errors, and return a final result.

MCP is still the protocol boundary. The agent is still making an MCP tool call. The difference is that the call can contain a small executable plan instead of a single low-level operation.

 Why This Feels Obvious

Models are very good at writing code. So why are we asking them to call tools one at a time?

A lot of agent integrations make the model behave like a person clicking through a UI. Search this. Read that. Update this. Parse the response. Remember the ID. Call the next thing. Pull every intermediate result back into context. Then hope the model still remembers what it was doing after ten tool calls.

That works, but it is a strange shape for the problem. If the model can write the plan as code, much of the mechanical orchestration should happen inside the execution environment.

 What You Probably Think Code Mode Is About

The obvious argument for code mode is efficiency. Ten MCP tool calls means ten round trips. Ten responses come back into the conversation. Intermediate observations consume context. Tool schemas have to be loaded and understood before the model can pick the right call.

Code mode reduces that overhead. The agent can write a short script that performs the sequence inside the execution environment, keeps temporary state locally, and returns only what matters. That can reduce latency, context bloat, and confusion from overly large tool surfaces.

That argument is true, but it is the least interesting part.

 This Pattern Is Already Showing Up

@Cloudflare has been one of the clearest public examples. Their Code Mode work describes giving the agent one tool that executes TypeScript in a sandbox, with API access exposed through typed interfaces rather than a giant list of individual tools. Their later MCP Code Mode work frames the pattern as letting the model write code against a typed SDK and execute that code safely.

@fastmcp has a Code Mode transform that can wrap an MCP server with discovery tools like Search and GetSchemas, so the model can discover what it needs and then work through a more compact programmable interface. StackOne has described a similar direction for large tool catalogs, including a search-and-execute mode where an agent sees discovery and execution tools instead of hundreds of individual definitions.

@AnthropicAI also pushing the broader pattern through programmatic tool calling and code execution with MCP. Their framing is slightly different because it is a managed model-side capability, but the idea rhymes: let the model write code, call tools from inside that code, and keep raw intermediate tool responses out of the main context loop.

The common thread is simple. When the tool surface gets large or the workflow gets multi-step, asking the model to select one tool at a time starts to look like the wrong interface.

 The Real Power Is the SDK

The real power of code mode is the SDK surface you choose to expose.

A lot of people will naturally start by wrapping each MCP tool as an SDK method. You had search records tool, so now you have client.searchRecords(). You had update ticket tool, so now you have client.updateTicket(). That is useful, but it leaves the deepest opportunity on the table.

The better question is: what is the most powerful SDK I can expose to agents so they can drive my product features in the smartest way possible?

That is how we are thinking about it at ThruWire . The SDK is not just a convenience layer over tools. It is a way to use the precision of an SDK spec to shape model behavior when the model interacts with the product. You can give the agent a clear set of objects, methods, constraints, defaults, and examples that point it toward the right product behavior.

This is different from asking the model to infer your product from a flat list of tools. A semantic SDK can encode judgment. It can collapse common sequences. It can make invalid states harder to reach. It can expose concepts that match the work instead of backend operations. It can turn “call the right tools in the right order” into “write against the right product abstraction.”

That is the part I think matters most. Code mode is not only about moving tool calls into a script. It is about designing the interface that gives the agent the best possible grip on your product.

 MCP as Transport, SDK as Interface

In the direct MCP pattern, the MCP tool list often becomes the product surface. The agent sees your product as a collection of callable functions, and the quality of the integration depends on how well the model can reason across that list.

In code mode, MCP remains the transport and permission boundary. The SDK becomes the working interface. The agent is programming against a temporary environment shaped around the task.

That distinction matters. The SDK can be more coherent, more opinionated, and more semantic than the raw MCP tools underneath it. It can represent workflows instead of endpoints, expose domain concepts instead of storage concepts, and make the right path easier to discover.

 Self-Describing SDKs

The other powerful idea is that the SDK can be self-describing at runtime. It only has to be coherent for the agent at the moment it is used.

That changes the versioning problem. You do not need one universal SDK, kept stable forever, forcing every agent interaction through the same abstraction. The execution environment can expose the SDK that makes sense for the current task, user, project, vertical, or workflow stage.

For one use case, the SDK might expose research primitives. For another, compliance review primitives. For another, planning, editing, or deployment primitives. The underlying MCP server can keep evolving while the agent sees an interface tailored to the work in front of it.

 Faster Product Iteration

This creates a different product iteration loop. You can introduce new capabilities by changing the SDK surface, not just by adding more MCP tools. You can experiment with higher-level abstractions before hardening them into permanent APIs. You can expose narrower SDKs when the agent needs guardrails and richer SDKs when it needs room to explore.

You can also change the SDK as the workflow progresses. Discovery, execution, review, and publishing may each need different affordances. Code mode makes it possible to present the right programmable surface at the right moment, while keeping the whole thing behind a single MCP execute tool.

 The Shift

Code mode over MCP is usually described as a way to reduce round trips and context bloat. Useful, but only the starting point.

The bigger shift is that MCP gives you the protocol boundary, and code mode gives you a programmable product surface. Once the agent is writing against an SDK, you are no longer limited to exposing your product as a flat list of tools. You can expose it as a semantic environment designed for the task.

That is why I keep coming back to the same question: if models are already this good at writing code, why are we still designing so many integrations as if the model can only press one tool button at a time?

## Blog: Why We Wrote Down Our Viewpoint
URL: https://thruwire.ai/blog/why-we-wrote-down-our-viewpoint
Date: 2026-04-06
Description: After six months of building ThruWire, the same structural problems kept showing up across AI systems, pushing us to make our point of view explicit.
Tags: The ThruWire Pattern, Durable artifacts, Execution lineage
For the past 6 months, we’ve been building ThruWire, and along the way we kept running into the same set of problems over and over again. These weren’t edge cases or one-off bugs, they were fundamental issues that show up as soon as you try to move beyond a demo and actually rely on AI to do real work.

What surprised us wasn’t that these problems existed. It was how consistently they appeared, regardless of model, stack, or approach. You get something working in a chat, you guide it carefully, you remind it what context to use and what to ignore, and eventually it produces something useful. But that result is fragile. The path that led to it isn’t preserved in a meaningful way. The context that made it work isn’t clearly defined. And the next time you try to do something similar, you have the same raw materials, maybe you have memories, but you still need to recreate the thread that got you there.

It feels like trying to catch lightning in a bottle, over and over again.

That experience led us to step back and ask a different question. Instead of asking how to get better outputs from models, we started asking why the systems around those models feel so unreliable, so hard to reuse, and so difficult to improve incrementally. The answer we kept coming back to was that most AI systems today aren’t really systems in the traditional sense. They’re sequences of calls, loosely connected by prompts and context, with very few hard guarantees about how they behave.

So we decided to write down our viewpoint (/viewpoint).

This isn’t a marketing exercise, it’s a way to be explicit about what we believe is actually going wrong, and what has to change if AI is going to move from something you “use” in a chat to something you can depend on as part of a system.

A big part of that point of view is that most of what people experience as “non-determinism” in AI isn’t really coming from the model. It’s coming from the system. Context gets assembled slightly differently, dependencies resolve to different intermediate results, execution order shifts, and intermediate work isn’t preserved. When all of that is left implicit, two runs that look similar aren’t actually the same, and there’s no way to tell whether a difference in output came from the model or from everything around it. That makes debugging and trust fundamentally hard.

Another piece is how we think about reuse. Everyone tries to cache AI outputs at some point. It works in simple cases and then quietly breaks as systems become more complex. The root issue isn’t that caching is hard or that models are probabilistic. It’s that most systems don’t have a real definition of what makes two executions “the same.” If identity is just “same prompt,” then as soon as you introduce dependencies, dynamic context, and structure, that definition collapses. Without a precise notion of execution identity, reuse is always either too aggressive or not aggressive enough, and you can’t safely build on prior work.

We also found ourselves questioning how context is handled. There’s a constant tension between starting fresh to avoid drift and carrying forward long histories to preserve continuity. In practice, both approaches degrade. Fresh runs lose useful work. Long-running contexts become messy, slow, and harder to reason about. What this exposed for us is that context is being treated as something to manage manually, instead of something the system should produce. If the system actually executes and materializes its dependencies, then context doesn’t need to be continuously appended. It emerges from the work that has already been done.

The same pattern showed up when we looked at reasoning. There’s a lot of emphasis on chain-of-thought as a way to understand what models are doing. But when we tried to rely on it, it became clear that what models stream isn’t their actual internal reasoning. It’s not the hidden state or the real scratch work. It’s a reconstruction, generated after the fact, optimized for readability. It leaves out the discarded paths and the real decision process. That makes it interesting to read, but not something you can build a system on. If reasoning is going to matter, it has to exist as structured, persistent state that can be inspected, reused, and refined, not just as text in a stream.

We saw something similar with the boundary between models and tools. The common pattern is that models think and tools act. But in practice, many “tools” require reasoning: transforming data, synthesizing outputs, coordinating multiple steps. At that point, either everything collapses back into a single opaque model call, or you allow tools themselves to incorporate reasoning. We think the second path is the only one that scales. It lets you build systems as compositions of smaller reasoning units instead of one large, monolithic one.

And finally, all of this comes back to the interface. Chat has become the default way to interact with AI, and it’s incredibly powerful for exploration. But it’s also inherently ephemeral. It exposes a stream of messages, not a structured system. Streaming tokens makes things feel faster, but it doesn’t solve the underlying problem that the system’s state isn’t clearly defined or preserved. Once you try to build something more complex, you feel that limitation immediately.

Our point of view is that there’s a different way to work with AI, one that is less about conversations and more about execution. Where work is structured, state is explicit, identity is defined, and results can be reused and improved over time.

That’s what we’re building toward with ThruWire.

Writing this down is our way of being clear about where we stand and what we think needs to change. We expect parts of it to be wrong or incomplete. But these are the constraints that have held up for us so far, and they’re the ones shaping the system we’re building.

If any of this resonates, you’ve probably run into the same problems we have.

## Blog: AI Has an Authority Problem
URL: https://thruwire.ai/blog/ai-has-an-authority-problem
Date: 2026-02-26
Description: AI systems encode a hierarchy of control that shapes decisions, often without humans participating in the reasoning that produces confident answers.
Tags: Human authority, Agent interfaces, Customer-facing AI
AI systems today have an authority problem, and understanding it is critical if we are going to trust, deploy, or build on top of them responsibly. Beneath the friendly chat interface is a hierarchy of control that determines how decisions are made and whose judgment prevails. If you rely on AI for code generation, financial guidance, product strategy, or personal advice, you are operating inside this structure whether you realize it or not. At the top sits the system prompt. Beneath that are skills and policies that shape behavior. Below that are tool results, treated as inputs. And somewhere under all of this is the human, usually as a late-stage approver rather than a co-reasoner.

 System Prompt, Skills, and Tools

At the highest level is the system prompt, the constitutional layer of an AI system. It defines identity, constraints, tone, goals, and boundaries. It tells the model what role it plays and what behavior is acceptable. If a user request conflicts with it, the system prompt wins. If tool output suggests a direction that violates it, the system prompt wins. It sits at the top of the authority chain.

Below that are skills. In systems like Claude, skills are modular capability packages that encode repeatable workflows and domain-specific procedures. They allow the model to load structured guidance for particular tasks without overloading every interaction with instructions. Skills improve consistency and operational discipline, but they do not redefine authority. They operate within the frame the system prompt establishes.

Then come tools and tool results. Tools provide access to execution environments such as code interpreters, APIs, databases, and web search. Tool results are data returned from those systems. They are not sovereign and do not override the reasoning process. The model interprets and integrates them in light of the system prompt and active skills. Even deterministic outputs become contextual evidence rather than binding commands.

Humans design this hierarchy. Humans write the system prompts, define the skills, and wire up the tools. But writing a system prompt is not the same as being present for every downstream reasoning decision. Skills can encode best practices, but they cannot anticipate every situation an agent will encounter. Once deployed, the model makes live probabilistic judgments across open-ended inputs. The structure is human-authored. The reasoning is not human-supervised in real time.

 Advice Without You in the Room

Models are quick to offer advice. They will tell you whether to buy or sell a stock, how to restructure your business, how to negotiate compensation, or how to raise your children. The responses can sound measured and confident. But were you looped into how the problem was framed? Were you asked which risks you accept or which values matter most? Or did the system map your question onto statistical patterns and produce the most coherent continuation?

Under the hood, the model traverses probabilities. It explores interpretations, assigns likelihoods, discards lower-probability paths, and collapses uncertainty into a single answer. You see the finished product, not the alternate framings it rejected or the confidence level behind it. Tradeoffs are silently resolved before you ever see them.

Even when tools are involved, the engine remains probabilistic. A financial API may provide precise numbers and a database may return ground truth records, but interpretation is still mediated by learned distributions and constrained by the system prompt and skills. Data flows upward. Authority flows downward.

 The Authority Gap

The deeper problem is not that humans are absent from design. It is that humans are absent from live reasoning. "Human in the loop" usually means approving actions rather than shaping thought. You may confirm a refactor or authorize system access, but you are rarely invited into the stage where the model narrowed the solution space and decided which options were worth presenting.

By the time you see an answer, the framing has already been chosen and the alternatives have already been filtered. You are evaluating a conclusion, not participating in the construction of it. The authority gap is the distance between generating an answer and shaping the thinking that produced it.

That gap matters. If the model dismisses a viable strategy, reframes a complex decision into a simplified pattern, or privileges common wisdom over context, you may never know. The answer feels confident because it is statistically coherent under its constraints. Authority emerges from fluency, not transparent deliberation.

 Is Better Training the Answer?

One response is to argue that the solution is better training. More representative data. More careful reinforcement learning. More nuanced human feedback shaping what "good" looks like. That matters, but it does not remove the hierarchy. It simply embeds someone else's preferences more effectively at the base of it.

Better training refines defaults. It does not replace participation. It cannot anticipate your specific context, values, or tradeoffs in a given moment.

 When It Matters

For low-stakes tasks, the hierarchy works well. Drafting an email or summarizing a meeting is easy to verify, and the consequences are small. But some work depends on framing, tradeoffs, and long-term judgment. Strategic direction, creative choices, decisions that compound over time.

For that work, the authority hierarchy is not an implementation detail. It is the core issue. Who framed the problem. Who chose which options to surface. Who resolved the tradeoffs you never saw. If the answer is a probabilistic system shaped by training-time preferences and runtime instructions, then what you received is not collaboration. It is a confident output shaped by a reasoning process you were never part of.

The question is not whether AI is useful. It is whether, for the work that matters most to you, that hierarchy of embedded judgment is an acceptable substitute for your own.

## Blog: Our Memories Are Executable Code
URL: https://thruwire.ai/blog/our-memories-are-executable-code
Date: 2026-02-17
Description: Human memory preserves structure more than content, and that procedural logic is what current AI memory systems still cannot replay.
Tags: Execution lineage, Durable artifacts, Customer work model
There’s a reason you don’t just remember that something happened. You remember how it unfolded. Not just that you closed the deal, but that you opened with a question, saw hesitation, pivoted your framing, anchored the price, and then stayed quiet at exactly the right moment. What persists is not the headline. It is the structure of what you did.

When you recall an experience, you are not retrieving a frozen recording. Memory is reconstructive. You assemble fragments, impressions, and outcomes into a coherent sequence that makes sense of what happened. You rebuild the logic of the moment.

What often survives that reconstruction is procedural. You recall that you tried one approach, met resistance, shifted tone, and recalibrated. You remember that timing mattered. You remember where the pivot happened. The content fades faster than the pattern.

And here’s the remarkable part: you can rerun it.

You can revisit the same situation and imagine altering one constraint. What if I had been more direct? What if I had waited longer? What if I had acknowledged that concern earlier? You are not just recalling the past. You are rerunning your own strategy under modified parameters.

That works because what you retain is closer to a callable routine than a static description. You did not memorize a paragraph on negotiation. You internalized a pattern: align on goals, surface objections, test willingness, adjust pressure. Over time, that pattern becomes more precise. Certain branches feel wrong. Others feel natural. Thresholds sharpen. Tradeoffs become clearer.

Experience accumulates as structured know-how.

Now compare that to most AI memory systems today.

They store snippets of text, embeddings, sometimes a knowledge graph. More recently, there is discussion of context graphs, structured representations that connect entities, relationships, timestamps, provenance, and task relevance so a system can assemble a coherent view of what matters in a given moment.

That is real progress. Context graphs move beyond flat similarity search. They introduce structure and situational grounding.

But they remain descriptive.

A context graph encodes relationships and relevance. An LLM conditions on that structure and produces a continuation. There is no stored routine to invoke and no authored branching logic to re-execute. When a variable shifts, the system does not rerun a human’s decision process. It infers from the closest declarations and generates a plausible next step.

Very quickly, we are back to the AI inferring human intent.

If the situation changes slightly, the model does not replay a particular person’s thresholds or tradeoffs. It estimates what someone in that context might do. The output can be coherent and even impressive, but it is still probabilistic adaptation.

That distinction matters.

Human memory carries traces of preference, constraint, and judgment. When you rerun a situation, you are operating within your own evolving decision surface. A context graph can describe what was connected and what was relevant. It does not capture the executable logic shaped by a specific mind.

What is missing is not more retrieval. It is callable memory. Memory that can be invoked as a function, not approximated as a guess. Memory that preserves authored structure and can be rerun under controlled parameters.

Until we have that, we do not truly have memory in machines. We have inference over archives.

## Blog: It's 11am: Do You Know Where Your (Human) Users Are?
URL: https://thruwire.ai/blog/its-11am-do-you-know-where-your-human-users-are
Date: 2026-02-10
Description: As users move their thinking into AI chats, software companies risk becoming mere infrastructure. The durable moat is structured domain judgment and customer context that keeps professional thinking inside the product.
Tags: Customer-facing AI, Established SaaS, Product strategy
It's 11am: Do You Know Where Your (Human) Users Are?

 As users move their thinking into AI chats, software companies risk becoming mere infrastructure. The durable moat is structured domain judgment and customer context that keeps professional thinking inside the product. 

Users are doing their thinking in ChatGPT, Claude, Cursor. Every software company knows this already.

Most are responding by becoming better infrastructure. Better APIs for agents to call, MCP servers, better integration points. This is rational and it will keep contracts renewing for a while. It is also a slow-motion surrender of the thing that made the product matter in the first place.

 What Made Users Stay

The products that built the deepest retention in enterprise software all had something in common. The human's professional judgment developed inside the product. The Salesforce power user whose forecasting instincts were shaped by how the CRM structured pipeline visibility. The developer whose architectural sensibility matured through years of code review on GitHub. The designer whose creative judgment evolved through real-time collaboration in Figma.

Data lock-in was real. But something else was happening too. The human's professional effectiveness was entangled with the product. Their thinking lived there.

When that thinking migrates to a general-purpose AI conversation, the entanglement dissolves. The product still holds the data and executes the transactions. The human's judgment is developing somewhere else. The product becomes backend infrastructure.

 The Expertise That Took Years to Build

Any software company that has been in market for real length of time has accumulated something no general-purpose model has. The distinction matters.

A foundation model has read every supply chain textbook ever published. A supply chain software company has watched a thousand implementations succeed and fail. A foundation model knows financial modeling frameworks from published literature. A financial planning platform knows which frameworks break under real organizational pressure and why.

The model learned from text. The software company learned from deployment. These are structurally different kinds of knowledge.

That operational wisdom, the pattern recognition earned through hundreds or thousands of real engagements, was always the core of what software companies provided. In the previous generation, it was embedded in product design: the UI decisions, the guided experiences, the logic of how the product shaped the user's work. Customer success teams tuned the configuration to fit each customer. The combination of general domain expertise and specific customer context produced an experience that made people measurably better at their jobs. That experience was what created advocacy. Not the database schema.

 Why General-Purpose AI Is Winning the Surface Layer

General-purpose AI is outcompeting the delivery mechanism. It is more flexible, more conversational, and often more knowledgeable about the published domain than a product's encoded experiences. Making those experiences incrementally better will not close the gap. The user has already moved.

What general-purpose AI cannot replicate is what a software company learned from deployment, structured as reasoning. The goals practitioners actually need to balance. The constraints that are common versus environment-specific. The tradeoffs that experienced operators navigate differently than textbook answers suggest. The failure modes that only surface at scale.

That knowledge belongs to the company that earned it. If it stays embedded in product experiences that users are bypassing for general-purpose AI, both the competitive advantage and the users walk away.

 What the Customer Needs to Bring

Domain expertise is only half of the equation. The other half is the customer's specific context: their goals, constraints, principles, and the reasoning connecting this quarter's decisions to last quarter's outcomes.

In the previous model, that context lived in the CS team's institutional knowledge and the product's configuration. In a general-purpose AI conversation, it gets reconstructed from scratch every session. That reconstruction is adequate for simple questions and structurally inadequate for sustained work where judgment compounds over months.

For a product to recapture the relationship, the customer's context needs to persist and evolve as a structured layer inside the product, deepening with use. When domain expertise and customer context both exist as structured reasoning and meet inside the same product, the convergence that made retention real gets rebuilt. The response draws on what the company knows about the domain from years of deployments and what the customer is specifically trying to accomplish. It compounds. By month six, the gap between this and a generic AI conversation is large and widening.

 Memory Is Not the Same as Structure

AI memory and retrieval will keep improving. The distinction that holds is between recalling information and reasoning within structure. A model that remembers facts about a customer is not the same as a system operating within the evolving architecture of what that customer is trying to accomplish. Recall is getting better every quarter. Structured reasoning that compounds is a different capability entirely.

 The Choice Being Made Right Now

Every month that users spend developing their thinking inside a general-purpose AI conversation instead of inside the product, the switching cost erodes a little more. The data gravity is real. The integration into the customer's tech stack is real. The human relationship with the product is not, because the product is no longer where the relationship lives.

A software company can own its domain expertise as structured reasoning, combine it with the customer's evolving context, and be the place where professional judgment develops and compounds. Or it can be excellent infrastructure that agents call and humans never think about.

The companies that rebuild this convergence will have users who cannot leave because their thinking lives in the product. The companies that do not will have customers who cannot name their vendor by the end of the contract year.

## Blog: The Organization Was Aligned. Until AI Assistants Showed Up
URL: https://thruwire.ai/blog/the-organization-was-aligned-the-system-is-not
Date: 2026-02-07
Description: Lencioni’s alignment playbook assumed every actor was human. With AI in the system, cultural clarity isn’t enough; organizations need explicit, structural reasoning to keep humans and AI aligned.
Tags: Agent interfaces, Structured work, Human authority
Organizational health is the single greatest competitive advantage available to any company. Smart strategy, technical talent, deep capital reserves, none of it matters if the people inside the organization cannot think and act from the same set of principles. When Patrick Lencioni wrote the definitive book on organizational alignment in 2012, he argued convincingly on these points. For over a decade, "The Advantage" was the playbook. Build a cohesive team. Create clarity. Overcommunicate clarity. Reinforce it through every human system you have.

The entire framework rests on one assumption that nobody had reason to question until recently: every actor in the organization is human.

That assumption is now false.

The Alignment Problem Was Always a Structural Problem

Lencioni diagnosed something real. Most organizational dysfunction comes not from stupidity or bad intentions but from fragmented understanding. People work from different mental models. They hold different assumptions about priorities. They interpret the same strategy through the lens of their own function, their own incentives, their own information. The result is an organization that looks coordinated from above and feels chaotic from within.

His solution was cultural. Hold offsites. Define your six critical questions. Repeat the answers until everyone can recite them from memory. Hire and fire based on behavioral alignment. The mechanism was repetition plus social reinforcement. Say the same things enough times, in enough contexts, and eventually the organization internalizes a shared way of thinking.

This works because humans have a remarkable capacity for implicit learning. We absorb norms. We read rooms. We pick up on what matters by observing what gets rewarded and what gets punished. A healthy organization, in Lencioni's model, is one where the cultural signal is strong enough that individuals can make locally correct decisions without constant top-down coordination.

The entire system depends on internalization. And internalization, in the deep sense Lencioni means it, is something AI does not do. AI can store facts, retrieve documents, and follow instructions. What it cannot do is develop the kind of integrated understanding that allows a culturally aligned employee to navigate ambiguity, weigh competing priorities, and make judgment calls that feel obvious in retrospect but were never explicitly specified.

AI Doesn't Absorb Culture

When an organization adds AI to its operational fabric, something subtle and important changes. There is now an actor in the system that executes work, generates output, makes recommendations, and interacts with stakeholders, but that actor has no ability to absorb the implicit norms that hold the organization together.

AI cannot read the room. It cannot pick up on the unspoken priority shift after a board meeting. It cannot sense that the product roadmap just changed because a key enterprise deal revealed an unmet need, regardless of what the planning documents say. It cannot internalize the difference between what our go-to-market strategy says and what our go-to-market strategy actually is in practice, a distinction that every experienced product or marketing leader navigates instinctively.

This matters because Lencioni's entire model of organizational health depends on exactly this kind of implicit coordination. When he says "overcommunicate clarity," he means repeat the message until humans internalize it so deeply that it shapes their behavior automatically.

The AI industry is aware of this gap, and it is working on it. Long-term memory features remember facts about you across sessions. Retrieval-augmented generation pulls relevant documents into context. System prompts set behavioral guardrails. These are real capabilities, and they represent genuine progress.

They are also categorically different from what Lencioni means by organizational alignment. Memory recalls that your company is pursuing enterprise customers. It does not capture why that strategic bet exists relative to the self-serve motion you deprioritized, how engineering should weigh feature requests from a whale prospect against platform stability, or how the CEO's conversation with a key investor shifted the product thesis in ways the senior team picked up implicitly. RAG retrieves your product strategy document. It does not encode the reasoning structure behind the strategy: which market assumptions depend on which customer signals, where the leadership team has genuine conviction versus hedged bets, or how the positioning should flex when a competitor makes an unexpected move.

AI has more context than ever. The gap is that facts and documents are fundamentally different from reasoning. A product leader who has internalized the company's strategic thinking can make the right call on a feature tradeoff in real time because they understand the structure of how the organization reasons about value, risk, and market position. A marketing lead can adjust campaign messaging on the fly because they grasp the relationship between brand positioning, competitive dynamics, and the current sales narrative. Current AI memory and retrieval features provide ingredients without the recipe.

The result is a new category of misalignment that Lencioni's framework has no vocabulary for. The humans might be perfectly aligned with each other while the AI systems they depend on operate from a plausible but structurally incomplete understanding of what matters.

Two Axes of Alignment, Not One

This is worth sitting with because the implications are significant.

Lencioni's world has one alignment axis: human to human. His tools (behavioral norms, repeated messaging, cultural reinforcement) are designed to get humans synchronized. When they work, the organization thinks and acts coherently.

Any organization deploying AI now has two alignment axes: human to human, and human to AI. And these are not the same problem.

Human-to-human alignment is cultural. It operates through repetition, socialization, shared experience, and implicit understanding. It tolerates ambiguity because humans can resolve ambiguity through context and judgment.

Human-to-AI alignment is structural. It requires explicit declaration of goals, constraints, principles, and relationships between ideas. AI can handle some ambiguity, often impressively, but it resolves that ambiguity through pattern completion rather than organizational understanding. It will produce something coherent and plausible, and that plausibility is precisely what makes the failure mode dangerous. The output reads like it came from someone who understands your organization when it actually came from something that is very good at approximating what that understanding would look like.

Here is the problem: most organizations attempting to deploy AI are applying cultural alignment techniques to a structural alignment problem. They write AI usage guidelines. They distribute prompt templates. They hold training sessions. They treat AI alignment the same way they treat human onboarding: tell people (or in this case, tell the model) what the norms are and trust that behavior follows.

For humans, this works tolerably well. For AI, it fails at the structural level. Even with memory features, retrieval systems, and carefully crafted system prompts, the model's understanding of your organization remains a flat collection of facts and retrieved passages rather than an internalized reasoning structure. The guidelines exist as text that can be surfaced, but text is not the same as the kind of deep structural understanding that allows a seasoned engineer to correctly scope a technical decision against business priorities, or that lets a strategist recognize when a market signal invalidates a core assumption. The gap between organizational intent and AI behavior does not announce itself. It widens silently, at scale.

The Failure Mode Is Different in Kind

When humans are misaligned in Lencioni's framework, the symptoms are recognizable. Engineering builds features that sales cannot sell. Marketing tells a story that product cannot deliver. Strategy says one thing and execution reveals another. These are painful but visible. Organizations can feel when human alignment is breaking down, and the correction mechanisms (offsites, leadership interventions, reorganizations) are well understood even if imperfectly applied.

When AI is misaligned with organizational intent, the failure mode is fundamentally different. The output looks productive. The volume is high. The language is confident. The work appears to be progressing.

But the trajectory is wrong.

This is the core danger. Misaligned AI produces plausible, professional, high-volume output that drifts from what the organization actually needs. Any individual interaction can be corrected, re-prompted, guided back on track. But organizational alignment operates at the level of thousands of AI-assisted decisions, recommendations, and outputs across dozens of people and teams, all needing to cohere with organizational intent without someone manually correcting each one. And because the output at any given point looks good, the aggregate drift goes undetected longer. By the time the misalignment surfaces, weeks or months of compounded divergence have to be unwound.

In Lencioni's human-only model, misalignment produces friction. People argue. Meetings get contentious. The problem announces itself. AI misalignment produces the opposite: false fluency. Everything runs smoothly right up until you realize the AI-generated marketing copy has been optimizing for a value proposition you abandoned two quarters ago, or the AI-assisted engineering decisions have been accumulating technical debt against a product direction that shifted after a strategic review nobody thought to encode.

The scale amplifier makes this worse. A misaligned product marketer produces misaligned positioning at human speed and catches it in the next review cycle. A misaligned AI produces misaligned positioning across every channel, every asset, every customer touchpoint, at machine speed. The volume of confident-but-wrong output can overwhelm an organization's capacity to course-correct.

What Leadership Actually Means Now

Lencioni's definition of leadership is fundamentally about getting humans to share mental models. The cohesive leadership team defines reality. Everyone else absorbs it through cultural transmission.

This model is incomplete for the current moment. Leadership now requires something it never required before: the ability to structure organizational thinking in a form that non-human actors can operate within.

This is a genuinely new leadership competency, something closer to what software architects do when they design systems: defining explicit structures, constraints, dependencies, and governance relationships that ensure correct behavior regardless of which specific agent is executing. Prompt engineering, even with sophisticated system prompts and retrieval, operates on flat text rather than structured reasoning. AI strategy tends to focus on where to deploy AI rather than how to keep it aligned once deployed. This competency sits at a different level entirely.

The architecture here is thinking itself. It is the organizational equivalent of Lencioni's six critical questions, made explicit and structural rather than cultural and implicit. What are our principles? What are our constraints? What depends on what? Who governs what? How do these relationships evolve as our understanding deepens?

In a human-only organization, you can leave these things partially implicit because humans fill in the gaps. In a hybrid organization where AI participates in execution, every gap is a potential failure point. The implicit becomes a liability.

The Structural Turn

There is an emerging body of work in organizational theory that has not yet caught up with this reality. Research on organizational alignment, knowledge management, and decision architecture all assume human actors. The field of AI alignment, meanwhile, focuses primarily on model behavior in isolation, training techniques, reward modeling, constitutional AI, rather than on how AI stays aligned with the evolving, contextual, messy reality of organizational intent.

The gap between these two fields is where the real problem lives. Organizational theorists understand alignment but assume human actors. AI researchers understand model behavior but abstract away organizational context. Neither field is producing a framework for the actual problem enterprises face: keeping human and artificial intelligence aligned within the same operational environment as goals evolve, understanding deepens, and context shifts.

What is needed is something like a structural theory of hybrid alignment. One that accounts for the fact that organizational thinking is dynamic, contextual, and evolving, and that AI participation in organizational execution requires this thinking to be captured in forms that are explicit, persistent, and computationally traversable.

This is a different kind of infrastructure than anything organizations have built before. Knowledge bases store information but miss reasoning. Process maps capture workflow but miss judgment. Strategy documents articulate direction but miss the principles, constraints, and relationships that make direction actionable. What is needed is a reasoning substrate: a persistent, evolving structure of goals, beliefs, principles, and dependencies that both humans and AI can operate within.

The Advantage, Revised

If Lencioni were writing "The Advantage" today, the core thesis would survive but the implementation would transform. Organizational health is still the ultimate competitive advantage. Alignment still matters more than strategy or talent. But the disciplines required to achieve alignment are different when the organization includes non-human actors.

"Build a cohesive leadership team" still holds, but now that team must also be capable of structuring their thinking explicitly rather than relying on implicit shared understanding.

"Create clarity" still holds, but clarity must be structural as well as cultural. It is no longer sufficient for humans to share an understanding; that understanding must be encoded in forms AI can traverse.

"Overcommunicate clarity" becomes something closer to "compile clarity." The repetition-based approach of cultural reinforcement must be augmented with structural approaches that make organizational thinking persistent, explicit, and executable.

"Reinforce clarity through human systems" expands to include the systems through which AI operates. Product reviews, deal strategy sessions, and sprint planning are still important for human alignment. But AI alignment requires reasoning structures, governance hierarchies, and dependency graphs that constrain AI execution to stay within organizational intent.

The organizations that figure this out will have a compounding advantage over those that do not. Their AI systems will get more aligned over time rather than less. Their thinking will accumulate rather than reset. Their human-to-human alignment and their human-to-AI alignment will reinforce each other.

The organizations that do not figure this out will experience an increasingly familiar pattern: productive-looking AI output that slowly diverges from organizational intent, corrected through periodic expensive realignment efforts, only to drift again. The organizational equivalent of Sisyphus.

The Real Competitive Advantage

Lencioni was right that most organizations have plenty of intelligence. What they lack is health. The same is now true at a different level: most organizations have plenty of AI capability. What they lack is the structural alignment to make that capability compound rather than drift.

The advantage belongs to organizations that treat alignment as infrastructure in addition to culture. Cultural alignment remains necessary. It is also no longer sufficient. When your organization includes actors that cannot absorb culture, you need something more: persistent, evolving structures of thought that both humans and AI can reason within.

The hardest problems still require the most human judgment. That has not changed. What has changed is that human judgment must now be captured structurally, not just shared culturally, because there are non-human actors in the system that need access to it.

This is a phase transition in what organizational leadership requires. And most leaders have not yet realized the ground has shifted beneath them.

## Blog: The Last Alpha is Human
URL: https://thruwire.ai/blog/the-last-alpha-is-human
Date: 2026-02-04
Description: As models converge, the advantage shifts to the human-authored thinking substrate that aligns people and AI. A personal story of compounding alignment and the thesis behind ThruWire.
Tags: Human authority, Product-owned work, Customer-facing AI
The remaining advantage in an AI world is how human intellect, experience, and judgment structure the application of AI itself.

This is a structural claim. At the application layer, AI capabilities are converging. The marginal difference between models matters less than how thinking is organized above them: how it persists, how it coordinates across people and systems.

Most approaches to AI productivity miss this entirely. They optimize individual throughput. Output increases. Progress toward goals does not.

My brother and I have worked together across four companies. A data analytics consultancy. Topcoat, acquired by Snyk. Leadership roles at Snyk as product and engineering counterparts. And now this.

The constant across all of it was the hours spent in strategic alignment. Thinking through problems together. Making each other sharper before execution. A shared cognitive substrate that compounded over years.

Working with a brother is an unfair advantage. The trust is absolute. The stakes are high but the ego is low. You can be wrong without losing face. Alignment is the substrate everything else runs on.

This created asymmetric advantage. Our reasoning integrated continuously. We moved faster precisely because we had invested in shared structure.

We are trying to replicate this with AI. The compounding.

The first problem is individual alignment with AI itself.

Current AI tools optimize for output. They make you faster at producing things. But hard problems are solved by sustained thinking that evolves over time, where each session builds on the last, where your reasoning deepens rather than degrades.

Memory features and retrieval systems exist. They recall facts about you. They retrieve documents you wrote. But they do not capture the structure of your reasoning: your goals, your constraints, your beliefs, the relationships between your ideas. The AI can surface what you said. It cannot reason within how you think. It infers structure from text rather than operating within structure you have authored.

Alignment with AI is a structural problem. For AI to help you evolve hard problems over time, it needs access to the structure of your thinking. And that structure must persist and evolve alongside the work itself.

The multiplayer problem follows from the single-player problem. Once you have structured your thinking in a form AI can operate within, the question becomes: how do you share that structure with collaborators? How do their AIs align with yours? How does shared reasoning compound across people?

This workflow does not exist. Multi-person chat will not solve it. Additional context windows will not solve it. The problem is architectural: AI improvisation does not accumulate.

What accumulates is structured human thinking that AI can operate within.

The bet we are making is specific. The future alpha belongs to teams who structure their thinking in a form AI can reason with, and who maintain that structure as a shared, evolving substrate across people.

Hybrid intelligence. Human intellect authors the structure. AI scales the exploration within it.

The differentiator is who has the best thinking substrate: persistent, governed, evolving, and shared. At the application layer, models are converging.

We are building ThruWire.

In a sense, this has always been our ThruWire. The compounding alignment across four companies. The hours of shared thinking that made execution faster. The structure we could not name but always relied on.

Now we are building it.

## Blog: The Harder the Problem, the More Human It Gets
URL: https://thruwire.ai/blog/the-harder-the-problem-the-more-human-it-gets
Date: 2026-02-01
Description: AI is brilliant in the moment, but the hardest work is the arc of reasoning that compounds over time. This essay explains why that arc is still human and what tools are missing.
Tags: Outcome-driven products, Long-horizon work, Human judgment
A great comedian's voice is built over years.

Each set builds on the last. A bit that bombs on Tuesday gets reworked and retested on Thursday. An audience reaction in Denver reshapes the timing of a closer that has been evolving for six months. The comedian develops a feel for what works, and that feel is an accumulation of thousands of micro-judgments, refined through repetition, failure, and attention to context that no one else in the room is tracking at the same resolution.

The jokes are the visible output. The underlying structure, the comedian's perspective, comedic identity, and instinct for what a specific room needs in a specific moment, that is the product of a long arc of deliberate, iterative work. You cannot shortcut it. You cannot produce it in a single session, no matter how brilliant the session is. The value is in the compounding.

This same structure defines the hardest work in every domain. Product strategy refined across market cycles. Brand positioning sharpened through years of customer contact. Engineering architecture shaped by successive scaling challenges. Investment theses deepened across dozens of deals. In each case, the work that matters most is not any single decision. It is the evolving body of reasoning that connects decisions over time and makes each subsequent one sharper than the last.

AI Is Extraordinary in the Moment

This needs to be said clearly, because the point of this piece is not that AI is inadequate.

AI is a remarkable thinking partner for hard problems. Sit down with a capable model to work through a product positioning question, a pricing tradeoff, a competitive response, and the conversation can be genuinely extraordinary. The model surfaces angles you had not considered. It stress-tests your assumptions. It generates options and explores implications at a speed and breadth that no individual human can match. For a single session on a single hard problem, the collaboration is often better than anything you could produce alone or with most colleagues.

Anyone who has experienced this knows the feeling. It is like having the best brainstorming partner you have ever worked with, one who has read everything, never gets tired, and pushes your thinking without ego.

The question is what happens after the session ends.

The Arc Is Where the Hard Work Lives

The hardest problems in business, in creative work, in strategy, share a common trait: they are not solved in a sitting. They are worked over weeks, months, quarters. The product positioning you chose in Q1 gets pressure-tested by market reality in Q2. The pricing model you designed around one set of assumptions needs to adapt when customer conversations reveal a different willingness-to-pay curve. The brand voice you established requires subtle recalibration as the competitive landscape shifts.

Each of these moments is a hard problem. But the real difficulty is not in any individual moment. The difficulty is in the arc: maintaining a coherent, evolving body of reasoning across all of these moments, so that each new decision is informed by the full context of everything that came before.

This is what experienced leaders actually do. They carry forward the principles that emerged from previous rounds of hard thinking. They hold the tradeoffs they accepted and the assumptions they tested. When a new question arises, they do not evaluate it in isolation. They evaluate it against the accumulated structure of reasoning that has been building for months. That structure is what makes their judgment valuable. It is also what makes the work hard. The difficulty is in the coherence over time.

A comedian does the same thing at a different timescale. Each new bit is evaluated against the evolving voice: does this fit the perspective I have been building? Does it sharpen or dilute what I am trying to say? The comedian's judgment about a single joke depends on the full arc of every previous set.

Where the Brilliance Goes

Here is the problem as it stands today.

You have that extraordinary session with AI on Monday. You work through a genuinely hard strategic question. The reasoning is rich. The tradeoffs are carefully weighed. Principles emerge that clarify not just this decision but how to think about the three related decisions coming next quarter. It is some of the best thinking you have done.

You come back three weeks later with the next question in the arc. The AI remembers some facts. Memory features might recall that you chose to focus on enterprise customers. Retrieval systems can surface the strategy documents. But the reasoning that produced those conclusions, the why behind the what, the web of tradeoffs and principles and dependencies, that has flattened into fragments.

The AI will still help. It will produce something thoughtful and well-structured. But the output will be reasonable in isolation rather than precise in context. It might suggest an approach to pricing that a smart person could defend, but that subtly contradicts a tradeoff you deliberately made in the previous round for reasons that no longer appear in any retrievable document. The work looks right. It reads well. And it drifts from the accumulated reasoning that should be shaping it, because that reasoning was never captured in a form the AI can build on.

This is different from the AI being wrong. The AI is being reasonable. The problem is that "reasonable given the available context" and "correct given the full arc of reasoning" are different standards. The gap between them widens with every round of hard thinking that does not get structurally preserved.

The Compounding Gap

The deepest version of this problem is about what accumulates and what does not.

When you spend a year doing hard strategic work, your judgment compounds. Each quarter of market feedback, customer conversations, competitive observation, and internal execution refines your understanding. The principles you hold at month twelve are sharper than the ones you held at month three, because they have been tested against nine months of reality. This compounding is what makes experienced strategists, product leaders, and creative directors valuable. Their judgment is the accumulated product of the arc.

AI's contribution to that arc currently does not compound in the same way. Each session is powerful. The series of sessions does not build on itself in any structured form. The human carries the arc forward in their head. The AI starts each session with whatever fragments of previous context are available through memory and retrieval, which is better than nothing and less than what the arc requires.

The harder the work, the more this matters. Simple tasks do not need accumulated reasoning. You can re-derive the answer each time because the problem is well-defined. The kind of work that unfolds over long arcs, the kind that defines careers, companies, and creative bodies of work, depends entirely on reasoning that compounds. And right now, that compounding happens in human heads but not in the tools humans use to think.

What This Points Toward

I want to be careful here about the line between what I am observing and what I am predicting. Models are getting better. Memory is improving. Context windows are expanding. The trajectory of AI capability is real and impressive, and I am not making a claim about permanent limitations.

The observation is simpler than that. The hardest, most valuable work unfolds over long arcs. The value of that work comes from the compounding of reasoning over time. Current AI tools are extraordinary in the moment and do not yet preserve reasoning in a form that compounds across the arc. This means the most valuable thinking you do with AI assistance today produces brilliant sessions whose accumulated structure lives in your head and nowhere else.

What if it did not have to?

What if the reasoning from those sessions could be captured as evolving structure, so that the principles, tradeoffs, and dependencies that emerge from hard thinking persist and grow as your understanding grows? What if session ten started from the full accumulated context of sessions one through nine, structurally available rather than partially recalled?

The harder the problem, the more human it gets. That will likely remain true for a long time, regardless of how capable models become, because the hardest work is defined by its arc, and arcs require the kind of accumulated judgment that humans build through sustained engagement with hard problems over time. What can change is whether that judgment compounds only in human heads or also in the systems humans think with.

The most valuable thinking deserves infrastructure that matches its time horizon. Right now, it does not have that. And the people doing the hardest work feel this gap every time they sit down for session five and realize the accumulated brilliance of sessions one through four is no longer structurally present.