Turn expert corrections into reviewed evidence, targeted evaluations, and software changes. A practical approach to improving AI agents after launch.
This note draws on OpenAI and Thrive Holdings’ Tax AI case study. The workflow below is our interpretation for teams improving AI products. Read the source article.
The phrase “self-improving agent” can make the wrong thing sound impressive. The interesting part is not a model changing itself in the background. It is a product that converts the work of experts into evidence an engineering team can use.
That distinction matters in any workflow where correctness depends on context: finance, operations, support, legal review, healthcare administration, or internal systems with years of accumulated exceptions. A person’s correction is useful. But a correction alone does not explain what the system missed, where the miss happened, or whether the same intervention should become an automated rule.
The durable pattern is an evidence loop. Experts correct meaningful work. The product records what happened. The team reviews repeated patterns. Then, and only then, an agent receives a scoped change with a clear evaluation target.
An original diagram of an expert-feedback improvement loop.
A correction is not yet a bug
Start with the reviewer’s intent
Imagine a specialist reviewing an agent-produced result. They change a value before they approve the work. That change could mean the model extracted the wrong number. It could mean the source document was ambiguous. It could mean a later system applied a rule the agent does not own. It could also reflect a domain preference that is valid for one account and wrong for another.
Do not treat every override as a label
Treating every edit as a label is tempting because the data is already there. It is also a quick way to teach a system the wrong lesson.
The first task is classification. Which corrections are actual failures? Which are expected review work? Which are caused by missing product support, an incomplete mapping, a workflow handoff, or a specialist’s judgment? The answer should not live only in someone’s memory or an unstructured ticket.
That is why the user interface matters as much as the model. The review experience should preserve the proposed value, the final value, the underlying source, and the reason for the adjustment when one is known. It should make the evidence useful to the person reviewing the case today and to the team investigating a pattern later.
Production traces make failures explainable
Capture consequential steps
An agent is rarely one prompt followed by one output. A real product may classify files, extract fields, retrieve policy, call an external system, map data into a downstream format, and ask a person for approval. A failure can occur at any of those steps.
If the system only stores the original input and final output, engineering must reconstruct the rest by hand. That makes learning slow and unreliable. It also makes it difficult to distinguish model behavior from product behavior.
Record evidence, not just events
A production trace is a record of the consequential path. It can include the source artifact, the intermediate representation, confidence signals, citations, tool calls, mappings, validation decisions, and the final review action. The precise format will vary by product, but the goal is stable: make an observed difference traceable to a specific decision.
This is not logging for its own sake. Each field should answer a future question. What evidence led to this value? Which component made the decision? Did the source support it? Did the reviewer correct the extraction, the mapping, or something outside the agent’s responsibility?
A useful trace reduces the need to reconstruct a failure. The team can inspect the same evidence the user saw. It can compare cases, identify a shared failure mode, and decide whether a product change is justified.
Turn recurring evidence into an eval
Build a representative set
One bad case may be important, but it does not always define a safe change. A useful eval target emerges after review identifies a repeated, bounded pattern.
For example, suppose several experts correct the same field across similar source documents. The team can gather representative cases, state the expected output, and include counterexamples where a correction would be wrong. That becomes a focused evaluation set.
Define the success condition
The eval does more than measure a model. It defines the engineering task. A developer or coding agent can now ask a concrete question: which part of the pipeline causes the failure, and which change improves the target without breaking the surrounding workflow?
This is a better input than a loose issue such as “the agent sometimes misses this.” A bounded task has source data, expected values, relevant code paths, and commands that prove whether the proposal works. It gives the model enough context to be useful without asking it to guess the definition of success.
The coding agent needs a real harness
Define the task and its limits
Codex or another coding agent is most helpful after the team has done the work of making a problem legible. The agent can inspect the trace, read the targeted eval, identify the responsible code path, and propose a change. It can run the narrow evaluation, then a broader regression suite.
That still does not make the agent the product owner. The system needs explicit boundaries:
- A reviewed finding defines the task.
- The evaluation defines success.
- The repository and tools define the permitted work surface.
- A person reviews the proposed change before it ships.
The order matters. If an agent receives an unreviewed production correction, it may optimize for an exception. If it receives a tested pattern with counterexamples, it can help improve the system while preserving the product’s judgment.
In this shape, the agent is not “self-improving.” The team has built an environment where improvement is repeatable. That is more valuable because it can be inspected, measured, and trusted.
What should stay human
Preserve accountable judgment
There is a useful division of responsibility. Practitioners decide what good work looks like in context. Product teams decide which outcomes matter and where automation belongs. Engineers decide how the system should change. Coding agents help investigate and implement within those decisions.
The goal is not to remove the expert from the loop. It is to use expert feedback to prevent repeated failures. A specialist should not need to rediscover the same defect every week. When their corrections reveal a recurring product problem, the product should preserve enough evidence to improve the underlying workflow.
That is especially important in high-consequence work. Ambiguous cases should return to a person rather than be forced into an automated path. A system earns autonomy by showing that it can recognize the boundary of its own evidence.
A practical starting point
Instrument one decision path
You do not need a complete autonomous loop to start. Pick one expensive, repeatable workflow where people already review agent output. Instrument one decision path. Record the source, the proposed value, the final value, and a small set of correction reasons.
After a few weeks, look for a pattern that meets four conditions:
- It occurs often enough to matter.
- The correct outcome can be stated clearly.
- The product owns the failure mode.
- A regression test can distinguish an improvement from a lucky fix.
Then create one small eval suite and one scoped engineering task. The result may be less dramatic than a promise of continuous self-improvement. It is also much more likely to produce a system that gets better in a way people can verify.
Connect feedback to a tested change
Production feedback becomes valuable when it is connected to evidence, not when it is merely collected. The feedback loop is a product capability: it determines what the team can learn from real use.
Build the trace. Review the pattern. Define the eval. Then let the engineering system act on a problem it can actually understand.