AI agent engineering moves agents from demos to workflows

A better prompt does not make a production agent. In harness engineering, the harness controls context, tools, verification, budgets, and execution: the infrastructure between a model proposing an action and a system checking whether that action is allowed and whether it worked. For a coding agent, that means isolated execution and test feedback. For a support agent, it means permissioned data access and checking API results before the workflow takes its next step.

Harness-engineering.ai defines the discipline as infrastructure that constrains, informs, verifies, and corrects agents. Those are system-design responsibilities. Instructions to the model cannot take them over.

What does harness engineering cover?

Harness engineering covers the code, configuration, infrastructure, and execution logic that let a model do useful work rather than only generate text. State, context assembly, tools, feedback, enforceable constraints, verification, cost controls, and observability all belong in harness engineering.

LangChain’s formulation is direct: “Agent = Model + Harness.” The model supplies intelligence. The harness supplies capabilities the model does not have out of the box, including durable state and code execution.

Vishal Mysore draws the boundary with four questions. They also make a useful checklist for reviewing an agent’s design:

• What it sees: the data, systems, and context it is permitted to access.
• What it can do: the actions, tools, and workflows available to it.
• What stops it: guardrails, approvals, permissions, and safety limits that constrain execution.
• What feedback it gets: errors, audits, metrics, and signals from people.

For ByteMuni readers moving beyond a text-only demo, start with the work the agent must do. A coding assistant needs repository context and test feedback. A support workflow needs account data and conversation history. A research agent needs coordination across search APIs, document parsers, and synthesis.

An agent framework is a starting point, not a complete production system. LangChain, CrewAI, and AutoGen provide scaffolding: base classes, tool abstractions, and orchestration primitives. The production harness still needs verification, observability, cost control, evaluation, and error recovery.

Inside the harness: state, tools, context, and feedback

Each model decision passes through a controlled cycle: assemble context, select an action, execute it, inspect and verify the result, then preserve state before continuing.

1. Assemble task-specific information. Select relevant code, architecture documentation, and commit history instead of loading the entire repository. Persisted AGENTS.md instructions enter context at session start. Editing that file changes what gets injected into later sessions, not the model’s weights. When a task needs current external information, web search and Context7 can supply it, including updated library versions.
2. Let the model select an action. The model chooses its next action using the assembled context and the available tool descriptions. Bash and code execution serve as general-purpose tools: the agent can write and execute code rather than require a predefined integration for every action it might need.
3. Execute through controlled tools. Tool orchestration validates inputs, parses outputs, handles errors, and manages timeouts. The sandbox contains the language runtimes, packages, git and testing CLIs, and browsers installed for execution. Command allow-lists and network isolation can restrict access further. A sandbox is one control, not a replacement for deciding which commands and network access to allow.
4. Update durable state. Filesystems serve as both a workspace and off-context storage for code, intermediate outputs, and progress across sessions. Git supplies history, branching, and rollback. Later work can recover the project’s state from those records instead of depending only on whatever information still remains in the model’s context at that point.

Keep the working context usable

Context rot is the decline in reasoning and task performance as context fills. Compaction deals with accumulated context by offloading and summarizing it. If a tool output exceeds a token threshold, the harness retains its head and tail tokens and writes the full output to files for later inspection. Skills use progressive disclosure: load front-matter first, not every tool or MCP server into startup context before the agent needs them.

Build a sandboxed coding workflow with a verification loop

A repository-backed coding agent can edit code in isolation, run build, test, and lint tooling, then revise the change using the failures. It is an illustrative workflow assembled from Sources 1–3.

Connect repository context to executable feedback

1. Prepare the workspace. Put the repository in a sandbox, such as a Docker container. Configure the language runtime, dependencies, git, and testing tools inside it so the agent can actually use them.
2. Assemble relevant context. Include codebase architecture, documentation, and historical commit messages. Filter irrelevant files as Source 2 describes; the whole repository is not equally useful for every task.
3. Edit and check. Let the agent change code, then run build scripts, test suites, and lint checks in the isolated environment. A plausible patch remains a proposed change until it is checked.
4. Feed failures back. Return terminal output, compiler errors, lint results, and failing-test stack traces so the agent can rewrite the change. Source 3 describes hooks that run a predefined test suite and loop back with error messages on failure.

In Source 1’s single-agent pattern, verification sits between actions. One agent plans, executes a tool, checks the result, and incorporates feedback before it continues. Schema checks answer a narrow question: are the required fields, expected types, and nonempty values present? They cannot establish semantic correctness. Nor can they prove that tests actually ran. Those are separate questions, requiring task-aware evaluation and executable checks rather than a structurally valid response.

The illustrative Python sketch below assembles context, executes tools, checks output, applies a retry policy, tracks a token budget, and updates task state.

def run_agent_with_verification(task, tools, cost_ceiling):
    context = assemble_context(task)
    total_cost = 0

    while not task.is_complete():
        # Agent decides next action
        action = agent.plan(context, tools)

        # Execute the action
        result = execute_tool(action)

        # Verify the result before proceeding
        verification = verify_output(result, action.expected_schema)
        if not verification.passed:
            if verification.retry_recommended:
                result = retry_with_backoff(action, max_retries=3)
            else:
                return TaskResult(status="failed", reason=verification.reason)

        # Check cost envelope
        total_cost += result.tokens_used
        if total_cost > cost_ceiling:
            return TaskResult(status="budget_exceeded", partial=context)

        # Update context for next step
        context = update_context(context, result)

    return TaskResult(status="complete", output=context.final_output)

What the sketch leaves unimplemented

This is architectural pseudocode with undefined helpers, not a runnable SDK example. Context assembly, tool execution, verification, and completion-state handling all need implementations.

• Budget timing: the sketch checks spending after execution. Source 1’s prose recommends checking before each billable model call and stopping if the next call would exceed the ceiling.
• Retry verification: the retry allows max_retries=3, but the sketch does not visibly reverify the returned result before adding it to context. That leaves a verification gap on the retry path.
• Complete accounting: adding only result.tokens_used does not visibly count planning, model-based verification, or all retry spending. Those costs must also enter the task’s accounting.

Support tickets and insurance claims need different harnesses

These are domain examples from Source 2, not named production deployments with measured outcomes.

Trace the support workflow

Source 2 connects support agents to Salesforce-style CRM systems, Slack, email, and internal wikis. Every integration needs user-scoped permissions. Connecting a system must not let a support agent read executive HR information simply because that information exists somewhere in the organization.

1. Retrieve the ticket and context. Assemble relevant account data and conversation history, restricting retrieval to information the user is authorized to access.
2. Update the ticket’s status. Perform the workflow’s status change through an API, as part of the ticket workflow.
3. Send an email response. Send the customer-facing reply through the email integration.
4. Log the interaction. Write a record of the interaction to a database.

Check for API errors, timeouts, and missing response fields at the verification boundary, before continuing to the next action. Failures can then be retried, rerouted, or escalated. Evaluation needs Source 2’s user-satisfaction and CSAT feedback too, alongside the operational traces. Successful API calls tell you the calls succeeded. They do not tell you whether the customer was satisfied with the interaction.

Different domains, different stopping rules

What Codex, Vercel, and LangChain results actually show

The reported results support a specific claim: harness engineering can improve agent performance, cost, and throughput in particular settings. SOURCE 1 reports the figures below.

Vercel: fewer choices at each decision

Vercel’s example makes the agent’s decision surface concrete. SOURCE 1 describes reducing the available tool set. Fewer tools mean fewer alternatives for the agent to distinguish when choosing an action. The rationale is that wrong tool choices can compound across a multi-step task.

LangChain: a first-party ranking claim, not the same measurement

In SOURCE 3, LangChain says harness changes alone moved its coding agent from Top 30 to Top 5 on Terminal Bench 2.0. As a first-party account, it supports the narrower claim that changing the harness can improve benchmark standing.

What these numbers cannot establish

These accounts measure different things on different tasks: accuracy, completion, leaderboard position, token use, speed, cost, and output volume. They lack enough evaluation detail for a controlled cross-company comparison. Code volume and merged-PR counts measure production activity, not correctness directly. I would not use these reports to promise equivalent gains or accept SOURCE 1’s broader claim that harness engineering improvements consistently outweigh model upgrades.

Verification cannot replace budgets, permissions, or tracing

A document-processing scenario reported in Source 1 shows the budget problem. Malformed upstream responses triggered 340 retries over six hours, with every retry consuming a full context window for replanning. The bill reached $2,400 against usual daily spending of $180. Verification caught the bad data. The retry policy kept spending anyway, because no cost envelope stopped the loop.

Separate correctness checks from execution limits

Bound recovery. Set retry limits and enforce tool timeouts.

Model replacement is not compatibility assurance

Source 1 describes models as increasingly interchangeable commodity inputs. Source 3 qualifies that claim: model/harness co-training can make tool changes harmful. Its example is the Codex-5.3 prompting guide’s apply_patch behavior, which shows training-linked sensitivity to changes in patch method.

When a supervisor helps—and when not to add agents

I’d start with one agent plus verification. Add a supervisor when semantic mistakes justify another model review, or parallel specialists when the task genuinely divides into useful independent work.

Supervision checks meaning, not authorization

The supervisor receives the primary agent’s context and output, evaluates quality, and then approves, overrides, or requests revision. It looks for semantic errors, rather than the structural errors a schema check catches. Source 1 suggests this approach for financial reports, customer-facing output where mistakes damage trust, and production-code changes.

Automated approval is not human authorization. Approval from a supervisor agent does not satisfy the requirement for a human decision, even if the supervisor has checked the proposed action.

Parallel work needs a coordination surface

Source 3 describes independent subagents with isolated contexts. They collaborate through shared files, while git records history, branches, and rollback. In Source 1’s shared-harness design, verification, context, tools, cost tracking, and observability are centralized rather than duplicated across agents.

Long tasks can use planning files and continuation mechanisms. A Ralph loop intercepts an exit attempt, reinjects the original prompt into fresh context, and resumes work from filesystem state.

A basic Q&A bot without tools or multistep work does not need this full stack.

Roll out the harness against representative tasks, not a polished demo

1. Audit 50–100 tasks. Source 1 recommends representative tasks with every failure categorized: missing context, incorrect tool selection, tool errors, and other failures, including hallucinated actions. If a tool integration swallows errors, repair the integration.
2. Add schema verification and structured traces. Check expected fields, types, and non-empty values after every tool call. At each step, record the tool, input, output, verification result, and tokens consumed.
3. Establish a cost baseline. Measure median task cost across a representative sample. Source 1 suggests an initial ceiling of 3x the median, terminating overruns and sending them to review. Treat it as a starting heuristic, not a universally safe setting.
4. Build 20–50 regression tasks with expected outcomes. Run the set daily for high-volume agents and weekly for low-volume agents. Track completion rate, output quality scores, and cost per task.
5. Test domain boundaries. Applying Source 2’s constraints, I’d check that a support agent cannot read executive HR data and that an insurance workflow stops for human-manager approval before transferring funds.
6. Compare changes on the same task set. I’d evaluate model upgrades, tool reductions, and context changes individually against that baseline, rather than crediting the latest change for an unrelated gain.

Apply Source 1’s “design for deletion” principle to the harness itself. Source 3 describes model/harness co-evolution: useful harness primitives influence training, and improving models absorb some workarounds. I’d retain durable state, controlled execution, and verification, but remove workaround complexity once the regression evidence shows it is no longer needed.