The evolution of AI development: From prompt and context to harness, loop, and graph

When I first used AI to write software, I focused on the prompt. How should I describe the feature? Should I name the stack? Would a few examples make the model produce the right answer in one pass? It felt like searching for the correct spell.

As projects grew, the problem changed. Even a detailed prompt could not stop an AI from reading the wrong file, forgetting an architectural decision, choosing the wrong tool, or leaving untested code behind. The prompt had not become useless. One instruction simply could not contain the entire working environment.

Not five eras, but five layers of responsibility

Prompt, context, harness, loop, and graph are often presented as a timeline in which each new term replaces the last. In practice, they are better understood as five layers of engineering responsibility expanding outward.

A graph still needs prompts, and every node still needs context. Loops usually run inside a harness, while one graph may contain several loops. The evolution is not about old methods disappearing. It is about admitting how much more we must manage if an AI is expected to complete real work reliably.

Layer one: Prompt engineering — how should we ask?

Prompt engineering deals with the instruction itself: the task, constraints, response format, and examples of a good result. For summarization, classification, rewriting, or a one-off code fragment, this layer may be enough.

A good prompt does not have to be long. It should make the goal, essential background, hard constraints, and success criteria clear. As models improve, there is less reason to encode every tiny step as an if-else rule. Too many rules can conflict and consume attention the model could use to understand the task.

The limitation is that prompt engineering usually assumes the required information is already present. When the answer depends on a changing codebase, database state, or previous tool results, the question moves from how to say it to what the model should see right now.

Layer two: Context engineering — what should it know now?

Context is more than the user’s latest sentence. System instructions, tool descriptions, files, conversation history, search results, errors, memory, and examples may all occupy the same context window. Context engineering curates the most useful information within that finite attention budget.

The common mistake is assuming more context is always better. A model may accept a long input without giving every part equal attention. Signal density matters: what belongs in the window now, what can remain as a path or index until the agent needs it, and what stale output should be summarized or removed?

For a coding agent, a clear directory structure, searchable specifications, versioned decisions, and documentation close to the code are all context engineering. They help humans take over, but they also let the agent retrieve information when needed instead of waiting for someone to paste it in.

Layer three: Harness engineering — where and how can it act?

Knowing the project is not the same as being able to complete the work. The model also needs an environment it can observe and act within. The filesystem, terminal, browser, sandbox, permissions, tests, lint, CI, logs, metrics, and deployment pipeline are all parts of the harness.

Harness engineering is not about wrapping a model in a polished interface. It makes correct behavior easier, exposes mistakes, and enforces important boundaries. An agent cannot debug without logs, can only guess at a UI it cannot launch, and cannot choose a useful next step when tests fail without explaining why.

In its harness engineering case study, OpenAI says early progress was limited not because Codex could not write code, but because the environment lacked enough tools, abstractions, and internal structure. The engineer’s work moves from writing every line to designing a workplace the agent can understand, use, and cannot casually escape.

Layer four: Loop engineering — what happens after failure?

One tool call rarely means the work is complete. An agent becomes useful when it can observe a result and choose the next action: inspect state, plan, act, verify, and correct until it reaches the goal or a stopping condition. That is the agent loop.

Loop engineering is not yet as standardized a discipline or job title as prompt engineering, so I use it as a practical design lens. We still need to decide what each iteration can observe, how errors return to context, how often the same failure may be retried, what evidence counts as done, and when the task must return to a human.

A loop without a stopping condition is just expensive repetition. A loop without a verifier can keep producing different-looking versions of the same mistake. Tests, graders, reviews, diffs, screenshots, and human approval are not decorations around the loop; they are the feedback signals that let it converge.

Layer five: Graph and orchestration engineering — how do steps coordinate?

A single loop works for one goal that can be corrected repeatedly, but real software workflows branch. Research and UI exploration can happen in parallel. A database migration must precede backend deployment. A failed test returns to implementation, while a security-sensitive change may require human approval. This is no longer a straight line.

A graph represents work as nodes and edges. Each node reads state, completes a bounded task, and chooses the next path from its result. It can express sequences, branches, parallel work, loops, handoffs, and human-in-the-loop gates. State no longer has to be hidden inside one long conversation; the workflow can manage it explicitly.

Graph engineering is not a fixed name used by every team, so graph and orchestration engineering is more precise. The point is not to require LangGraph or any particular framework. It is to move process control out of an overloaded prompt and into software structure that can be observed, tested, and recovered.

START
  → gather_context
  → implement
  → test
      ├─ failed → implement       # loop
      └─ passed → review
                    ├─ changes_requested → implement
                    └─ approved → human_gate → deploy

OpenAI’s Symphony uses the issue tracker as a control plane for coding agents. LangGraph models an agent runtime with shared state, nodes, branches, and loops. Their implementations differ, but both reveal the same shift: as concurrent agent work grows, the bottleneck moves from generating code to assigning work, tracking state, and managing human attention.

A login feature across all five layers

At the prompt layer, we might write: “Add email login using the existing React and API architecture, preserve the current visual style, and include tests.” The goal is to make intent and success criteria explicit.

At the context layer, the agent reads the routes, auth service, data schema, design rules, and previous security decisions. It does not need to ingest the whole repository at once. It can begin with high-signal documents and search for the rest when needed.

The harness lets it create a branch, edit files, start the development environment, operate the login screen, inspect the console, run tests, and stay away from production secrets.

The loop reads a failed test, corrects the implementation, runs it again, and verifies the real flow in a browser. The graph can separate database checks, frontend work, backend work, and security review, run safe tasks in parallel, and wait for human approval before deployment.

Not every task needs a graph

A clear prompt is enough to shorten a paragraph. Add context when the answer depends on project knowledge. Build a harness when the model must change a real system. Add a loop when the work needs repeated verification. A graph begins to pay for itself only when branches, parallel work, durable state, or approval gates actually appear.

One-shot generation       → Prompt
Needs project knowledge   → Context
Acts in a real system     → Harness
Verifies and self-corrects → Loop
Branches, parallelism, or approval → Graph

Turning every task into a multi-agent graph does not automatically improve the result. More nodes also mean more latency, cost, state synchronization, and failure modes. Engineering judgment is not about always choosing the highest layer. It is about choosing the smallest structure that can complete the task reliably.

The engineer did not disappear; the role moved outward

As AI writes more code, an engineer’s value does not collapse into prompt writing. The scarce skills become defining the right problem, curating useful context, building executable environments, designing feedback, limiting permissions, and deciding which results may advance.

Prompt engineering taught us how to speak to a model. Graph engineering forces a harder question: once the model starts doing real work, how does the surrounding system know it is still moving in the right direction?

The evolution of AI development is not a story about models needing less engineering. The more models can do, the more we need to turn rules that once lived in a person’s head into systems the AI can read, execute, and verify. Reliability is no longer one perfect answer; it is distributed across prompt, context, harness, loop, and graph.