How to Build a Coding Agent That Can Run Tests and Fix Failures

Before you start

You have a codebase, a failing test suite, and you are tired of manually reading stack traces to find the one line that broke. You want an automated agent that can run your tests, see the failure, and propose a fix. This guide walks through building a minimal but functional coding agent in TypeScript that does exactly that.

We will use the ReAct pattern: the agent reasons about the failure, decides which tool to call (run tests, read files, edit files), observes the result, and repeats until tests pass or it gives up. We will use the OpenAI API for the LLM, but the same loop works with any model that supports tool calling.

By the end, you will have a script that you can point at a repository, and it will attempt to fix a failing test automatically. You will also learn how to add safety guards so the agent does not run destructive commands.

mkdir coding-agent && cd coding-agent
npm init -y
npm install typescript tsx @types/node openai dotenv
npx tsc --init --module nodenext --target es2022 --outDir dist --rootDir src

• Node.js 20+ installed
• An OpenAI API key (or compatible endpoint)
• A small test project to experiment with (we will create one)
• Basic familiarity with TypeScript and shell commands

Step 1: Define the agent tools

The agent needs a set of tools it can call. Each tool has a name, a description, and a function that executes the action. We will start with three tools: run a test command, read a file, and write a file (with a safety check).

Create a file src/tools.ts with the tool definitions and implementations. We use the OpenAI tool schema format, which is widely supported.

import { execSync } from 'child_process';
import { readFileSync, writeFileSync } from 'fs';

export const tools = [
  {
    type: 'function',
    function: {
      name: 'run_test',
      description: 'Run the test suite and return the output. Use this to check if tests pass or fail.',
      parameters: {
        type: 'object',
        properties: { command: { type: 'string', description: 'The test command to run, e.g. npm test' } },
        required: ['command']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'read_file',
      description: 'Read the contents of a file in the repository.',
      parameters: {
        type: 'object',
        properties: { path: { type: 'string', description: 'Relative path to the file' } },
        required: ['path']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'write_file',
      description: 'Write content to a file. Use this to apply a fix. Only allows editing files in the current working directory.',
      parameters: {
        type: 'object',
        properties: {
          path: { type: 'string', description: 'Relative path to the file' },
          content: { type: 'string', description: 'Full content to write' }
        },
        required: ['path', 'content']
      }
    }
  }
];

export async function executeTool(name: string, args: any): Promise<string> {
  switch (name) {
    case 'run_test': {
      try {
        const output = execSync(args.command, { stdio: 'pipe', encoding: 'utf-8' });
        return 'Tests passed. Output:\n' + output;
      } catch (e: any) {
        return 'Tests failed. Output:\n' + e.stdout + '\n' + e.stderr;
      }
    }
    case 'read_file': {
      try {
        const content = readFileSync(args.path, 'utf-8');
        return content;
      } catch (e: any) {
        return 'Error reading file: ' + e.message;
      }
    }
    case 'write_file': {
      // Safety: ensure path is inside the project directory
      const resolved = require('path').resolve(args.path);
      const projectRoot = process.cwd();
      if (!resolved.startsWith(projectRoot)) {
        return 'Error: path is outside the project directory';
      }
      try {
        writeFileSync(resolved, args.content, 'utf-8');
        return 'File written successfully';
      } catch (e: any) {
        return 'Error writing file: ' + e.message;
      }
    }
    default:
      return 'Unknown tool: ' + name;
  }
}

Step 2: Build the agent loop

Now we create the main agent loop. It sends the conversation history to the LLM along with the tool definitions. If the model responds with tool calls, we execute them, append the results, and loop. We also set a maximum number of iterations to avoid infinite loops.

Create src/agent.ts with the loop and a simple prompt that instructs the agent to run tests first, then read and fix files.

import OpenAI from 'openai';
import { tools, executeTool } from './tools.js';

const openai = new OpenAI();

const systemPrompt = `You are a coding agent. Your goal is to make the test suite pass.
Start by running the tests to see the failure. Then read the relevant files, identify the bug, and write a fix.
You can run tests multiple times. Do not stop until tests pass or you have tried 5 times.
Always run the tests after making a change.`;

export async function runAgent(initialCommand: string) {
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    { role: 'system', content: systemPrompt },
    { role: 'user', content: `Run this command to start: ${initialCommand}` }
  ];

  const maxIterations = 10;

  for (let i = 0; i < maxIterations; i++) {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o',
      messages,
      tools,
      tool_choice: 'auto'
    });

    const message = response.choices[0].message;
    messages.push(message);

    if (message.tool_calls) {
      for (const toolCall of message.tool_calls) {
        const result = await executeTool(toolCall.function.name, JSON.parse(toolCall.function.arguments));
        messages.push({
          role: 'tool',
          tool_call_id: toolCall.id,
          content: result
        });
      }
      continue;
    }

    // No tool calls means the agent is done
    console.log('Agent final message:', message.content);
    break;
  }
}

// For standalone execution
const command = process.argv[2] || 'npm test';
runAgent(command).catch(console.error);

Step 3: Create a test project to fix

To see the agent in action, we need a project with a failing test. Create a simple JavaScript file with a bug and a test that expects the correct behavior.

Make a directory test-project inside the coding-agent directory, and add the following files.

mkdir test-project && cd test-project
npm init -y
npm install --save-dev jest
# add a script to package.json: "test": "jest"

// math.js
function add(a, b) {
  return a + b; // correct, but we will break it later
}
module.exports = { add };

// math.test.js
const { add } = require('./math');

test('adds 1 + 2 to equal 3', () => {
  expect(add(1, 2)).toBe(3);
});

Step 4: Introduce a bug

Now break the add function so the test fails. Change the plus to a minus. This simulates a real bug the agent must find and fix.

Run the test to confirm it fails.

cd test-project
# edit math.js to change + to -
sed -i 's/return a + b;/return a - b;/' math.js
npm test

Step 5: Run the agent

Now run the agent from the coding-agent root, pointing it at the test project. The agent will run npm test, see the failure, read math.js, and fix it.

Make sure the OpenAI API key is set in the environment.

cd .. # back to coding-agent
# set your API key
export OPENAI_API_KEY=your-key-here
# run the agent, passing the test command
npx tsx src/agent.ts "cd test-project && npm test"

Verify it worked

After the agent finishes, check the test again. If the agent found and fixed the bug, the test should pass. You can also inspect math.js to see what changed.

The agent may take a few iterations, but you should see it run the test, read the file, write a fix, and run the test again until green.

cd test-project
npm test
# Should output: Tests: 1 passed

Recommended setup for a real project

The minimal agent works, but for a real codebase you need more safety and structure. Here is what I would add before using this on anything important.

First, restrict the tools to a whitelist of safe commands. Second, use a separate git branch so the agent can be reverted. Third, add a timeout for each tool call and the whole run.

// Example of a safer run_test tool
export async function runTestSafe(command: string): Promise<string> {
  const allowedCommands = ['npm test', 'pytest', 'go test'];
  if (!allowedCommands.some(c => command.startsWith(c))) {
    return 'Error: command not allowed';
  }
  // Run with timeout
  return execSync(command, { timeout: 30000, encoding: 'utf-8' });
}

• Use a container or sandbox to run untrusted code
• Limit the agent to a specific directory and never allow absolute paths outside it
• Set a maximum number of iterations and a total time limit
• Log every tool call and result for debugging
• Require human approval for write_file or run_test commands that are not on the allowlist

Troubleshooting

If the agent does not fix the bug, check a few common issues. The most frequent problem is that the model does not have enough context because it only reads one file. You can add a tool to list files in a directory, or automatically include the project tree in the system prompt.

Another issue is that the agent may keep running the same test without reading the file. Make sure the system prompt explicitly says to read the relevant source file after seeing a failure.

Finally, if the model returns malformed JSON for tool arguments, catch the parse error and return it to the model so it can correct itself.

• Add a list_files tool to help the agent explore the codebase
• Include the failing test output in the first user message to give the agent a starting point
• Increase the max iterations if the bug is complex
• If the agent writes an incorrect fix, you can manually revert and try again with a more detailed prompt

FAQ

Answers to the questions that come up most often on this topic.

• Q: Can I use a different LLM provider? A: Yes, any provider that supports OpenAI-compatible tool calling works. Change the baseURL in the OpenAI client and use the appropriate model name.
• Q: How do I prevent the agent from running dangerous commands? A: Use a command allowlist and run the agent inside a Docker container or a sandboxed environment.
• Q: The agent keeps editing the same file incorrectly. What can I do? A: Give it more context by including the full file content in the tool result, and ask it to explain its reasoning before making changes.
• Q: Does this work with Python projects? A: Yes, just change the test command to pytest or similar, and ensure the agent can read Python files. The loop is language-agnostic.

Try it on code.live

You can experiment with the agent loop directly in your browser using the API Mock Data Generator to simulate tool responses, or use the JSON to YAML converter to format your tool definitions. These tools help you prototype the agent without setting up a full environment.