What AI Coding Agents Still Get Completely Wrong

The promise vs. the reality

AI coding agents promise to take over the boring parts of software development. In practice they still stumble on the same problems again and again. You ask for a simple refactor and get a broken build. You ask for a new endpoint and it invents a library that does not exist. You approve a change and it deletes a file you did not mention.

This article is not about whether AI agents are useful. They are. It is about knowing exactly where they fall short so you can plan around it. I have spent the last few months using several agents on real projects and have collected the most common failure patterns. For each one I will show you a minimal reproduction and a workaround you can apply today.

Before you start

To follow along, you need a terminal with Node.js 18 or newer and git installed. The examples use the OpenAI API for the agent loop, but the same patterns apply to any LLM-based agent. You will also need an API key if you want to run the agent loop, but the failure analysis works without it.

node -v
npm -v
git --version

Step 1: The agent loop that misses context

The first thing agents get wrong is context. They see the file you point them at but ignore the surrounding project. They do not read the README, the package.json scripts, or the existing test conventions. The result is code that compiles but does not fit the project.

Here is a minimal agent loop that calls an LLM with only the current file. It will fail on any non-trivial task because the prompt lacks project context.

import OpenAI from 'openai';
import fs from 'fs';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function runAgent(filePath, task) {
  const fileContent = fs.readFileSync(filePath, 'utf8');
  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'system', content: 'You are a coding assistant. Modify the file as requested.' },
      { role: 'user', content: `Task: ${task}\n\nFile content:\n${fileContent}` }
    ]
  });
  return response.choices[0].message.content;
}

// Example: runAgent('src/index.js', 'Add error handling to the fetch call')

• The agent only sees the file, not the package.json, tsconfig, or test setup.
• It does not know the coding style or the existing patterns.
• It may suggest changes that break the build because it cannot see the dependencies.

Step 2: The fix that still fails

Even with context, agents still make mistakes. The next failure is more subtle: they do not verify their own output. They assume the code is correct without running tests or checking the type system.

#!/bin/bash
# Generate a context pack for the agent
{
  echo '## Project Structure'
  find . -type f -not -path './node_modules/*' -not -path './.git/*' | head -50
  echo '\n## package.json'
  cat package.json
  echo '\n## tsconfig.json'
  cat tsconfig.json
  echo '\n## README.md'
  cat README.md
} > context.txt

# Then include context.txt in the prompt

Step 3: The verification gap

Most agents do not run your test suite or linter before presenting a solution. They generate code that looks right but may have syntax errors or type mismatches. The workaround is to add a verification step to the agent loop. After the LLM returns code, run the tests and feed the errors back to the model.

import { execSync } from 'child_process';

function runTests() {
  try {
    execSync('npm test', { stdio: 'pipe' });
    return { success: true, output: 'All tests passed' };
  } catch (error) {
    return { success: false, output: error.stderr.toString() };
  }
}

// In the agent loop, after getting code from LLM:
const testResult = runTests();
if (!testResult.success) {
  // Send testResult.output back to the LLM for a fix
}

• Always run the project's test command after an agent edit.
• If tests fail, feed the exact error output back to the model.
• Do not trust the agent's claim that the code works; verify yourself.

Step 4: The hallucination of dependencies

If the agent suggests a package, run these commands. If the package does not exist, npm will error. If it is deprecated, you will see a warning. This simple check saves hours of debugging.

npm view <package-name> version
npm view <package-name> deprecated
npm view <package-name> dependencies

Step 5: The permission problem

When agents have too much freedom, they can make destructive changes. They might delete files, overwrite configs, or run commands without your knowledge. The solution is to use a permission system that requires approval for every action.

Here is a simple permission wrapper you can use with any agent. It prompts the user before executing a command.

import subprocess
import sys

def run_with_permission(command):
    print(f"About to run: {command}")
    response = input("Allow? (y/n): ")
    if response.lower() != 'y':
        print("Command skipped.")
        return
    subprocess.run(command, shell=True)

# Example usage
run_with_permission("rm -rf build")

• Always review commands that modify the filesystem or install packages.
• Use a separate git branch for agent changes so you can revert easily.
• Set a timeout for agent runs to avoid infinite loops.

Step 6: The testing blind spot

This forces the agent to think about the expected behavior before writing code. It also gives you a way to verify the solution.

Task: Implement the function `calculateTotal` in `src/calculate.js`.

First, write a test file `test/calculate.test.js` that covers these cases:
- Returns 0 for an empty array
- Returns the sum for a list of positive numbers
- Handles negative numbers correctly

Then, implement the function to make the tests pass.

Run `npm test` and show the output.

Step 7: The context window trap

Include only the relevant files in the prompt. This reduces the chance of the agent getting confused by unrelated code.

grep -rn "function calculateTotal" src/
grep -rn "calculateTotal" --include="*.js" --include="*.ts" .

What I would do: a recommended agent setup

Based on these failure patterns, here is a setup I recommend for using AI coding agents safely and effectively. It combines a permission wrapper, a context pack, and a verification step.

Create a script that runs the agent with these safeguards. Here is a starter script you can adapt.

#!/bin/bash
# agent-run.sh
set -e

# 1. Build context pack
./build-context.sh > context.txt

# 2. Ask the agent for a change (using your preferred agent CLI)
# Example with a generic CLI: agent "Fix the bug in src/index.js" --context context.txt

# 3. After the agent finishes, run tests
npm test

# 4. If tests fail, feed errors back to the agent and repeat
# (loop until pass or max iterations)

• Always run in a git branch so you can roll back.
• Use a permission prompt for every destructive command.
• Verify with tests, not just the agent's word.
• Keep the context small and focused.

Troubleshooting common agent failures

Here are quick fixes for the most common issues you will hit.

If the agent produces code that does not compile, check the TypeScript or lint errors and feed them back.

If the agent installs a wrong package, uninstall it and verify the correct one with npm view.

If the agent deletes a file, revert it with git checkout -- <file>.

• Use git diff to review every change before committing.
• Run npm run lint after an agent edit to catch style issues.
• If the agent goes off track, stop it and narrow the task.

FAQ

Can AI coding agents replace junior developers?

Not yet. They still need supervision, especially for context and verification.

How do I stop an agent from hallucinating?

Provide a clear context pack and verify all package suggestions with npm view.

Should I let an agent edit files directly?

Only if you have a permission system and a git branch to revert.

What is the best model for coding?

It depends on the task, but always test the output; do not trust any model blindly.

Next action

The next time you use an AI coding agent, add a verification step. Run your test suite after each change, and keep the context tight. Start with the script above and adapt it to your workflow.

Create a git branch, run the agent on a small task, and inspect the diff. You will see the difference immediately.