How to Build a Personal AI Developer Assistant

Before you start

You have been hearing about AI pair programmers, but you are not ready to send your code to a cloud service. You want a personal assistant that runs on your machine, respects your privacy, and costs nothing per token. This tutorial walks you through building a local AI developer assistant using Ollama, Continue, and a few custom tools.

By the end, you will have a working assistant that can answer questions about your codebase, generate boilerplate, and run simple commands. You will also have a foundation for extending it with your own tools and scripts.

node --version
npm --version
python3 --version
pip3 --version

• A computer with at least 16 GB of RAM (8 GB works but is slower)
• macOS, Linux, or Windows with WSL2
• Node.js 18+ and npm installed
• Python 3.10+ and pip installed
• Basic familiarity with the terminal

Step 1: Install Ollama

Ollama is a tool that lets you run open-source large language models locally. It provides a simple API and a command-line interface. Start by installing Ollama on your system.

After installation, verify it is working by pulling a small model. We will use 'llama3.2' for this tutorial because it is fast and works well on consumer hardware.

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2
ollama run llama3.2 "Say hello in one sentence"

• If you are on Windows, use WSL2 and run the same commands inside the Linux terminal.
• The first run will download about 2 GB, so be patient.
• If you have an NVIDIA GPU, Ollama will use it automatically for faster inference.

Step 2: Set up the Continue extension

Continue is an open-source AI code assistant that integrates with VS Code and JetBrains IDEs. It can connect to local models through Ollama. Install the Continue extension from the marketplace.

Once installed, you need to configure it to use your local Ollama model. Open the Continue configuration file by clicking the gear icon in the Continue sidebar, or by editing 'config.json' in your project's '.continue' folder.

{
  "models": [
    {
      "title": "Llama 3.2 Local",
      "provider": "ollama",
      "model": "llama3.2"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Llama 3.2 Local",
    "provider": "ollama",
    "model": "llama3.2"
  }
}

• After saving the config, restart VS Code to apply changes.
• You should see 'Llama 3.2 Local' in the model selector.
• If you have trouble, check the Continue logs in the output panel.

Step 3: Create a custom tool for your assistant

A personal assistant is more useful when it can run commands and scripts. We will create a simple Node.js script that acts as a tool: it can fetch the latest git commit message and generate a commit message suggestion. This script can be called by Continue through a custom command.

First, create a directory for your assistant tools and write the script.

mkdir -p ~/dev-assistant/tools
cd ~/dev-assistant/tools
npm init -y
npm install simple-git

const simpleGit = require('simple-git');
const git = simpleGit();

async function getCommitMessage() {
  try {
    const log = await git.log({ maxCount: 1 });
    const lastCommit = log.latest;
    if (!lastCommit) {
      console.log('No commits yet.');
      return;
    }
    console.log(`Last commit: ${lastCommit.message}`);
    // Suggest a conventional commit based on the diff
    const diff = await git.diffSummary(['HEAD~1', 'HEAD']);
    const filesChanged = diff.files.length;
    const suggestion = `feat: update ${filesChanged} files`;
    console.log(`Suggested commit message: ${suggestion}`);
  } catch (err) {
    console.error('Error:', err.message);
  }
}

getCommitMessage();

Step 4: Connect the tool to Continue

Continue allows you to define custom commands in your config. These commands can run shell scripts and capture the output. We will add a command called 'commit-helper' that runs our script.

Edit your Continue config.json to add a 'customCommands' array.

{
  "models": [
    {
      "title": "Llama 3.2 Local",
      "provider": "ollama",
      "model": "llama3.2"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Llama 3.2 Local",
    "provider": "ollama",
    "model": "llama3.2"
  },
  "customCommands": [
    {
      "name": "commit-helper",
      "prompt": "Run the commit helper script and use its output to help the user write a commit message.",
      "commands": [
        {
          "type": "run",
          "command": "node ~/dev-assistant/tools/commit-helper.js"
        }
      ]
    }
  ]
}

• The 'run' command executes the script and feeds the output back to the model.
• You can add as many custom commands as you like.
• Make sure the path to your script is correct.

Step 5: Build a simple agent loop with Python

If you want more control than Continue offers, you can build a minimal agent loop yourself. This Python script uses Ollama's API to chat with the model and execute simple shell commands when the model outputs a special marker.

This is a basic example, but it shows the core pattern: prompt, parse, act, repeat.

import requests
import subprocess
import json

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "llama3.2"

def ask(prompt):
    response = requests.post(OLLAMA_URL, json={
        "model": MODEL,
        "prompt": prompt,
        "stream": False
    })
    return response.json()["response"]

def run_command(cmd):
    result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
    return result.stdout + result.stderr

prompt = """You are a helpful assistant. If you need to run a command, output it inside triple backticks like this: ```ls```. Otherwise, just answer."""

while True:
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        break
    full_prompt = prompt + "\nUser: " + user_input
    response = ask(full_prompt)
    print("Assistant:", response)
    if "```" in response:
        cmd = response.split("```")[1]
        print("Running command:", cmd)
        output = run_command(cmd)
        print("Output:", output)
        # Send output back to the model for further reasoning
        follow_up = ask("The command output was: " + output)
        print("Assistant:", follow_up)

• This agent can execute arbitrary commands, so be careful with what you ask it to do.
• In a real application, you would add permission checks and sandboxing.
• You can extend this to use tools by defining a schema and parsing the model's output.

Verify it worked

After following the steps, you should be able to use the Continue sidebar to ask questions and get code suggestions from your local model. Your custom command 'commit-helper' should appear in the command palette.

To test the Python agent, run it in a directory with a git repository and ask it to suggest a commit message.

cd ~/dev-assistant/tools
python3 agent.py
You: suggest a commit message based on the last change
Assistant: ...

Troubleshooting

If you run into issues, here are common problems and fixes.

Ollama not responding: make sure the Ollama service is running. On macOS, you can start it with 'brew services start ollama'. On Linux, use 'systemctl start ollama'.

Continue cannot connect to Ollama: check that your config.json is valid JSON and that the model name matches exactly what you pulled with 'ollama list'.

Python agent fails with connection error: ensure Ollama is running and the URL is correct. You can test with 'curl http://localhost:11434/api/generate'.

• If the model is slow, try a smaller model like 'llama3.2:1b'.
• For GPU issues, check Ollama logs with 'ollama serve' in a terminal.
• If Continue shows 'No model available', reload the window.

What I would do

For a production setup, I would add a few things: a persistent history for the agent, a permission system for commands, and a way to index your codebase for better context.

Here is a recommended starter configuration for Continue that includes a codebase indexing feature (via embeddings) and a safer custom command setup.

{
  "models": [
    {
      "title": "Llama 3.2 Local",
      "provider": "ollama",
      "model": "llama3.2"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Llama 3.2 Local",
    "provider": "ollama",
    "model": "llama3.2"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  },
  "customCommands": [
    {
      "name": "explain",
      "prompt": "Explain the selected code in detail.",
      "commands": []
    }
  ]
}

ollama pull nomic-embed-text

FAQ

Answers to the questions that come up most often on this topic.

• Q: Can I use a different model? A: Yes, any model supported by Ollama works. Try 'codellama' or 'deepseek-coder' for better code generation.
• Q: How much does this cost? A: Nothing beyond electricity. All models run locally.
• Q: Is my code safe? A: Yes, no data leaves your machine unless you configure a cloud provider.
• Q: Can I add more tools? A: Absolutely. Create scripts and add custom commands in Continue, or extend the Python agent with more functions.

Next steps

You now have a personal AI developer assistant running on your machine. The next step is to integrate it into your daily workflow: use Continue for inline code completion, run the agent for quick questions, and expand your toolset.

Start by running the commit helper in one of your projects and see if the suggested message matches your style.

cd your-project
git add .
node ~/dev-assistant/tools/commit-helper.js