Build an AI Agent That Updates Your Documentation
Learn to build a Python AI agent that detects code changes and automatically updates your project's docs.
Before you start
Your codebase changes every day, but your README and API docs fall behind. You find yourself manually updating the same sections over and over. This article shows you how to build a small AI agent that watches your code, detects changes, and rewrites the relevant documentation for you.
You will need Python 3.9 or later, a GitHub repository, and an OpenAI API key (or any LLM API). The agent will use the GitHub API to fetch diffs and the OpenAI API to generate doc updates. You will run everything locally, so no additional infrastructure is required.
python --version
pip install openai PyGithub pyyaml- Python 3.9+ installed
- A GitHub personal access token with repo scope
- An OpenAI API key (or compatible endpoint)
- A local clone of the repository you want to document
- Basic familiarity with git and command line
Step 1: Set up the environment
Create a new directory for the agent and set up a virtual environment. Then store your API keys in a .env file so they are not hardcoded.
mkdir doc-agent
cd doc-agent
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install openai PyGithub python-dotenv
echo "OPENAI_API_KEY=sk-..." > .env
echo "GITHUB_TOKEN=ghp_..." >> .envStep 2: Fetch the latest commit diff
The agent needs to know what changed. We will use PyGithub to get the latest commit and its diff. The diff will be the input to the LLM.
from github import Github
import os
g = Github(os.getenv('GITHUB_TOKEN'))
repo = g.get_repo('your-username/your-repo')
commit = repo.get_commits()[0]
diff = commit.files
for file in diff:
print(file.filename, file.status)
if file.patch:
print(file.patch[:500])Step 3: Build the prompt for the LLM
Now we construct a prompt that tells the LLM to update a specific documentation file based on the diff. We will focus on README.md, but you can extend it to any file.
def build_prompt(diff_text, doc_content):
return f"""
You are a documentation maintainer. Update the README below to reflect the code changes.
Only modify sections that are directly affected.
Code diff:
{diff_text}
Current README:
{doc_content}
Return the updated README.
"""Step 4: Call the LLM and apply changes
Use the OpenAI API to generate the new documentation. Then write it back to the file and create a commit.
import openai
openai.api_key = os.getenv('OPENAI_API_KEY')
def generate_updated_docs(diff_text, doc_content):
prompt = build_prompt(diff_text, doc_content)
response = openai.ChatCompletion.create(
model='gpt-4',
messages=[{'role': 'user', 'content': prompt}],
temperature=0.2
)
return response.choices[0].message.content
# Load current README
with open('README.md', 'r') as f:
current_doc = f.read()
# Generate updated content
new_doc = generate_updated_docs(diff, current_doc)
# Write back
with open('README.md', 'w') as f:
f.write(new_doc)Step 5: Automate with a cron job
To run this automatically, we will create a shell script that executes the agent, and schedule it with cron. The script will pull the latest changes, run the agent, and commit the updated README.
#!/bin/bash
cd /path/to/doc-agent
git pull origin main
python agent.py
git add README.md
git commit -m "docs: auto-update README"
git push origin mainRecommended setup
For a production-ready agent, I recommend using a GitHub Action instead of a cron job. This way, the agent runs on every push to the main branch, and you can review changes via pull request.
name: Update Docs
on:
push:
branches: [ main ]
jobs:
update-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.10'
- run: pip install openai PyGithub python-dotenv
- run: python agent.py
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- uses: peter-evans/create-pull-request@v5
with:
commit-message: 'docs: auto-update README'
branch: auto-docs
title: 'Automated documentation update'Verify it worked
After the first run, check the commit history to see the updated README. You should see the new commit with the message you set. Open the README and confirm that the changed functions or features are documented.
- Run the agent manually: python agent.py
- Check git log to see the new commit
- Review the README diff to ensure accuracy
- If the LLM output is poor, adjust the prompt or temperature
Troubleshooting
If the agent fails to authenticate, verify your tokens have the correct permissions. If the LLM returns invalid markdown, add instructions to the prompt to preserve formatting. If the agent overwrites unrelated sections, narrow the prompt to only touch specific sections.
- Check that GITHUB_TOKEN has repo scope
- Ensure OPENAI_API_KEY is valid
- Use a lower temperature (0.2) for more consistent output
- Limit the diff to relevant files by filtering file paths
FAQ
- Q: Can I use a local LLM instead of OpenAI? A: Yes, you can use any model that supports chat completion, such as Llama or Mistral, by changing the API endpoint and model name.
- Q: How do I handle multiple documentation files? A: Extend the agent to loop over a list of files and apply the same logic.
- Q: What if the diff is too large for the LLM context? A: Summarize the diff first or only include the changed hunks.
- Q: Can I run this on a schedule? A: Yes, use cron or GitHub Actions as shown.
- Q: How do I avoid commit loops? A: Add a check to skip the commit if the README content did not change.
Key takeaways
- Apply one concrete change from this post before collecting more reading.
- Prefer browser-side tools when the work involves secrets, tokens, or PII.
- Document the why next to the how so the next reviewer inherits context.
FAQ
- Who is this guide on ai-agent for?
- Working developers who need a practical take on build an ai agent that updates your documentation — not a marketing overview. Skim the sections, apply one tip, then come back when you hit an edge case.
- Do I need an account to use the related tools?
- No. code.live tools run in your browser with no signup. Nothing you paste is uploaded to a server for the client-side utilities linked from this post.
- How often is this article updated?
- This post was published November 10, 2026. Fundamentals stay stable; check linked tool pages and official docs when version-specific behavior matters.