How to Give a Local AI Model Access to Your Files

The problem: your local model is blind

You installed a local LLM, but when you ask it to summarize a file or edit a config, it says it cannot access the filesystem. Most local AI models are stateless and have no built-in file access. To make them useful, you need to wire them to your files through a tool-calling loop.

This guide shows you how to give a local model safe, controlled access to your files using a minimal Python script, a sandbox directory, and a permission system. You will end with a runnable agent that can read and write files within a designated area.

Before you start

You need a local model that supports tool calling. Ollama with a model like llama3.1 or qwen2.5 works well. You also need Python 3.9 or newer and the requests library.

Install the dependencies and pull a model:

pip install requests
ollama pull llama3.1

• Ensure Ollama is running in the background.
• Test with a simple prompt before adding tools.

Step 1: Define the tool schema

Ollama's OpenAI-compatible API supports tool calling. You define a JSON schema for each tool. Here we define two tools: read_file and write_file. The model will call these with arguments like path and content.

Create a file named agent.py and start with the tool definitions.

tools = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read the contents of a file inside the sandbox.",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string"}
                },
                "required": ["path"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "write_file",
            "description": "Write content to a file inside the sandbox.",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string"},
                    "content": {"type": "string"}
                },
                "required": ["path", "content"]
            }
        }
    }
]

• The path must be relative to the sandbox root.
• The model will generate arguments as JSON.

Step 2: Implement the tool functions

Now implement the actual file operations. Use a sandbox directory to keep the model from touching anything outside. Resolve the absolute path and verify it stays inside the sandbox.

Add these functions to agent.py.

import os

SANDBOX = os.path.abspath("./sandbox")

def safe_path(path):
    full = os.path.abspath(os.path.join(SANDBOX, path))
    if not full.startswith(SANDBOX + os.sep):
        raise PermissionError("Path escapes sandbox")
    return full

def read_file(path):
    full = safe_path(path)
    with open(full, "r") as f:
        return f.read()

def write_file(path, content):
    full = safe_path(path)
    os.makedirs(os.path.dirname(full), exist_ok=True)
    with open(full, "w") as f:
        f.write(content)
    return f"Wrote {len(content)} bytes to {path}"

• Create the sandbox directory before running.
• The safe_path function prevents directory traversal.

Step 3: Build the agent loop

The agent loop sends the user prompt and tool definitions to the model. If the model responds with tool calls, execute them and feed the results back. Repeat until the model gives a final answer.

Add this loop to agent.py.

import requests
import json

OLLAMA_URL = "http://localhost:11434/v1/chat/completions"

def call_model(messages):
    payload = {
        "model": "llama3.1",
        "messages": messages,
        "tools": tools
    }
    resp = requests.post(OLLAMA_URL, json=payload)
    return resp.json()

def execute_tool(tool_call):
    name = tool_call["function"]["name"]
    args = json.loads(tool_call["function"]["arguments"])
    if name == "read_file":
        return read_file(**args)
    elif name == "write_file":
        return write_file(**args)
    else:
        return f"Unknown tool: {name}"

def agent_loop(user_message):
    messages = [{"role": "user", "content": user_message}]
    while True:
        response = call_model(messages)
        msg = response["choices"][0]["message"]
        messages.append(msg)
        if msg.get("tool_calls"):
            for tc in msg["tool_calls"]:
                result = execute_tool(tc)
                messages.append({
                    "role": "tool",
                    "tool_call_id": tc["id"],
                    "content": result
                })
        else:
            return msg["content"]

• The loop stops when the model returns a normal message.
• Tool results are appended as tool messages.

Step 4: Run it

Create the sandbox directory and test with a prompt that asks the model to write and read a file.

Run the script:

mkdir -p sandbox
python agent.py

• The script will prompt for input.
• Try: Write a short note to notes.txt, then read it back.

Verify it worked

After the run, check that the file was created inside the sandbox with the expected content.

Run these commands to confirm:

cat sandbox/notes.txt

• If the file exists, the agent successfully used the write_file tool.
• If you see a permission error, check that the path is relative.

Step 5: Add permissions and logging

In a real setup you want more control. Add a permission check that asks the user before write operations, and log all tool calls for auditing.

Update the execute_tool function:

def execute_tool(tool_call):
    name = tool_call["function"]["name"]
    args = json.loads(tool_call["function"]["arguments"])
    print(f"Tool called: {name} with args {args}")
    if name == "write_file":
        confirm = input(f"Allow write to {args['path']}? (y/n): ")
        if confirm.lower() != "y":
            return "Write denied by user"
    if name == "read_file":
        return read_file(**args)
    elif name == "write_file":
        return write_file(**args)
    else:
        return f"Unknown tool: {name}"

• Always log tool calls in production.
• Use a permission file to allow or deny paths without prompting.

Recommended setup for production

For a more robust setup, use a framework like LangChain or LlamaIndex that handles tool loops and permissions. But the core idea is the same: define tools, enforce a sandbox, and log everything.

Here is a minimal Dockerfile to run your agent in an isolated container:

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY agent.py .
RUN mkdir /app/sandbox
CMD ["python", "agent.py"]

• Mount a volume for the sandbox so files persist.
• Run with --read-only root filesystem for extra safety.

Troubleshooting

If the model never calls a tool, ensure the prompt clearly asks for file access and the tool schema is correct.

If you get a 404 from Ollama, check that the model is pulled and the API path is correct.

• Model not calling tools: try a model with stronger tool support, like llama3.1 or qwen2.5.
• Permission errors: verify the sandbox path is absolute and the file is inside it.
• Slow responses: tool-calling loops can take multiple round trips; use a smaller model for speed.

FAQ

• Can I give the model access to my whole home directory? Yes, but you risk accidental deletion or data exposure. Use a sandbox.
• Does this work with any local model? Only models that support tool calling (function calling). Check your model's documentation.
• How do I handle binary files? Read and write in binary mode, and base64 encode content for the model.
• Is it safe to run this on my personal machine? With a sandbox and permission prompts, it is reasonably safe, but always review tool calls.

Next step

You now have a working local AI agent with file access. Extend it with more tools like list_files or search, and integrate it into your daily workflow.

Run this command to try a more complex prompt:

python agent.py