Build an AI Agent That Reads Your Slack Messages

The Problem: Your Team's Knowledge Is Trapped in Slack

Every day, your team asks the same questions in Slack: 'What's the deploy process?', 'Where's the API key?', 'Who owns this service?'. The answers are buried in threads, DMs, and files. Building an AI agent that reads your Slack messages can surface that knowledge instantly, answer questions, and even take actions like creating Jira tickets or fetching logs.

In this tutorial, you'll build a Slack AI agent from scratch. It will listen to messages in a channel, fetch conversation history, and use an LLM to generate responses. You'll set up the Slack app, write the agent logic in Python, and deploy it locally with ngrok. By the end, you'll have a working agent that answers questions based on your team's Slack history.

Before You Start

You'll need a few things: a Slack workspace where you can create apps, Python 3.9 or later, and an OpenAI API key (or any LLM API you prefer). You'll also need ngrok for local development to receive Slack events.

Here's what we'll use: the Slack Events API to receive messages, the Slack Web API to fetch history and post replies, and LangChain to orchestrate the agent loop with tools. We'll store conversation history in memory for context.

• Python 3.9+ installed
• Slack workspace with admin rights to install apps
• OpenAI API key (or another LLM provider)
• ngrok account and CLI installed
• Basic knowledge of Python and REST APIs

Step 1: Create a Slack App and Configure Events

Go to api.slack.com/apps and click Create New App. Choose From scratch, name it, and select your workspace. Then, under Add features and functionality, enable Event Subscriptions and set the Request URL to your ngrok URL (we'll set that up in Step 3).

Subscribe to the message.channels event to receive messages from public channels. Also add the following Bot Token Scopes: channels:history, channels:read, chat:write, and groups:history. Install the app to your workspace and copy the Bot User OAuth Token.

# Install the Slack SDK and LangChain
pip install slack-sdk slack-bolt langchain langchain-openai python-dotenv

• Use the Bot User OAuth Token (starts with xoxb-) for API calls.
• Keep the token in a .env file, never commit it.
• Request URL must be HTTPS and publicly accessible (ngrok provides this).

Step 2: Set Up the Project Structure

Create a project directory with a virtual environment. We'll organize the code into a single app.py for simplicity, but you can split it into modules for larger projects.

mkdir slack-agent && cd slack-agent
python -m venv venv
source venv/bin/activate
touch app.py .env

Step 3: Write the Slack Event Handler

We'll use Slack Bolt to handle events. The app will listen for message events, ignore messages from bots, and call a function to generate a reply. We'll also fetch the conversation history to provide context to the LLM.

# app.py
import os
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
from dotenv import load_dotenv

load_dotenv()

app = App(token=os.environ["SLACK_BOT_TOKEN"], signing_secret=os.environ["SLACK_SIGNING_SECRET"])

@app.event("message")
def handle_message(event, say):
    if event.get("bot_id"):
        return
    text = event.get("text", "")
    channel = event["channel"]
    # Fetch history and generate response (we'll implement later)
    response = generate_response(text, channel)
    say(response)

if __name__ == "__main__":
    handler = SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"])
    handler.start()

• Socket Mode avoids needing a public endpoint for events; you can use it for development.
• You need an App-Level Token with connections:write scope for Socket Mode.
• For production, use the Events API with HTTP endpoints instead.

Step 4: Implement the Agent Logic with LangChain

Now we'll create the generate_response function. It will fetch recent messages from the channel, build a prompt with that history, and call an LLM to generate a response. We'll use LangChain's ChatOpenAI for the LLM and a simple prompt template.

# Add to app.py
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from slack_sdk import WebClient

client = WebClient(token=os.environ["SLACK_BOT_TOKEN"])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.3)

def fetch_channel_history(channel_id, limit=20):
    result = client.conversations_history(channel=channel_id, limit=limit)
    messages = result["messages"]
    # Format as "user: text"
    formatted = []
    for msg in reversed(messages):
        if "bot_id" in msg:
            continue
        user = msg.get("user", "unknown")
        text = msg.get("text", "")
        formatted.append(f"{user}: {text}")
    return "\n".join(formatted)

def generate_response(user_text, channel_id):
    history = fetch_channel_history(channel_id)
    prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a helpful assistant that answers questions based on the Slack conversation history provided. Be concise and factual."),
        ("human", "Conversation history:\n{history}\n\nUser question: {question}\nAnswer:")
    ])
    chain = prompt | llm
    response = chain.invoke({"history": history, "question": user_text})
    return response.content

• The history is limited to 20 messages to control token usage.
• You can extend this to search across channels or use a vector store for long-term memory.
• The model responds only based on the given history; you can add tools for actions.

Step 5: Run the Agent Locally

Create a .env file with your tokens. Then run the app. It will start a Socket Mode connection and listen for messages.

python app.py

• Make sure your Slack app has the correct scopes and is installed to the workspace.
• If you use Socket Mode, you don't need ngrok for this step.
• Test by sending a message in a channel where the bot is present.

Step 6: Add a Tool to Take Actions

Then bind the tool to the agent and let the LLM decide when to use it. In the generate_response function, create an agent executor.

# Update generate_response to use tools
from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Use tools when necessary."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}")
])

tools = [create_jira_ticket]
agent = create_openai_tools_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

def generate_response(user_text, channel_id):
    history = fetch_channel_history(channel_id)
    # Combine history into the input
    input_text = f"Conversation history:\n{history}\n\nUser question: {user_text}"
    result = executor.invoke({"input": input_text})
    return result["output"]

• The LLM will decide when to call the Jira tool based on the user's request.
• You need to set JIRA_URL, JIRA_TOKEN, and JIRA_PROJECT in .env.
• Always validate tool inputs to avoid security issues.

Verify It Worked

Invite the bot to a channel and send a message like: 'What did we say about the deploy process last week?' The agent should fetch history and answer. To test the Jira tool, say: 'Create a Jira ticket to fix the login bug'. It should create a ticket and reply with the key.

• Check the terminal logs for LangChain's verbose output showing tool calls.
• If the bot doesn't respond, verify the event subscription and scopes.
• Check that the bot is in the channel and has permission to read messages.

What I Would Do: Recommended Setup for Production

For a production deployment, I'd use the HTTP Events API with a server like FastAPI, and deploy to a cloud platform. I'd also add a vector store to index historical messages for better retrieval, and implement rate limiting and permission checks.

# Example Dockerfile for deployment
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

• Use a database or vector store (like Pinecone) for long-term memory.
• Add authentication to verify Slack requests using the signing secret.
• Set up monitoring and logging for the agent's actions.

Troubleshooting

Answers to the questions that come up most often on this topic.

• Bot not receiving messages: Check event subscription and that the bot is in the channel.
• Permission errors: Ensure all required scopes are added and reinstall the app.
• OpenAI API errors: Check your API key and quota.
• Socket Mode connection issues: Verify the App-Level Token and that you're using the correct handler.

FAQ

Answers to the questions that come up most often on this topic.

• Can this read private channels? Yes, if you add the groups:history scope and the bot is invited to the channel.
• How do I limit the history to avoid token limits? Use the limit parameter and consider summarizing older messages.
• Can I use a different LLM? Yes, LangChain supports many providers; just change the ChatOpenAI class to another.
• Is this secure? Be careful with sensitive data; you can add filters to avoid responding to confidential messages.