What makes it an agent?

An LLM in a loop that decides which tools to call and when to stop.

Two things determine whether this loop works well:
how you design the tools, and how you manage state.

What the model actually sees

Every provider needs a JSON schema for each tool. Same concept, slightly different shapes.

{ "type": "function",
  "function": {
    "name": "complete_todo",
    "description": "Mark done.\n
      USE WHEN: ...\n
      DO NOT USE WHEN: ...",
    "parameters": {
      "type": "object",
      "properties": {
        "todo_id": {"type": "string"}
      }
    }
  }
}

{
  "name": "complete_todo",
  "description": "Mark done.\n
    USE WHEN: ...\n
    DO NOT USE WHEN: ...",
  "input_schema": {
    "type": "object",
    "properties": {
      "todo_id": {"type": "string"}
    }
  }
}

# Write Python, get the JSON for free

@tool
def complete_todo(
    todo_id: str
) -> str:
    """Mark done.

    USE WHEN: ...
    DO NOT USE WHEN: ...
    """
    ...

# Type hints → parameters
# Docstring → description
# Works with OpenAI or Anthropic

A bad tool

@tool  # ← LangChain decorator: registers this function as a tool
def complete_todo(todo_id: str) -> str:
    """Mark a todo as completed."""  # ← becomes the tool's "description" in the schema
    todo = db.get(todo_id)
    todo.completed = True
    db.commit()
    return f"Done: {todo.title}"

This looks fine. Here's what goes wrong:

A good tool

@tool
def complete_todo(todo_id: str) -> str:
    """Mark a todo as completed.

    USE WHEN: "I finished it", "took care of that"
    DO NOT USE WHEN: observing or planning.
    Args:
        todo_id: Full UUID from list_todos. DO NOT make up UUIDs.
    """
    if not valid_uuid(todo_id):
        return "ERROR: not a UUID. Call list_todos first."
    todo = db.get(todo_id)
    if todo.user_id != config.user_id:  # config is passed via code, not the prompt
        return "ERROR: not your todo."
    if todo.completed:
        return "Already done."  # safe to call twice
    todo.completed = True
    db.commit()
    return f"Done: {todo.title}"

What changed — and why

Bind the minimum tool set

A tool you don't give the agent cannot be called. This is the strongest safety constraint — enforced by your code, not the model's good behavior.

# When creating an agent, you choose which tools to include
# (syntax varies by framework — concept is the same)
query_agent  = Agent(tools=[get_context, search])
task_agent   = Agent(tools=[get_context, create, complete])
chat_agent   = Agent(tools=[])  # no tools = just text responses

What is state?

In web apps, "state" is React state or a session cookie. In agents, it's simpler:

State is what the system remembers — and for how long.

Every piece of information your agent works with has a lifetime. Some things live for one turn, some for one session, some forever. The LLM itself stores nothing — it starts from zero on every call. You decide what to send it each time.

The LLM is stateless

It has no memory between calls. Every call sends the entire conversation as a list of messages. Same concept across all providers.

client.chat.completions.create(
  model="gpt-4o",
  messages=[
    {"role": "system",
     "content": "You are..."},
    {"role": "user",
     "content": "Add a todo"},
    {"role": "assistant",
     "tool_calls": [...]},
    {"role": "tool",
     "content": "Created!"},
  ],
  tools=[...]
)

client.messages.create(
  model="claude-sonnet-4-20250514",
  system="You are...",
  messages=[
    {"role": "user",
     "content": "Add a todo"},
    {"role": "assistant",
     "content": [tool_use...]},
    {"role": "user",
     "content": [tool_result...]},
  ],
  tools=[...]
)

# Same message types, any provider
messages = [
  SystemMessage("You are..."),
  HumanMessage("Add a todo"),
  AIMessage(tool_calls=[...]),
  ToolMessage("Created!"),
]

# model is defined elsewhere, e.g.:
# ChatOpenAI(model="gpt-4o")
# ChatAnthropic(model="claude...")
model.invoke(messages)

Three lifetimes

The trap: state in the prompt

system_prompt = f"""You are Todo.

Current todos:
{format_todos(get_all_todos())}

Help the user manage their list."""

system_prompt = """You are Todo.

Use list_todos to see current items.
Use create_todo, complete_todo to act.
Do NOT rely on prior context for IDs."""

What should go in the prompt

Not dynamic state — but context that helps the model do its job. Helper functions gather the values, the builder injects them.

# These run BEFORE the LLM call to collect
# context the model will need this turn

def get_today() -> str:
    """So the model can resolve 'tomorrow'."""
    return date.today().isoformat()

def get_user_name(user_id: str) -> str:
    """Personalize the response."""
    user = db.get_user(user_id)
    return user.display_name

def get_page_context(request) -> str | None:
    """Bias the model toward relevant intents.
    e.g. user is on the 'My Home' page."""
    return request.headers.get("X-Page-Context")

# Static sections — identical every call (cached)
ROLE = """You are Todo, a task assistant."""
RULES = """Do EXACTLY what the user asked..."""
EXAMPLES = """..."""

# Build: static first, dynamic last
def build_prompt(today, user_name,
                 page_context=None):
    parts = [ROLE, RULES, EXAMPLES]
    parts.append(f"Today is {today}")
    parts.append(f"User: {user_name}")
    if page_context:
        parts.append(
            f"User is on: {page_context}")
    return "\n\n".join(parts)

# Called at the start of each turn:
# prompt = build_prompt(
#     get_today(), get_user_name(uid),
#     get_page_context(req))

Four memory strategies — simple

Use the simplest that works. Here's what each one looks like when retrieved.

-- what you get back --

user:      "remind me to fix faucet"
assistant: [calls create_todo]
tool:      "Created: Fix faucet"
assistant: "Added it!"
user:      "what's on my list?"
assistant: [calls list_todos]
tool:      "- Fix faucet (due Thu)"
assistant: "You have one todo."
user:      "I fixed the faucet"
assistant: [calls complete_todo]
...

-- what you get back --

Summary of earlier turns:
User created a "Fix faucet" todo
due Thursday. Checked their list
and saw one pending item.

--- recent turns (verbatim) ---
user:      "I fixed the faucet"
assistant: [calls complete_todo]
tool:      "Completed: Fix faucet"
assistant: "Marked it done!"

Four memory strategies — advanced

For when you need facts across sessions or fuzzy recall.

-- what you get back --

{
  "user_id": "a1b2c3",
  "facts": [
    { "type": "preference",
      "value": "prefers morning reminders" },
    { "type": "pattern",
      "value": "batches chores on Saturdays" },
    { "type": "context",
      "value": "homeowner, two kids" }
  ]
}

-- query: "kitchen preferences" --
-- returns top matches + scores --

[0.92] "User said they prefer
       modern kitchen appliances"

[0.87] "User wants to renovate
       the kitchen before winter"

[0.81] "Asked about energy-efficient
       dishwashers last session"

Memory scoping — the pitfall

If you add vector memory, always scope your queries.

# Searches ALL memories in the database
results = vector_db.search(
    query="kitchen preferences",
    limit=3,
)  # ← no user filter!

# Only searches THIS user's memories
results = vector_db.search(
    query="kitchen preferences",
    limit=3,
    filter={"user_id": current_user_id}
)