Where we left off

Last time: HITL — how to pause one agent safely.

Today: what happens when one agent isn't enough. Who decides which specialist runs?

The framing: workflow vs agent

Anthropic's taxonomy. Most teams now use the same words.

When do you need an agent?

Three signals the workflow isn't enough:

None of these? You don't need an agent. You need a function.

Why split at all?

One agent with too much to do will get worse.

The setup

One message in, one of four intents out, then a handler does the work.

Routers are cheap. Handlers that do real work run on smart-tier models. That's cost-aware routing.

Router 1 — Rules

Plain Python. No model. Sub-millisecond.

def route(msg):
    m = msg.lower().strip()
    if m in {"hi","hey","thanks"}:
        return "chitchat"
    if "refund" in m or "charge" in m or "invoice" in m:
        return "billing"
    if "password" in m or "log in" in m or "login" in m:
        return "account"
    if m.endswith("?"):
        return "technical"
    return "technical"  # safe default

Router 2 — Embeddings

Embed the intents once at startup. Compare the message against them.

# at startup — runs once
INTENT_VECS = {
    name: embed(desc)
    for name, desc in INTENT_DESCRIPTIONS.items()
}

# per message
async def route(msg):
    v = await embed(msg)
    scores = {n: cosine(v, iv)
              for n, iv in INTENT_VECS.items()}
    return max(scores, key=scores.get)

Why embeddings work — the 60-second version

An embedding is a list of numbers — a vector — that captures the meaning of a piece of text. Texts that mean similar things end up with similar vectors.

Router 3 — LLM classifier

Small, fast model. One job: return one label.

PROMPT = """Return ONE label, lowercase:
  account   - login, password, profile
  billing   - invoices, refunds, plans
  technical - how-to, bugs, integration
  chitchat  - greetings, thanks"""

async def route(msg):
    r = await llm.ainvoke([
        {"role":"system", "content": PROMPT},
        {"role":"user",   "content": msg},
    ], temperature=0, max_tokens=8)
    return validate(r.content)

Router 4 — Hybrid

Embeddings first. LLM only for the ambiguous cases.

async def route(msg):
    scores = embedding_scores(msg)
    best   = max(scores, key=scores.get)

    if scores[best] >= 0.55:          # confident → ship it
        return best

    # ambiguous → fall through to the LLM
    return await llm_classify(msg)

Same 12 messages, four routers

Accuracy. Latency. Cost. Side by side.

What the numbers say

The under-discussed win: sticky state

Sometimes the previous turn already made the answer obvious. The router doesn't need to think.

agent › Here are 3 slots:
         1. 2pm  2. 3pm  4pm
user  › the 3pm one
      // router weighs all 4 intents
      // gets it wrong 8% of the time

agent › Here are 3 slots:
         1. 2pm  2. 3pm  4pm
      // state: awaiting_slot_selection
user  › the 3pm one
      // router skipped. Handler: booking.

Wiring up sticky state

Two parts: prior state from the last turn, plus a lightweight pattern check on the current message.

async def dispatch(message, session):
    # 1. Did the prior turn set up a sticky flow?
    sticky = session.get("awaiting")  # e.g. "slot_selection"

    # 2. Does the message look like a continuation of that flow?
    if sticky == "slot_selection" and looks_like_slot_pick(message):
        return "booking"         # skip the router

    if sticky == "confirmation" and looks_like_yes_no(message):
        return session["pending_intent"]

    # 3. Otherwise, fall through to the normal router
    return await router.route(message)

Handlers set the sticky flag when they show a list; the next turn reads it.

Three ways to hand work off

Framework vocabulary is muddled. These shapes are different.

Picking a router

There is no "best" router. Pick trade-offs consciously.

What routing doesn't solve

Real users send messages that span two specialties.

"I was charged twice this month — can you also reset my password while you're at it?"

A strict single-intent router has to pick one. The other half of the message is lost.