AI for Builders
A crash course in the concepts, patterns, and practical knowledge you need to build an AI-powered MVP.
What Actually Happens When You Use AI
Built on the Transformer architecture (2017).
Every AI tool you've used runs this pipeline:
1. Tokenize
Text → tokens → vectors
2. Attention
Each token weighs all others
3. Compute
Next token probabilities
4. Sample
Pick one token, append
5. Repeat
Loop until done
The Reality
No stored facts — only learned patterns
Context window = temporary memory only
Same input can give different outputs
It simulates language, not retrieves truth.
For Builders
Plausible ≠ correct (hallucination)
Small prompt changes → big output shifts
You shape behavior through structure
You're programming behavior, not querying knowledge.
LLMs are stateless — memory must be built around them.
The Context Window
Think of it as a desk.
You can only have so many papers on it at once.
The model can only see what's on the desk.
The Unifying Insight
Every architecture pattern we'll learn today — RAG, memory, tool use — exists to manage what goes on this desk. That's the whole game.
Tokenization: The Currency of AI
How Text Becomes Tokens
// A token is NOT a word. It can be: // - a whole word - part of a word // - punctuation - a space + word // "Tokenization is surprisingly tricky!" ["Token", "ization", " is", " surprising", "ly", " tricky", "!"] // 7 tokens // Common suffixes (-ing, -tion, -ly) = 1 token // Rare words get split into pieces: "ChatGPT" → ["Chat", "G", "PT"] // 3 tokens "🏠" → ["🏠"] // 1 token "$450,000" → ["$", "450", ",", "000"] // 4 tokens
Why You Should Care
- Cost — you pay per token, both input and output
- Limits — context window is measured in tokens
- Speed — more tokens = slower response
- Multilingual — non-English text uses more tokens per word
The Cost Reality
100 API calls with Claude Sonnet costs ~$0.30. Your entire hackathon's API usage will probably cost less than lunch. Don't let cost anxiety slow you down.
The API Call: It's Just HTTP
Strip away every framework and abstraction.
This is all it is.
import anthropic # pip install anthropic client = anthropic.Anthropic() # uses ANTHROPIC_API_KEY env var response = client.messages.create( # send request → get response model='claude-sonnet-4-20250514', max_tokens=1024, messages=[ {'role': 'user', 'content': 'What is machine learning?'} ] ) print(response.content[0].text) # extract text from response
Five lines of real code. Everything else — LangChain, LlamaIndex, frameworks — is a wrapper around this.
Prompt Engineering
Give the model distinct sections.
Each one constrains its output differently.
The Prompt — one effective pattern
# Your code builds this template once # {vars} are injected at runtime — dates, user data, docs ROLE: You are a senior backend engineer who writes clean, production-ready Python. CONTEXT: Stack: FastAPI + SQLAlchemy, PostgreSQL 15. Today's date: {date} User timezone: {timezone} Order model: id, user_id, total, status, created_at CONSTRAINTS: Max 50 results, support pagination, sort by created_at DESC, return empty list if none, use typed Pydantic response models. FORMAT: Respond with only Python code. No explanation.
# This changes every turn — the actual request I need an endpoint that returns a user's most recent orders. Something like GET /users/{user_id}/orders
Why Each Section Matters
Role
Sets expertise level and voice. "Senior backend engineer" produces different code than "helpful assistant."
Context
This can be your stack, your data, your user — whatever your code injects at runtime.
Task
One clear instruction. Ambiguity here is the #1 cause of bad outputs.
Constraints
Prevents over-engineering and scope creep. The model will add features you didn't ask for without these.
Format
Controls output shape. "Only code, no explanation" eliminates preamble you'd have to strip.
How the API Call Works
The left card shows how these sections map to the actual API call. Both are sent together on every request:
System message — the model's job description. The template stays the same, but your code injects variables at runtime — current date, user preferences, retrieved documents, etc. LLMs are stateless, so you re-send it on every call.
User message — the work ticket. This is the only part that changes per turn.
This split is a guideline. If context varies per request (e.g., different data), move it to the user message.
You can call an LLM and prompt it well.
But what about YOUR data?
The model only knows what's in the context window.
It hasn't read your docs, your database, or your product.
How do you give it that knowledge?
This is where embeddings, RAG, and memory come in.
Embeddings: Coordinates in Meaning Space
Turn text into numbers so "similar" things are nearby.
Like GPS coordinates, but for meaning.
"dog ran fast"
↔ "puppy sprinted quickly"
0.92
cosine similarity
"bake a chocolate cake"
↔ "recipe for brownies"
0.89
cosine similarity
"dog ran fast"
↔ "stock market crashed"
0.11
cosine similarity
RAG: Retrieval-Augmented Generation
Don't train the model on your data.
Just retrieve the relevant parts and put them in the prompt.
1. Query
User asks a question
user input
2. Embed
Convert query to vector
model
3. Search
Find similar chunks
deterministic
4. Augment
Inject into prompt
deterministic
5. Generate
LLM answers with context
LLM
When RAG Works
- Q&A over documents, knowledge bases, product data
- Data changes frequently (no retraining needed)
- You need the model to cite sources
When RAG Fails
- Bad chunking retrieves the wrong passage
- Answer sounds right but pulls from irrelevant context
- RAG quality is an Information Retrieval (IR) problem, not an LLM problem
RAG quality is a search problem, not an AI problem. Bad answers? Fix your retrieval first.
Structured Output: Making AI Predictable
The bridge between "cool demo" and "reliable feature."
Free Text Response
The product has a rating of about 4.5 stars. Most users liked it. Some complained about the weight. Price is $299.
- Output varies every time
- No consistent structure to parse
- Your code has to guess the format
JSON Structured Output
{
"rating": 4.5,
"pros": ["battery life", "display"],
"cons": ["weight"],
"price": 299,
"recommendation": "buy"
}
- Predictable shape every time
- Parse with
json.loads() - Feed directly into your UI or DB
Memory: Making AI Remember
LLMs are stateless. Every API call starts fresh.
If you want a conversation, YOU send the history.
1. Conversation Buffer
Send the full chat history every call. The list grows each turn.
2. Each call sends the entire list to the AI
3. AI responds based on the full conversation
4. You add both the new message and response to the list
5. List grows every turn — AI sees everything
Simple. Start here. Works until you hit context limits.
2. Summary Memory
When history gets long, ask the AI to compress it.
2. Ask the AI to summarize older messages
3. Cache the summary (a short text string)
4. Future calls send: summary + recent messages
5. Re-summarize periodically as conversation grows
Trades detail for token savings. Good for longer sessions.
3. Vector Recall
Don't send everything — retrieve only the relevant past turns.
2. When a new question arrives, embed it too
3. Find the most similar past turns (cosine similarity)
4. Send only those relevant turns + the new question
5. AI responds with context it actually needs
Same cosine math as RAG. Best for apps with long histories.
4. Structured Knowledge
Extract facts from conversations. Store in a database. Query as needed.
2. Store them in a database (preferences, goals, etc.)
3. On future calls, query the DB for relevant facts
4. Inject those facts into the prompt
5. AI responds with precise, persistent knowledge
Most complex. Best for production systems.
Tool Use: Letting AI Do Things
The model requests actions. Your code executes them.
The model never runs code directly.
1. Define the Tool
2. Define what inputs it accepts (JSON schema)
3. The model reads this to understand what's available
2. Pass Tools to the API
2. Model decides if a tool is needed
3. If yes: returns tool name + inputs
4. Your code executes the actual function
User
"Weather in Dallas?"
Model
Requests get_weather
Your Code
Runs the function
Model
Formats the response
The model requests. You execute. That's the safety boundary.
Model Selection: The Tradeoff Triangle
Quality
Best reasoning, most capable. Slower, costs more.
Speed
Fast responses, good enough for most tasks. Great for real-time UX.
Cost
Cheapest per token. Good for high-volume, simpler tasks.
Use Different Models for Different Tasks
Your summarizer doesn't need the same model as your reasoner. A well-prompted smaller model often beats a poorly-prompted frontier model. Mix and match.
AI Product vs. AI Feature
This changes your architecture, UI, error handling,
and demo strategy.
AI IS the Product
The entire UX is the AI interaction. The AI is front and center.
- AI chatbot / assistant
- Content generation tool
- AI tutor or coach
Focus: prompt quality, conversation design, response speed
AI is a Feature
A traditional app enhanced with AI capabilities. AI works behind the scenes.
- Dashboard with AI-generated insights
- App with smart search / recommendations
- Workflow tool with AI automation
Focus: integration quality, graceful fallbacks, UX consistency
Decide which one you're building before writing code. It shapes every decision that follows.
AI System Design Patterns
One primary pattern per project.
Extend only when needed.
Pick one core pattern
Answer questions about data
RAG + Chat
Q&A over docs, knowledge bases
Have a conversation
Chat + Memory
Assistants, tutors, support bots
Generate content
Prompting + Structured Output
Copy, summaries, reports, code
+ Layer extensions as needed
Tool Use
Let the AI take actions
Multimodal
Analyze images / docs
Agents (advanced)
Multi-step automation
You know enough to start.
You don't need to understand everything — you need to understand enough to make good decisions and build something real.
Quick Recap
LLMs predict tokens. The context window is your constraint. Prompting is your highest-ROI skill. RAG gives the model your data. Tools let it take actions. Test with 15 golden examples. Start simple. Ship it working.
Join the Slack
Connect with your team, mentors, and the community.
promptengineersai.slack.com