ai-language-tutor
A voice-first English to Hindi tutor with memory, tests and evals.
- LiveKit
- Deepgram
- Cartesia
- FSM
- SQLite
See it working
Problem
Teaching is a harder conversation than answering. A receptionist agent completes a task and hangs up. A tutor has to run a lesson plan, switch between teaching, quizzing and clearing doubts, notice what you got wrong, and still remember it next week.
And the learner makes it harder: someone learning Hindi mixes Hindi and English in the same sentence. This build treats lesson structure, memory and bilingual speech as the core design problems, and then does the part most voice demos skip entirely: it tests the conversation.
Architecture
Decisions and trade-offs
1. The FSM owns the lesson, and every state has its own system prompt.
Five states decide what happens next; prompts.py gives each state its own instructions. The model is free to be natural inside a state but cannot wander off-curriculum, because curriculum is code, not vibes.
2. Four modes, one agent.
Teaching, Quiz, Conversation Practice and Doubt Resolution are separate handlers behind function tools. Ask a doubt mid-quiz and the agent switches mode, answers, and returns to the quiz where it left off.
3. Two kinds of memory, on purpose.
Short-term memory lives in-process for the session; long-term memory lives in SQLite across sessions. The next lesson starts from what you struggled with, not from scratch. Continuity is the product.
4. How do you test a conversation?
Split it. The deterministic parts get unit tests: 8 for FSM transitions, 10 for curriculum structure, 12 for memory. The conversation itself gets 7 scripted eval scenarios that drive the whole pipeline like a learner would. Every turn is logged as structured JSON, so a bad session can be replayed.
5. The cost of a lesson is written down.
A technical write-up documents latency measurements and per-session cost analysis before any talk of scaling. I shut a startup down over unit economics once; now I price systems while they are still small.
What I measured
30 unit tests: 8 FSM, 10 curriculum, 12 memory
7 scripted eval scenarios
per-turn structured JSON logs
written latency and cost analysis per session