Teacher's Pet — Adaptive AI Tutor for Kids

Overview

Teacher's Pet is a real-time, voice-first AI tutoring platform for kids aged 5–11. The system profiles each learner, generates a personalized Student Knowledge Profile (SKP) with an LLM, and classifies them into 10 learner archetypes to drive a tutor that actually speaks back, listens, and adapts mid-session.

I shipped the frontend at Algotix AI — the audio architecture, avatar engine, persistence layer, and AWS infrastructure via SST.

Audio Architecture

The platform standardized on ElevenLabs Text-to-Speech for output and the ElevenLabs SDK for capture and transcription. Earlier prototypes used WebRTC and OpenAI Realtime; both were removed because the single-vendor pipeline was simpler, faster, and easier to reason about under a single isSpeaking Zustand store.

User mic ─▶ ElevenLabs SDK (capture + transcription)
         ─▶ message stream ─▶ LLM (OpenAI / Vercel AI SDK)
         ─▶ ElevenLabs TTS ─▶ <audio> playback
         ─▶ avatarStore.isSpeaking → Rive state machine

• Transcription / messaging — speech is captured by the ElevenLabs SDK and emitted as text messages.
• TTS — assistant responses are rendered through ElevenLabs and streamed into a plain <audio> element.
• State — avatarStore.ts (Zustand) owns isSpeaking and feeds the Rive avatar state machine over WebSocket.

Architecture

Highlights

4 ElevenLabs Voice Agents + Rive Avatar

Integrated four ElevenLabs voice agents (each with a distinct persona) and bound their isSpeaking state to a Rive state-machine avatar running on @rive-app/webgl2. The avatar's mouth, eye, and gesture states animate in lock-step with the active voice agent over a WebSocket-backed event channel, with VAD (@ricky0123/vad-react) gating capture so the model never tries to speak over a child mid-thought.

Cross-Device Persistence on Redis

Selected slices of the Zustand store (progress, preferences) sync to Redis via a remotePersist wrapper:

• Key shape user:{userId}:state, value is a JSON blob with an updatedAt timestamp.
• Writes are debounced 1s, hydrate is local-storage first → remote merge.
• Last-writer-wins based on updatedAt, enforced atomically by a Redis Lua script.
• 90-day TTL keeps inactive accounts from accumulating in the cache; the long-term tier moves to Postgres JSONB.

The result: a kid can switch from the iPad to the family laptop and pick up the same lesson at the same checkpoint.

Resumable S3 Uploads

Long voice segments and artifact uploads use multipart resumable uploads to S3 with @aws-sdk/s3-request-presigner, so flaky home Wi-Fi doesn't kill a 5-minute capture.

Auth via ElectroDB on DynamoDB

Migrated NextAuth's auth layer onto a type-safe ElectroDB entity model over DynamoDB:

• User entity with the existing EmailIndex-index GSI preserved for email lookups.
• auth-utils.ts centralizes bcrypt password hashing, registration, and email/password verification.
• NextAuth keeps its JWT session strategy; only the data layer changed.

Adaptive Personalization

The tutor profiles each child by learning style and interests, then routes the conversation through one of 10 learner archetypes — auto-generated Student Knowledge Profiles keep the dialogue aligned to where the child actually is, not where the curriculum thinks they should be.

Tech Stack