Dex, the Language Learning Camera
Dex is an AI-powered handheld camera that lets children learn languages by exploring their surroundings. Currently, 11 languages and 43 dialects are supported. A child snaps a photo; multimodal LLM identifies the object and launches a learning experience—often a short speaking challenges in the target language. The child advances by speaking; ASR/NLU interpret the utterance and select the next branch/experience. Children can also pick titles from an Interactive Story Library and progress by speaking at decision points. Each discovery creates a collectible flashcard that is scheduled for review in flashcard games. When an interest scene (think encyclopedia-worthy topics) is captured, Dex unlocks a story that incorporates multiple elements from the photo that unveils wonders of the real world. A Parent Copilot app records sessions, selects languages/dialects, surfaces pronunciation scores, goals, and empower parents to create their own stories and games for children to discover with Dex.