TED the Bear: Building a Socially Aware Therapy Robot

Winning 1st place out of 186 teams at the 2025 Utah AI Hackathon was an incredible milestone. Our project, TED the Bear, was a full-stack AI therapy robot designed to provide emotional support through physical and verbal interaction.

How it Works

TED doesn't just talk; he "feels." By combining computer vision with LLMs, the robot can tailor its responses to the user's emotional state.

Architecture:

• Vision: OpenCV + DeepFace on a Raspberry Pi Camera to detect user expressions and emotions.
• Speech: Whisper AI for speech-to-text and OpenAI (RAG) for empathetic counseling responses.
• Tactile Interaction: Arduino + Stepper Motors to control physical "hugging" gestures based on ultrasonic depth sensors.
• Digital Twin: A software avatar that lip-syncs in real-time to the generated audio (via Wav2Lip).

Tech Stack

• Languages: Python (FastAPI), JavaScript (React + Vite)
• Embedded: Raspberry Pi 4B, Arduino, Stepper Motors
• Cloud: Supabase (PostgreSQL) for real-time data sync between hardware and software.

Building a hardware-integrated AI agent in just three days tested every facet of our engineering skills.