Real-Time Multimodal AI · Camera + Voice

Intelligent
Health Agent
in Real-Time

Aidora is a live AI agent that observes visual symptoms through your camera and listens to your voice instantly classifying medical urgency and delivering structured, safety first guidance.

Aidora Live Session
LIVE
30 FPS · HD
Urgency Assessment
Voice Input
AI Conversation
"Rash on my arm, 3 days, very itchy"
I can see redness and inflammation. Looks like contact dermatitis. No emergency see a dermatologist within 48 hrs.
Real-Time
Visual Analysis
Voice + Camera
Multimodal Input
Google Cloud
Powered Deployment
0 Diagnosis
Assessment Only · Safe by Design
Live Demo

See Aidora in Action

Watch the real-time agent analyze visual symptoms and engage in a live voice conversation.

Process

How Aidora Works

Multi-round autonomous reasoning visual observation, voice context collection, and intelligent urgency classification.

01
📸

Camera Stream Active

Continuous live frames via MediaDevices API analyzing skin conditions, wounds, bruising, swelling, or medication labels in real time without manual capture.

02
🎙️

Voice Conversation

Web Speech API transcribes the user's spoken description. Aidora asks intelligent follow-up questions to gather symptom duration, severity, and context.

Capabilities

Core Features

Everything that makes Aidora an intelligent first response health assistant.

🎤

Natural Voice Interaction

Bidirectional voice pipeline speech-to-text input, AI-generated contextual questions, and text-to-speech response output for hands-free assessment.

🔑

Google API Integration

Powered by Google APIs for seamless authentication, secure data access, and reliable cloud service connectivity throughout the session.

👁️

Real-Time Frame Processing

Real-time frame-by-frame processing identifies rashes, lacerations, swelling, burns, and medication labels with no manual capture needed.

🛡️

Safety-First Architecture

Hard-coded guardrails prevent diagnostic overreach. Every response reinforces that Aidora performs assessment not diagnosis and encourages professional care.

System Design

Architecture Diagrams

Three views of the system high level overview, full component detail, and step-by-step sequence flow.

Simple Overview
High-level system map inputs, AI core, and outputs at a glance.
View Diagram
Sequence Diagram
Step-by-step request and response flow from camera input to AI output.
View Diagram
Documentation

Project Documentation

Everything you need to understand, run, and extend Aidora.

🚀
Getting Started
Clone, set up, install dependencies and run locally in minutes.
→ View on GitHub
Backend API
FastAPI server, WebSocket endpoints, Google API integration, and urgency logic.
→ Read Docs
⚛️
Frontend App
React components for camera stream, voice input, urgency display, and conversation UI.
→ Read Docs
📊
Architecture Guide
Full system design, data flow, AI agent reasoning pipeline and component map.
→ Read Docs
Deployment Guide
Deploy frontend to Vercel and backend to any cloud with step-by-step instructions.
→ Read Docs
Built With

Tech Stack

Modern, production-grade technologies powering the real time multimodal pipeline.

⚛️
React.js
Frontend UI
FastAPI
Backend API
🔑
Google APIs
Auth & Services
☁️
Google Cloud
Deployment & Infrastructure
✦ Team Genix

We are a focused team of four skilled individuals building a Real-Time Health Agent a multimodal, safety-first assistant that combines camera vision and voice to assess minor medical concerns.

Ayesha
Ayesha
Team Lead
Hizar Abdullah
Hizar Abdullah
Backend Developer
Mahaz Noor
Mahaz Noor
Frontend Developer
Muhammad Faraz
Muhammad Faraz
Systems Architect
⚠️
Medical Disclaimer: Aidora is a health assessment tool only. It does NOT diagnose diseases, prescribe treatment, or replace licensed medical professionals. It evaluates urgency levels to support informed decisions about seeking timely care. Always consult a qualified healthcare provider for any medical concern.

Explore the Full Project

Source code, documentation, and architecture all on GitHub. Star it, fork it, build on it.

⭐ Star on GitHub 🌐 Visit Website ▶ Watch Demo