HomeViewAI is an interactive property exploration platform designed to solve the "static listing" problem. Instead of scrolling through photos and long descriptions, users can talk to a property. Using Computer Vision (Twelve Labs) and Large Language Models (GPT-4o), the platform indexes property walk-through videos, allowing for room-by-room navigation and semantic visual search.
- Semantic Video Search: Users can ask, "Does the kitchen have an island?" and the video automatically seeks to the exact timestamp where the island is visible.
- AI Room Indexing: Automatically generates a room-by-room timeline from raw video footage using Twelve Labs' computer vision, enabling structured navigation.
- Avatar Mode: An immersive interactive experience where users can speak directly with a photorealistic AI avatar (powered by Simli & Deepgram) to get a guided tour of the property.
- Location-Based Queries: Integrated Google Maps API allows users to ask about the surrounding area, such as "How far is the nearest school?" or "Is there a park nearby?"
- Agent Review Dashboard: A dedicated interface for real estate agents to review AI-generated insights, manually adjust room timelines, and verify listing accuracy before or after it goes live.
- Preference-Based Ranking: A custom matching algorithm that evaluates property data against user preferences (budget, neighborhood, must-haves) to provide a transparent "Match Score."
- Multilingual Support: Real-time language detection and translation for international buyers, powered by Google Cloud Translation.
- Conversational Assistant: A context-aware chatbot that combines listing data, seller notes, and video observations to answer deep technical questions about a home.
The project is built with a decoupled architecture to handle high-latency AI processing:
- Frontend: React + Tailwind CSS.
- Backend: FastAPI (Python) for high-performance asynchronous operations.
- Video Intelligence: Twelve Labs API for video indexing and semantic analysis.
- Cognition: OpenAI GPT-4o-mini for natural language processing and summary generation.
- Infrastructure:
- AWS S3: Scalable media storage for videos and property images.
- Amazon DynamoDB: NoSQL storage for listings, user profiles, and chat sessions.
- Audio/Visual: Deepgram (TTS) and Simli (AI Avatar) for immersive voice interactions.
├── backend/
│ ├── services/ # TwelveLabs, OpenAI, DynamoDB, S3 integrations
│ ├── main.py # FastAPI routes and AI orchestration
│ └── models.py # Pydantic data schemas
└── frontend/
├── src/
│ ├── components/ # VideoPlayer, ChatInterface, MatchBadge
│ ├── hooks/ # usePropertyMatch, usePreferences
│ └── pages/ # AppView, CreateListing, Processing
Developed as a Capstone Project to demonstrate the practical application of Multimodal AI in the Real Estate industry.