VisionAI - Real-Time Multimodal Vision System

VisionAI is a cutting-edge real-time video analysis application powered by Google's Gemini 2.5 Flash model. It transforms standard video feeds into intelligent data streams, capable of detecting objects, understanding complex scenes, and responding to natural language queries in real-time.

Designed for developers, robotics enthusiasts, and AI researchers, VisionAI provides a robust interface for experimenting with multimodal LLM capabilities on live video data.

🚀 Features

• 🎥 Real-time Video Analysis: Seamlessly connects to any video input source (webcams, USB cameras) for live processing.
• 🧠 Gemini 2.5 Flash Integration: Leverages Google's latest multimodal model for high-speed, accurate object detection and scene reasoning.
• 🎯 Dynamic Object Detection: Draws bounding boxes around detected objects based on natural language prompts.
• 💬 Natural Language Control: Instruct the AI what to look for using plain English (e.g., "Find all the safety hazards" or "Track the red ball").
• 📊 Live Session Analytics: Real-time monitoring of:
  • Input/Output Token usage
  • Request Latency
  • Estimated API Costs
• 🔋 Performance Optimized: Adjustable capture intervals and rotation settings to balance performance and granular analysis.

🛠️ Tech Stack

• Framework: React 19
• Build Tool: Vite
• Language: TypeScript
• Styling: Tailwind CSS
• AI Provider: Google Generative AI SDK

📂 Project Structure

The codebase is organized for clarity and scalability:

VisionAI/
├── src/
│   ├── App.tsx                 # Main application controller and state management
│   ├── components/
│   │   ├── VideoFeed.tsx       # Handles video rendering and bounding box overlays
│   │   └── ControlPanel.tsx    # Sidebar UI for settings, prompt input, and stats
│   ├── services/
│   │   └── geminiService.ts    # Direct integration with Google's Gemini API
│   └── types.ts                # Shared TypeScript interfaces for rigorous type safety
└── ...

🚀 Getting Started

Prerequisites

• Node.js: (Version 18+ recommended)
• Gemini API Key: Get one from Google AI Studio.

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/vision-ai.git
   cd VisionAI

2. Install dependencies:

   npm install

3. Configure Environment: You can enter your API key directly in the UI, or set up a local environment variable (optional).

4. Run the Application:

   npm run dev

   Open your browser to http://localhost:5173.

📖 Usage

1. Select Camera: Choose your video input source from the sidebar.
2. Enter API Key: Input your Google Gemini API key.
3. Define Prompt: Type what you want the AI to see or track (Default: "Describe what you see and track any interesting objects.").
4. Start Mission: Click "START MISSION" to begin real-time analysis.
5. Monitor Stats: Watch the live feed for visual overlays and the session report for usage metrics.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

This project is open source and available under the MIT License.