🛡️ Real-Time Fraud Detection System

A full-stack, real-time machine learning pipeline that detects fraudulent financial transactions in milliseconds. This project ingests a high-throughput stream of transactions, processes them via Apache Kafka, scores them using a LightGBM model deployed with FastAPI, and visualizes the results on a live Streamlit dashboard.

![Dashboard Screenshot]

---

🚀 Key Features

• Real-Time Streaming: Simulates 20M+ transaction volume using a custom Producer and Apache Kafka.
• Low-Latency Inference: Deployed LightGBM model via FastAPI (REST) with sub-50ms response times.
• Imbalanced Data Handling: Trained on the PaySim dataset (0.2% fraud rate) using dynamic class weighting to achieve 99% Recall.
• Live Dashboard: Interactive Streamlit interface to monitor transaction traffic, flagged fraud, and estimated saved revenue.
• Containerized Infrastructure: Uses Docker Compose to orchestrate Kafka services without manual setup.

---

🏗️ Architecture

The system follows a decoupled microservices architecture:

1. Producer: Generates synthetic financial transactions and pushes them to the financial_transactions Kafka topic.
2. Kafka Broker: Central messaging backbone for real-time streaming.
3. Inference Service (FastAPI): Serves a trained LightGBM model for ultra-fast predictions.
4. Detector / Consumer: Listens to Kafka, sends data to the API, logs alerts.
5. Dashboard: Monitors stream activity and fraud alerts in real time.

---

🛠️ Tech Stack

• Language: Python 3.12
• ML Model: LightGBM (Gradient Boosting)
• Streaming: Apache Kafka (KRaft mode)
• API Framework: FastAPI + Uvicorn
• Visualization: Streamlit + Altair
• Containerization: Docker & Docker Compose

---

⚙️ Installation & Setup

1. Prerequisites

• Docker Desktop installed and running.
• Python 3.9+

2. Clone the Repo

git clone https://github.com/MananxRobin/fraud-detection-system.git
cd fraud-detection-system

3. Install Dependencies

pip install -r requirements.txt

Use confluent-kafka instead of kafka-python for Python 3.12.

4. Start Infrastructure (Kafka)

docker-compose up -d

Wait ~30 seconds for Kafka to initialize.

---

🏃‍♂️ How to Run (The “Control Room”)

Open 4 terminals:

Terminal 1 — Start FastAPI (Inference Service)

python app.py

Terminal 2 — Start Streamlit Dashboard

streamlit run dashboard.py

Terminal 3 — Start Fraud Detector

python detector.py

Terminal 4 — Start Producer

python producer.py

Then open http://localhost:8501 to view your live dashboard.

---

📊 Model Performance

Metric	Score	Notes
Accuracy	99.8%	High due to class imbalance
Recall (Fraud)	99%	Most important metric
Precision	~11%	Acceptable false positives for safety

---

📂 Project Structure

├── docker-compose.yml                            # Kafka infrastructure
├── requirements.txt                              # Python dependencies
├── README.md                                     # Documentation
├── app.py                                        # FastAPI inference service
├── dashboard.py                                  # Streamlit dashboard
├── detector.py                                   # Kafka consumer logic
├── producer.py                                   # Data stream simulator
├── train_model.py                                # Training & feature engineering
└── fraud_model.pkl                               # Saved LightGBM model

---

🔮 Future Improvements

• Database Integration (PostgreSQL) for audit trails
• Feature Store (Redis) for velocity-based real-time features
• Cloud Deployment using AWS ECS or Kubernetes

---

Author: Manan Ambaliya License: MIT