Canbus Anomaly Detection EV - Two-Tiered Hybrid Intrusion Detection System

System Overview

This project implements a state-of-the-art Hybrid Intrusion Detection System (IDS) specifically designed for Electric Vehicle (EV) Controller Area Networks (CAN). It utilizes a distributed architecture that marries a lightweight O(1) Edge hardware tripwire with a mathematically stable Cloud/Edge-Compute inference server.

1. Tier-1 (Edge / ESP32): A resource-constrained ESP32 microcontroller running an ultra-fast, O(1) zero-math Shannon Entropy algorithm coupled with a 1D Kalman Filter. It acts as an instant hardware-level interrupt to block high-frequency volumetric attacks (DoS, Fuzzing) with near-zero latency, while simultaneously streaming raw window telemetry to Tier-2 over TCP.
2. Tier-2 (Cloud / Inference Server): A heavy-duty Python inference server running continuous Deep Packet Inspection (DPI) on the telemetry stream. It utilizes a Static Payload Content-Based Isolation Forest (PCB-iForest) fused with a Bayesian NLOG_PROB safety layer to catch stealthy payload mutations without suffering from incremental ML collapse. It also employs a deterministic Time-Travel (TIME_DELTA) Heuristic to identify Replay attacks.

---

Architecture & Step-by-Step Workflow

How the Hybrid System Works (The Data Pipeline)

Step 1: Edge Ingestion & Zero-Math Entropy

• The vehicle's CAN bus streams traffic to the ESP32 via UART/CAN transceiver.
• The ESP32 collects frames into sliding windows (W=128).
• The ESP32 calculates the Shannon Entropy of the CAN IDs in the window using an ultra-optimized precomputed Lookup Table (LUT), completely avoiding floating-point logarithms.

Step 2: Dynamic Kalman Filter & Hardware Interrupt (Tier-1)

• The ESP32 maintains dynamic Upper and Lower boundaries for normal entropy, generated natively via a 1D Kalman Filter.
• If a DoS attack occurs, the bus is flooded with a single priority CAN ID, crashing the entropy below the lower bound.
• If a volumetric Fuzzy attack occurs, the bus is flooded with random garbage IDs, spiking the entropy above the upper bound.
• The Hardware Interrupt: If the bounds are breached, the ESP32 drops the Kalman Gain to prevent learning the attack, triggers a physical hardware alert (LED pulse / interrupt), and transmits an alert to Tier-2.
• Continuous Streaming: The ESP32 continuously streams telemetry windows (and alerts) to the Cloud Server via TCP.

Step 3: Cloud Deep Packet Inspection (Tier-2 Fusion)

Because stealthy payload attacks do not alter structural CAN ID entropy, the Cloud continuously performs Deep Packet Inspection (DPI) on the incoming stream:

• Static PCB-iForest: Extracts physical inertia features (DATA_DIFF) from the payloads to isolate mechanical anomalies in the actuators. Deploying a static ensemble prevents mathematical collapse on zero-variance EV traffic.
• Bayesian NLOG_PROB Layer: Acts as a deterministic safety net. It independently scores every payload byte against a historically profiled empirical probability map. If a subtle Fuzzy payload mutation bypasses the static trees, the Bayesian layer safely overrides the assessment and flags the anomaly.

Step 4: The Impossible-Physics Replay Heuristic

• Replay Attack Detection: Hackers inject historically recorded sequences that cycle normally, preserving natural entropy and bypassing payload variance checks.
• Because replayed data cycles normally, the true invariant is a violation of temporal physics. By tracking the exact interval between matching CAN frames (TIME_DELTA), the Cloud detects historical pastes. If a sequence of frames jumps backward in time (t_delta < -100 ms), the Cloud deterministic safety net flags a Replay Attack.

---

How to Run the System & Dataset Selection Guide

Dataset Selection Guide: Which Logs to Use

To achieve the best results depending on your execution mode, use the recommended log files:

1. For Offline ML Evaluation & Google Colab:

   • Recommended Path: PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/test/Colab_test_files/
   • Recommended Files:
     • test_ZE0w24kWh_DoS.log
     • test_ZE0w24kWh_Fuzzy.log
     • test_ZE0w24kWh_Replay.log
     • test_ZE0w24kWh_Spike.log
     • (or the AZE0 equivalents for the 24kWh Generation-2 LEAF)
   • Why: These files have augmented attack density and concentrated anomaly patterns designed to generate rich, highly visual evaluation plots (e.g., ROC curves, Confusion Matrices, and high-contrast anomaly distribution figures in Jupyter/Colab).

2. For Real-Time Hardware-In-The-Loop (HIL) & GUI Simulation:

   • Recommended Path: PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/test/ (Regular Test Logs)
   • Recommended Files:
     • test_ZE0w24kWh_DoS.log
     • test_ZE0w24kWh_Fuzzy.log
     • test_ZE0w24kWh_Replay.log
     • test_ZE0w24kWh_Spike.log
     • Train Baseline: PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/ZE0w24kWhstock.log
   • Why: These logs represent continuous, realistic EV traffic cadence (~200 frames/sec). They are optimized for physical UART synchronization and sliding-window buffering without overflowing microcontroller serial buffers.

---

Option A: Offline ML Evaluation (Google Colab / Jupyter Notebook)

For algorithmic evaluation and paper replication without physical hardware:

1. Google Drive Setup (for Google Colab):

   • Upload the project (specifically the PC_SIDE folder and colab_evaluation.ipynb) to your Google Drive.
   • Recommended path: MyDrive/EV_Project/ (or update the path variable in cell 1 of the notebook).
   • Ensure the dataset logs are present under PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/.

2. Run Evaluation:

   • Open colab_evaluation.ipynb in Google Colab or local JupyterLab.
   • Point the test data path to PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/test/Colab_test_files/ for the best visualization outputs.
   • Run all cells sequentially to train the unsupervised baseline models (PCB-iForest, Bayesian probability maps, Shannon entropy baseline) and generate the comparative evaluation figures (ROC curves, confusion matrices, and detection summaries).

---

Option B: Hardware-In-The-Loop (HIL) with Physical ESP32

To execute the entire pipeline end-to-end on physical hardware:

1. Configure Edge Wi-Fi & Telemetry Settings: Open EDGE_SIDE/include/api_network.h and update the network definitions to match your local setup:

   • API_WIFI_SSID: Your Wi-Fi network name (SSID).
   • API_WIFI_PASSWORD: Your Wi-Fi network password.
   • API_TCP_SERVER_IP: Your PC's local IP address (e.g., 192.168.1.100) where the Python server will listen.
   • API_TCP_SERVER_PORT: Default is 9999.

   (Note: If building with ESP-IDF directly, these can also be configured interactively using idf.py menuconfig via EDGE_SIDE/Kconfig.projbuild).

2. Flash the ESP32 Firmware:

   • Open the project in PlatformIO / VS Code.
   • Connect your ESP32 board via USB.
   • In platformio.ini, confirm upload_port matches your board (e.g., COM12).
   • Run PlatformIO Upload:

     & "$env:USERPROFILE\.platformio\penv\Scripts\platformio.exe" run --target upload

3. Launch the Cloud Machine Learning Server & GUI Dashboard:

   • Open a terminal and navigate to the PC_SIDE directory:

     cd PC_SIDE
     pip install -r requirements.txt
     python SIMULATION/gui_main.py

   • Select your ESP32 COM port from the PORT dropdown (or click the refresh button to re-scan).
   • Select your training baseline log (ZE0w24kWhstock.log) and choose a regular test log from PC_SIDE/EV_CANlogs/Nissan LEAF/24kWh/test/ (e.g. test_ZE0w24kWh_DoS.log or test_ZE0w24kWh_Fuzzy.log).
   • Click INIT HARDWARE to synchronize the ESP32 with the PC. Press the EN (Reset) button on the ESP32 when prompted.
   • Click RUN BATCH to stream CAN traffic to the ESP32 and observe real-time Two-Tiered intrusion detection on the live dashboard.

---

Option C: Virtual Emulation (Wokwi in VS Code)

If hardware is not connected, you can use the Wokwi emulator inside VS Code to simulate the edge hardware:

1. Install the Wokwi Simulator extension in VS Code.
2. Configure wokwi.toml and diagram.json for the ESP32 in your project.
3. Start the Wokwi simulation and bridge the virtual serial output to PC_SIDE/SIMULATION/gui_main.py using a virtual COM port pair (e.g., com0com).