Graph-LLM-IDS: Explainable Intrusion Detection System

This project implements a novel Graph-LLM Intrusion Detection System (IDS) that combines the structural learning capabilities of Graph Attention Networks (GAT) with the semantic reasoning of Large Language Models (LLMs).

It detects network attacks by analyzing the Host Communication Graph and provides natural language explanations for why a flow is malicious.

🚀 Project Pipeline

The system operates in 4 main phases:

1. Data Preparation (datacleaner.py)

• Source: CIC-IDS-2017 Dataset (TrafficLabelling CSVs).
• Process:
  • Merges 8 separate CSV files into a single dataset.
  • Cleans column names (strips whitespace).
  • Encodes labels: BENIGN $\to$ 0, All Attacks $\to$ 1.
  • Removes NaN and Infinity values.
• Output: cleaned_data.csv (~2.8M flows).

2. Graph Construction (Graph-LLM-model.py)

We build a Host Communication Graph where:

• Nodes: Unique IP Addresses (Hosts).
• Edges: Network Flows (Communication between IPs).
• Edge Features: 80 numerical flow statistics (Duration, Packet Counts, Inter-arrival Times, etc.), normalized using StandardScaler.
• Topology: 19,129 Nodes, 2,827,876 Edges.

3. Model Training & XAI (Graph-LLM-model.py)

• Model: Graph Attention Network (GAT).
  • Node Embeddings: Learnable 64-dim embeddings for each host.
  • Architecture: 2 GAT Layers (4 heads $\to$ 1 head) + MLP Edge Classifier.
• Training:
  • Optimization: Trained on Apple M1 GPU (MPS) using a custom Random Node Subgraph Sampling strategy to handle the large graph efficiently without OOM errors.
  • Loss: Cross Entropy.
• Explainability (XAI):
  • Extracts Attention Weights from the GAT layers.
  • Identifies "High Attention" edges (flows) that contributed most to the classification.
  • Visualizes the suspicious subgraph (xai_graph.png).

4. LLM Reasoning (explain_flows.py)

• Integration: Bridges the Graph model with an LLM.
• Process:
  1. Extracts the top suspicious flows identified by the GAT.
  2. Converts flow features into a natural language description (e.g., "Flow from IP A to IP B, Port 80, Duration 3s...").
  3. Feeds this description to a DistilGPT-2 (or other) model to generate a human-readable explanation of the threat.

📊 Results

Metric	Value
Test Accuracy	99.92%
Precision (Attack)	1.00
Recall (Attack)	0.99
AUC	~1.00

• Confusion Matrix: Missed only ~585 attacks out of ~111,000 in the test set.
• XAI: Successfully identified attacker-victim pairs (e.g., 172.16.0.1 $\to$ 192.168.10.50) with high attention scores.

🛠️ How to Run

1. Install Dependencies:

   pip install -r requirements.txt

   (Requires torch, torch_geometric, pandas, transformers, networkx, matplotlib, scikit-learn)

2. Clean Data:

   python3 datacleaner.py

3. Train Model & Identify Suspicious Flows:

   python3 Graph-LLM-model.py

   • Trains the GAT model.
   • Generates confusion_matrix.png, roc_curve.png, xai_graph.png.
   • Saves top threats to suspicious_flows.json.

4. Generate LLM Explanations:

   python3 explain_flows.py

   • Reads suspicious_flows.json.
   • Generates text explanations using the local LLM.

📂 File Structure

• datacleaner.py: Data merging and preprocessing.
• Graph-LLM-model.py: Graph construction, GAT training, Evaluation, XAI extraction.
• explain_flows.py: Standalone script for LLM generation.
• suspicious_flows.json: Intermediate file passing threat data to the LLM.

Graph-LLM-IDS