An end-to-end, production-style fraud detection pipeline built on the IEEE-CIS Fraud Detection dataset. The notebook engineers behavioral features, trains and tunes an XGBoost classifier, explains every prediction with SHAP, and converts raw scores into analyst-ready risk tiers and notes.
| File | Description |
|---|---|
fraud_risk_scoring_system.ipynb |
The complete, self-contained Jupyter notebook (22 sections, EDA → feature engineering → modeling → tuning → SHAP explainability → monitoring → batch scoring → model persistence). |
README.md |
This file. |
It's strongly recommended to isolate this project in its own environment so its dependencies never conflict with other projects.
# 1. Create the project folder and move into it
mkdir fraud-risk-scoring && cd fraud-risk-scoring
# 2. Create an isolated virtual environment named "fraud-env"
python3 -m venv fraud-env
# 3. Activate it
# On macOS / Linux:
source fraud-env/bin/activate
# On Windows (PowerShell):
fraud-env\Scripts\Activate.ps1
# On Windows (cmd.exe):
fraud-env\Scripts\activate.bat
# You should now see (fraud-env) prefixed in your terminal prompt.conda create -n fraud-env python=3.10 -y
conda activate fraud-envOnce the environment is activated, install the required libraries:
pip install --upgrade pip
pip install pandas numpy matplotlib seaborn plotly scikit-learn xgboost shap joblib imbalanced-learn jupyter notebook ipykernelOr, save this as requirements.txt inside your project folder and run pip install -r requirements.txt:
pandas
numpy
matplotlib
seaborn
plotly
scikit-learn
xgboost
shap
joblib
imbalanced-learn
jupyter
notebook
ipykernel
Note:
imbalanced-learn(which providesSMOTE) is required for Section 9's class-imbalance comparison. All other libraries used in the notebook (pandas,numpy,matplotlib,plotly,seaborn,sklearn,xgboost,shap,joblib,warnings,os,gc) are either standard library or listed above.
Register the environment as a Jupyter kernel so it shows up in Kaggle/Jupyter's kernel picker:
python -m ipykernel install --user --name=fraud-env --display-name "Python (fraud-env)"This notebook expects the IEEE-CIS Fraud Detection competition data at:
/kaggle/input/competitions/ieee-fraud-detection/
├── train_transaction.csv
└── train_identity.csv
On Kaggle:
- Create a new Kaggle Notebook.
- Click Add Input → search for IEEE-CIS Fraud Detection → attach it.
- Upload/copy
fraud_risk_scoring_system.ipynbinto the Kaggle notebook environment (File → Upload Notebook, or copy-paste cells). - Run all cells top to bottom (Run All).
Running locally instead of Kaggle:
- Download
train_transaction.csvandtrain_identity.csvfrom the competition page on Kaggle (you must accept the competition rules first). - Place them in a local folder, e.g.
./data/. - In Section 3 of the notebook, change:
to:
DATA_DIR = "/kaggle/input/competitions/ieee-fraud-detection"
DATA_DIR = "./data"
# with the fraud-env environment activated
jupyter notebook fraud_risk_scoring_system.ipynbThen select the Python (fraud-env) kernel from the kernel menu, and run all cells sequentially (Cell → Run All or Run All on Kaggle).
The notebook is organized into 22 sections, each preceded by a short Markdown explanation. Broadly it flows as:
- Intro & setup (Sections 1–2)
- Data loading & EDA (Sections 3–4)
- Cleaning & feature engineering (Sections 5–6)
- Encoding & splitting (Sections 7–8)
- Imbalance handling & baseline models (Sections 9–10)
- Tuning & final training (Sections 11–12)
- Evaluation & SHAP explainability (Sections 13–14)
- Analyst notes & risk tiers (Sections 15–16)
- Feature importance & monitoring dashboard (Sections 17–18)
- Batch scoring, persistence & inference demo (Sections 19–21)
- Summary & future work (Section 22)
Running the full notebook produces the following files in the working directory:
xgboost_model.pkl— the trained, tuned XGBoost modellabel_encoders.pkl— dictionary of fittedLabelEncoders for categorical columnsfeature_columns.pkl— the exact ordered list of feature columns expected at inference timebatch_predictions.csv— example batch-scored output with fraud probability, risk level, and analyst notes
To run inference later without retraining:
import joblib
model = joblib.load("xgboost_model.pkl")
encoders = joblib.load("label_encoders.pkl")
feature_columns = joblib.load("feature_columns.pkl")
proba = model.predict_proba(new_transaction_df[feature_columns])[:, 1]deactivate # venv
# or
conda deactivate # condaTo remove the environment entirely:
rm -rf fraud-env # venv
# or
conda remove -n fraud-env --all # conda