BidBird: planning-risk models

BidBird is a planning risk-intelligence tool built by a UCL team at the Bidwells Real Estate Planning Hackathon (November 2025), where it won 1st place. This repository holds its modelling side: two classifiers that estimate whether a planning application will be approved and whether a planning appeal will be allowed, plus a small FastAPI service that scores new cases.

Data and models

Task	Data (as used in the notebooks)	Rows after cleaning	Selected model
Applications	Camden planning-applications open data	71,258	Random Forest, threshold 0.40
Appeals	Planning-appeal casework records from many local planning authorities	87,393	XGBoost, threshold 0.45

• notebooks/data_cleaning.ipynb maps each decision to a binary outcome (approved or allowed = 1), drops withdrawn applications and split decisions, extracts postcode districts and removes fields that are not known when an application or appeal is made.
• notebooks/modelling.ipynb builds one scikit-learn pipeline per task: target encoding for postcode district (and local planning authority for appeals), one-hot categoricals, binary site flags, numeric site measures for appeals and TF-IDF on one free-text field (the development description or the appeal reason). It compares Random Forest, XGBoost, LightGBM and HistGradientBoosting with stratified 5-fold cross-validation, picks the decision threshold that maximises F1, selects the model with the best CV F1 and refits it on the full dataset.
• The committed code searches hyperparameters with HalvingRandomSearchCV. The saved run behind the numbers below came from an earlier version of the code, which fitted 50 sampled candidates per model.

Results

Held-out split: 80% of a 50% random sample of each dataset (the search trained on the other 20%).

Metric	Applications, Random Forest	Appeals, XGBoost
Held-out rows	28,504	34,957
Positive class share	91.5% approved	28.7% allowed
F1, positive class (CV / held-out)	0.956 / 0.958	0.476 / 0.477
F1 of always predicting the positive class	0.955	0.446
Accuracy (majority-class baseline)	0.920 (0.915)	0.533 (0.713)
Recall on the minority class	0.088 (not approved)	0.742 (allowed)
Macro-F1	0.56	0.53
Balanced accuracy	0.54	0.60

• Applications. The F1 of 0.958 mostly reflects class balance: always predicting "approved" scores 0.955. The model flags 214 of the 2,431 applications that were not approved (8.8% recall), so it does not yet pick out risky applications reliably.
• Appeals. Only slightly better than chance. Balanced accuracy is 0.60, and accuracy (0.53) is below the 0.71 obtained by always predicting "not allowed".
• Caveats. The notebook tunes the threshold on the same held-out split it reports, so these figures are slightly optimistic. The exported models are refitted on all rows and have no separate held-out score.

The 1st place was a team result for the BidBird product as a whole; the figures above describe only the models in this repository.

Scoring API

src/api/main.py is a FastAPI app with three endpoints: GET /health, POST /predict/applications and POST /predict/appeals. Each prediction returns the class (0 or 1), the probability, the threshold and the model metadata. The request fields are the Pydantic models in the same file, and the model file names are fixed to the models selected in the saved run.

The data and the trained models are not committed, so the API cannot start from a fresh clone. It loads these files at start-up:

outputs/models/appeals/XGBoost.joblib            (+ _threshold.pkl, _metadata.pkl)
outputs/models/camden_pa/RandomForest.joblib     (+ _threshold.pkl, _metadata.pkl)

How to run

There is no pinned environment. The notebooks and API need Jupyter, pandas, numpy, matplotlib, scikit-learn, xgboost, lightgbm, category_encoders, fastparquet, joblib, fastapi and uvicorn.

1. Place the raw files in data/raw/ as planning_application_camden.csv and planning_application_appeal_uk.csv.
2. From notebooks/, run data_cleaning.ipynb, then modelling.ipynb. Before running the second one, add from sklearn.experimental import enable_halving_search_cv above the HalvingRandomSearchCV import (scikit-learn requires it) and create outputs/models/appeals/ and outputs/models/camden_pa/.
3. Start the API from src/api, which it uses to find outputs/models and its utils.py:

cd src/api
uvicorn main:app --reload

# in a second terminal
curl -X POST http://127.0.0.1:8000/predict/applications \
  -H 'Content-Type: application/json' \
  -d '{"in_conservation_area": 1, "in_neighbourhood_area": 0,
       "application_type": "Full Planning Permission", "ward": "Belsize",
       "conservation_areas": "Belsize Park", "neighbourhood_areas": "none",
       "postcode_district": "NW3", "development_description": "Single storey rear extension"}'

Layout

notebooks/data_cleaning.ipynb   raw CSV to cleaned Parquet
notebooks/modelling.ipynb       model search, threshold tuning, export
notebooks/test_model.ipynb      scores two example appeals with an exported model
src/api/                        FastAPI scoring service and input cleaning
src/utils/utils.py              copies of the notebook's cleaning helpers

Credits

Built by a UCL team at the Bidwells Real Estate Planning Hackathon, November 2025. Commits in this repository are by Mathis Weil. Built with scikit-learn, XGBoost, LightGBM, category_encoders and FastAPI.