End-to-End ML for Retail Analytics — live demo

A scientific notebook turned into a deployable web service. Nine sklearn / XGBoost / LightGBM / CatBoost pipelines covering invoice and cost regression, transaction-type / payment / day-of-week classification, and customer-level churn / LTV / discount-sensitivity / next-purchase models, trained on ~6.4M transactions across 1.28M customers.

← Home LinkedIn Notebook View on GitHub
About this project

This project started as a research notebook exploring nine prediction tasks on a synthetic retail dataset (~6.4M transactions across 1.28M customers): two regressors (Invoice Total, Production Cost), three transaction-level classifiers (Transaction Type, Payment Method, Day-of-Week), and four customer-aggregation models (Churn, 90-day LTV, Discount Sensitivity, Next-Purchase amount). I then refactored it into a production-shaped commercial demo: nine serialisable sklearn.pipeline.Pipeline objects, a FastAPI backend, this static HTML form, and a frozen 100-record holdout so the comparisons below stay reproducible.

After a code review I made a weekend improvements pass (full doc in IMPROVEMENTS.md): dropped four leaky features that made Churn and Discount-Sensitivity look near-perfect for the wrong reasons; added category_encoders.TargetEncoder for high-cardinality columns (Customer ID, City, Product ID, SKU, Job Title) in the regression pipelines; ran RandomizedSearchCV on Invoice Total and Production Cost; switched LTV to Tweedie loss + log1p target. The current numbers are honest.

Invoice Total R²: 0.755 Production Cost R²: 0.999 Churn ROC AUC: 0.766 Discount Sensitivity ROC AUC: 0.859 LTV MAE: 93.27 Next Purchase R²: 0.528 Training rows: 6.4M Holdout rows: 100

Try it: load any of the 100 holdout examples below. Each card shows the model's prediction next to the actual value from that row, so you can sanity-check the leakage-free models against the ones whose targets are still derivable from features (Transaction Type and Day-of-Week keep their original notebook formulations, which is why they score high).

Tech stack: scikit-learn · XGBoost · LightGBM · CatBoost · category_encoders · FastAPI · pydantic v2 · joblib · vanilla HTML/JS (no framework). Training scripts, the original notebook, and the improvement notes are in the GitHub repository — see README.md, PLAN.md, IMPROVEMENTS.md, and learning-notes-from-improvment.md.

Regression

Transaction-level classification

Customer-level aggregation models

Empty fields are sent as null and imputed by each pipeline. Aggregation models receive computed customer-level features (total spent, recency, etc.) so the picker auto-fills them — typing those by hand is unrealistic. The four customer-aggregation models and the two regressors were leak-fixed and retrained in the weekend pass; the three transaction-level classifiers (Transaction Type, Payment Method, Day-of-Week) keep their original notebook formulations and may still show notebook-level signal from features that overlap with the target.