End-to-End Stock Prediction (LSTM & Transformer) — live demo

This started as a research notebook comparing LSTM and Transformer architectures on Microsoft stock — a typical "trained a model on stock prices, plotted some charts" Jupyter artifact (preserved verbatim in research/). I then refactored it into a production-shaped commercial demo: a dual-head Transformer (direction probability + return %) and an XGBoost baseline, a FastAPI backend with mtime hot-reload, this static HTML dashboard, daily prediction logging, and a weekly retraining loop that atomically swaps the production artifact via tmp + fsync + os.replace.

The notebook had a few classic research-mode shortcuts: a normalizer fit on the full dataset (data leakage), a denormalization bug at lines 1634–1636, and one ticker only. The production version fixes the leakage (train-only fit, applied to val/test/inference), sidesteps the denormalization bug by predicting return percentage instead of absolute price, and ships two tickers (MSFT + S&P 500) via config blocks. A pre-declared honesty gate in the bake-off says the production model must beat naive_always_up by ≥3 pp on direction accuracy to count as real signal — neither winner does, on either ticker, and the dashboard shows the failure verdict in red below.

The point of this project is the productionization, not the predictive accuracy. The model honestly reports it cannot extract next-day directional signal from this feature set; the system around it (fetch → cache → train → bake-off → promote → serve → log → backfill → retrain) is real, atomic, idempotent, and would immediately surface any future feature engineering that did move the gate.

Tech stack: TensorFlow / Keras 3 · XGBoost · scikit-learn · FastAPI · pydantic v2 · yfinance · pandas · pyarrow · vanilla HTML/JS (no framework, no CDN). Full source, plans, and journey writeup in the GitHub repository — see README.md, PLAN.md, LEARNING_NOTES.md, and docs/journey.md.

Disclaimer: Predictions are informational only and are not financial advice. The honesty gate explicitly says the directional signal is not extractable from this feature set on a 1-day horizon — treat displayed predictions accordingly.