Back to the projects

Work

Forecasting pipeline & monitoring app

Monthly unit and revenue forecasts for sales – automated, for hundreds of time series at once. Instead of one single model, several models compete for every time series and the best one wins. A monitoring shows how good the forecasts are and warns when they get worse.

  • Python
  • SQL
  • Qlik
  • Time series
  • Machine learning

How the pipeline works

  1. Load sales data from the data warehouse – one time series per channel and product group
  2. Check the data: fill gaps, account for returns, drop series without recent sales
  3. Calculate seasonal factors per month (smoothed over several years) and remove seasonality
  4. Train several models for every time series and compare them in a backtest (holdout)
  5. The best model forecasts the next months; seasonality is added back
  6. Top-down distribution to channels and subcategories, including gross/net revenue and gross profit

Models in competition

  • Theta method – strong on calm series with a trend
  • Exponential smoothing (ETS / Holt-Winters)
  • Structural time series models (unobserved components)
  • Linear and robust regression (Huber) for the trend
  • For every series the model with the smallest backtest error wins – the “model mix” shows how often each one wins

Monitoring

  • Accuracy overall and per series: WAPE (weighted percentage error) and bias (systematically too high or too low?)
  • Automatic email warning when the error exceeds a threshold
  • Monitoring app in Qlik: forecast vs. actuals – by category, against the previous year and across forecast horizons

My part

  • Concept, development and operation of the pipeline in Python
  • Data provisioning with SQL from the data warehouse
  • Monitoring app in Qlik

Tech

  • Python: pandas, NumPy, statsmodels, scikit-learn
  • SQL / Exasol
  • Qlik Sense