Natan Oliveira Martins
← Back to index
04 MLOps · data platform In construction

A feature store without leakage

One definition per feature, served offline for training and online for inference, with point-in-time joins that never peek at the future.

  • Feast
  • DuckDB / Parquet
  • Redis
  • Apache Airflow
  • Great Expectations

Code and roadmap on GitHub →

Interactive demo

Join strategy

Drag a flag along the timeline to move its label · drag the background to orbit

Training rows this join produces

CustomerLabelpurchases_30dValue fromStatus

The experiment

Same synthetic customers, same model (logistic regression on 3 features), two ways of building the training set. “Offline” is the validation score you would report; “production” is how the model actually scores when, at prediction time, only the past exists.

Things to try

  1. Switch to “Latest value”: every join line that reaches past its flag turns red. Those rows teach the model what churn looks like after it already happened.
  2. Drag C-102’s flag left and right: the point-in-time join always picks the last bar strictly before it, never one after.
  3. Lower the TTL until C-104 loses its value: a stale feature becomes missing instead of silently pretending to be fresh.
  4. Run the experiment: the leaky model looks perfect offline and collapses to a coin flip in production. Nothing in the offline metrics warns you.

The problem

Most production models are worse than their notebooks because training used a value that did not exist yet at prediction time. A feature store exists to make that mistake impossible, and to stop five teams from computing “active users” five different ways.

How it works

  1. Features are declared once (source, entity, transformation, freshness) in a registry.
  2. The offline store keeps full history; training sets are built with point-in-time joins: for each label, the latest feature value strictly before the label’s timestamp.
  3. Materialisation jobs push the latest values to Redis for millisecond online reads.
  4. A consistency check compares online and offline values for the same entity and time.
Sources events · tables Registry feature defs Offline Parquet history Online Redis latest Training point-in-time Inference < 5 ms read
Fig. 04 — How it works

What is real, what is simulated

Real

  • The point-in-time join, the TTL rule and the leak detection in the timeline are the actual logic, running in your browser.
  • The experiment really trains two logistic regressions on 2,800 rows and scores them on 1,200, computing AUC with the Mann–Whitney statistic.

Simulated in this demo

  • Customers and their behaviour are synthetic, generated so that activity stops after churn, which is exactly what makes “latest value” leak.
  • The registry entry is illustrative; no Feast server sits behind this page.

Not built yet

  • Feast repository with offline (Parquet/DuckDB) and online (Redis) stores
  • Materialisation job in Airflow
  • Online/offline consistency check with Great Expectations