Natan Oliveira Martins
← Back to index
01 Finance · low latency In construction

Real-time fraud detection

Thousands of card transactions per second, each scored and blocked or approved within a 20 ms latency budget.

  • Redpanda
  • Apache Flink
  • Redis
  • XGBoost → TL2cgen
  • Prometheus

Code and roadmap on GitHub →

Interactive demo

Drag to orbit · scroll to zoom · Each dot is a sampled transaction; the metrics use the full simulated stream.

Throughput––
Latency p50–
Latency p99––
Precision–
Recall–
Legit customers blocked–
Latency p99, last 60 s

Recently blocked

    Things to try

    1. Push transactions past ~80% utilisation and watch p99 blow the SLO long before the system is “full”. Queueing is non-linear.
    2. Raise parallelism to buy back latency: that is the architecture answer, not a faster model.
    3. Lower the threshold: recall rises, but so does the count of legitimate customers you block. Pick it from the cost of each error.
    4. Launch the attack: the first small charges get through, then the 60-second velocity window catches up.

    The problem

    A fraud decision is only useful before the payment clears. That turns a modelling problem into a latency budget: features must be computed over sliding windows, user profiles fetched from memory, and the model evaluated, all inside a single authorization round-trip.

    How it works

    1. Transactions land on a partitioned Redpanda topic, keyed by card so each card’s events stay ordered.
    2. Flink keeps per-card state: transaction velocity over 60 s, amount z-score against the card’s history, distance from the last location.
    3. Redis holds the long-lived profile (typical merchants, home region) so the stream job never touches a database.
    4. A gradient-boosted model, compiled to native code, turns features into a fraud probability. A threshold, tuned on the cost of false blocks versus missed fraud, makes the call.
    POS / API card swipes Redpanda partitioned log Flink windowed features Redis user profile XGBoost compiled scorer Approve p ≤ threshold Block p > threshold
    Fig. 01 — How it works

    What is real, what is simulated

    Real

    • The scoring function and threshold logic in the demo run live in your browser.
    • Precision, recall and latency percentiles are computed live from every simulated transaction, not just the drawn ones.

    Simulated in this demo

    • Transactions are synthetic, generated from two distributions (legit, fraud).
    • Stage latencies come from a queueing model, not from a measured cluster.
    • The demo model is a hand-weighted logistic score, standing in for XGBoost.

    Not built yet

    • Redpanda + Flink + Redis pipeline (docker-compose)
    • XGBoost trained on a public fraud dataset
    • Load test with measured p99