What does a feature platform need for real-time ML?

The real-time ML architecture, minimally

Strip a real-time decision system to its skeleton and four things remain: a request arrives carrying some data; a feature vector is assembled within a latency budget; a model scores it; a decision returns. Feature infrastructure owns step two — and step two is where most real-time ML programs quietly succeed or fail, because it is where training-world and production-world must agree.

Freshness and latency are different requirements

They get conflated constantly:

• Latency — how fast the vector is assembled after the request arrives.
• Freshness — how recent the information inside the vector is.

A cached lookup gives superb latency with whatever freshness the last precomputation had. Request-time execution gives freshness bounded only by the inputs, at the cost of computing in the decision path. Which one a feature needs is a per-feature question, not a platform-wide one: avg_transaction_amount_90d tolerates hours of staleness; “amount of the transaction being scored right now” tolerates none — it does not exist until the request arrives.

Correctness is the third axis — and the least visible

A real-time system can be fast, fresh, and wrong: wrong because the online implementation drifted from the training implementation (code skew), or because the model was trained on information timing that production never experiences (temporal skew). Neither failure appears on a latency dashboard. Both appear in live model performance, weeks later, unattributed.

Dependencies must execute online too

Real features are graphs, not scalars: a burden ratio depends on a payment aggregate and an income estimate; a fraud signal combines transaction, customer, and merchant features. At request time, something must resolve that graph in the right order. If the answer is “the service code calls things in the order someone wrote down,” dependency logic has forked from the training path — and dependent features are precisely where small divergences compound. A platform should resolve the declared graph itself, identically in batch and online execution.

Latest-value stores

The standard component for lookup-style access is a fast key-value store holding the most recent value per feature per entity — Redis-family systems are typical; ClearFeature uses Valkey. Latest-value stores are necessary and well-understood. The design mistake is treating one as the whole online architecture: a cache answers “what was last computed?”, not “what is true of this request?”

Request-time source computation

Decision systems in lending, fraud, and trading share a trait: the most important information arrives with the request — the application, the transaction, the current market context. No precomputation schedule can supply features derived from data that did not exist a second ago. The architecture that fits is request-triggered execution: current source reports enter the platform, the feature DAG executes, the vector returns. This is ClearFeature’s primary online model, and it is worth stating what it is not: not a stream processor continuously recomputing every feature in the background — computation happens when a decision asks.

The divergence trap in batch/live code

The default engineering path to real-time features is the duplicated one: training features in Python or SQL, online features rewritten inside a low-latency service. Every argument in the skew guide applies with extra force here, because the online copy is optimized under pressure — and optimizations are edits, and edits drift. The structural fix is a single implementation executed on both paths, which converts an ongoing synchronization discipline into a property of the architecture.

Operational failure modes to design for

• Source unavailability at request time — a report fetch fails; define per-feature behavior deliberately, not by exception path.
• Version mismatch — the model was trained against feature version N; serving must execute N, not “latest.” Versioned feature artifacts with explicit promotion make this governable.
• Silent drift — without one implementation, parity decays invisibly; with one, the failure class is structurally absent.
• Unreproducible incidents — when a decision is questioned, can you reconstruct its vector? Availability-aware history makes the answer computed rather than forensic.

When precomputation is appropriate

Deep-history aggregates, expensive computations over slow-moving data, features shared across very high request volumes, and lookup-latency budgets too tight for any computation. Precompute them, store them, look them up — and keep their computation on the same platform and logic as everything else, so precomputed does not come to mean “differently implemented.”

When request-time computation is appropriate

Features derived from request-scoped data; domains where freshness is correctness (fraud, underwriting, market state); dependency graphs that must reflect current inputs end to end. Here, request-time execution with training-identical logic is not an optimization — it is the requirement.

How ClearFeature approaches it

ClearFeature’s position in this design space is deliberate: one versioned Feature Project executed by the platform in every path; request-triggered online computation over current source reports with dependencies resolved from the declared DAG; latest values in Valkey for lookup access; and availability-aware history underneath for honest training data and reconstruction. It is not the universal real-time architecture — continuously-computed streaming aggregates, for instance, are a different center of gravity — but for request-driven decision systems it fits the requirements above by construction. (The full model →)

An evaluation checklist

1. For each feature: what freshness does it require — and what latency can its supply afford?
2. What executes dependent features online, and is it the same graph the training path used?
3. Can serving pin a feature version to the model that was trained on it?
4. What happens, feature by feature, when a source is unavailable at request time?
5. Could you reconstruct the vector behind a specific decision from last quarter?