Point-in-time correctness: what did the model actually know?

Definition

A training dataset is point-in-time (PIT) correct when every row contains only information that was available to the system at that row’s decision moment. Not information that existed somewhere in the world. Not information that describes an earlier time. Information the system could actually have read, then.

The definition sounds pedantic until you see what violating it does: a model trained on late-arriving information learns patterns its production counterpart can never use. Offline metrics look excellent; live performance disappoints; and the gap is invisible in code review, because every individual query looks reasonable.

Why historical joins leak the future

The naive way to build training data is a join on time: for each historical decision, pull the most recent source records by their timestamps. The flaw is that a record’s timestamp usually says what the record describes — not when your system received it. A bureau report describing Monday may have arrived Thursday. The naive join hands your Tuesday training row a report your Tuesday production system had not seen. The model trains with a few days of clairvoyance, consistently, across millions of rows.

This failure is systematic, not random — which is what makes it dangerous. Random noise hurts offline metrics; systematic leakage improves them, so the better your leaked backtest looks, the more confidently you ship a model that cannot repeat it.

The three clocks

Untangling this requires naming the clocks separately. ClearFeature’s data model uses these terms; the concepts apply to any system.

Event time

event_ts (or report_ts / data_ts): the time the data describes. A bureau report’s event time is the moment the credit picture it captures was true. A market tick’s event time is when the trade happened.

Availability time

available_at: when your system could first know the data — when the feed delivered, the file landed, the API responded. Availability is a property of your ingestion reality, not of the world. Two systems consuming the same vendor can have different availability for the same event.

Observation time

observation_ts (decision time): the moment the historical decision was made — the moment each training row is anchored to, and the moment that defines what “the past” means for that row.

A useful fourth clock, for auditability rather than eligibility, is computation time (calc_ts): when the platform actually computed a feature value.

A concrete timeline

The eligibility rule that falls out of the three clocks is one sentence: a value may enter a training row only if it belongs to the past and was available by the observation time. In the timeline above, the report fails the second condition — it describes the past, but the system learned it too late.

Why event time alone is insufficient

Filtering on event_ts < observation_ts feels safe and is the most common bug in training-data construction. It enforces “the data describes the past” while ignoring “the system knew it.” Everything with delivery lag — bureau files, vendor feeds, operational reports, revised forecasts — slips through, describing the past while arriving from the future. Only available_at <= observation_ts closes the gap.

Multiple sources, multiple lags

Real decisions read several sources, each with its own delivery behavior: the application data was instant, the bureau report took minutes, the income verification took days. PIT correctness must be enforced per source, per row. A dataset can be perfectly correct with respect to one source and quietly leaky through another — this is why the discipline belongs in the platform that assembles training data, not in each team’s query hygiene.

Constructing PIT training datasets

The construction that follows from the model: take the set of historical observations (entity, observation_ts, label). For each observation, select from each source the latest record with available_at <= observation_ts; execute the feature logic over exactly that information set; emit the vector. In ClearFeature this is a platform operation — fsctl build-training-dataset consumes observations and produces availability-correct rows by construction, using the same feature implementation that runs online.

Credit example

An underwriting model is trained on two years of decisions. Bureau reports in that history were delivered with hours-to-days of lag. Built on event time, the training data gives every historical decision a slightly newer credit picture than underwriting actually had — the model learns to lean on freshness that production never sees, and validation later struggles to explain a persistent offline/online gap. Built on availability, the training rows match decision reality. (ClearFeature for credit risk →)

Trading example

A forecasting model backtests beautifully: it “reacts” to market events within a minute. The research dataset was joined on event time; the vendor feed delivers with two minutes of lag. The strategy’s edge is exactly the leak. In markets, minutes of future information do not inflate a backtest — they are the backtest. (ClearFeature for energy & trading →)

Reproducibility is the same problem, reversed

PIT correctness looks forward (build honest training data); reproducibility looks backward (reconstruct what a past decision knew). Both need the same substrate: sources stored with availability, decisions anchored to observation times, and versioned feature logic. If your platform can build a PIT-correct dataset for March 3rd, it can also answer what the model knew on March 3rd — same query, different purpose. (Model reproducibility →)

How ClearFeature models it

In ClearFeature the temporal model is structural, not conventional: sources carry report_ts and available_at; training construction is anchored to observation_ts; eligibility — past and available — is enforced by the platform when datasets are built; and because the same versioned Feature Project executes historically and online, temporal correctness and implementation consistency come from one mechanism rather than two disciplines. The platform page shows where this sits in the wider execution model.

Further reading

• Online vs offline feature stores — where PIT construction lives in the architecture.
• What is a feature store? — the category from first principles.
• Training-serving skew — the code-side twin of temporal skew.