Real-time Analytics

Events read while they are still current.

Real-time analysis fails for a mundane reason: the analysis system sees a copy, and the copy is old.

A threshold checked against data that is still being written.

The workload

Windows and aggregates over data that is still being written.

Reads run against the operational layer while writes continue, so a window, a rate or a threshold is computed from the current data. There is no second pipeline, which is also why there is nothing to fall behind. The same access path serves records, documents and time-ordered data.

Data foundation · 3 data shapes · 4 stages of the operation.

A threshold checked against data that is still being written.

Environment

Live writes

The operational system keeps writing while the question is being asked.

Time series · SQL

What one layer changes

Two estates, drawn.

The same shapes, held two ways. The difference is not the storage, it is where the agreement between them lives.

Separate systems Copies kept in step
a copy, and a job to keep it honest
A streaming pipeline copies data into an analytical store, and the lag between the two becomes a number somebody has to watch. The pipeline itself needs maintaining forever.
One layer One plan · one contract
one layer · one plan read from one place, as one answer
The analytical read is a query against the live layer, so freshness is a property of the system rather than of a pipeline’s health.
Data shapes

What this workload moves.

Every shape below is a surface of the layer, not a format to be converted into one. They are read and written together.

  • Time series Measurements and events in time order
  • SQL Records, keys and joins
  • Events Operational events as they happen
Architecture

How the workload reaches the data.

The models this workload names, the path a request takes through them, and the surfaces that speak to the layer.

Real-time Analytics · architecture
Workload

The shape of the work itself.

  • Ingest
  • Aggregate
  • Alert
  • Explore
Data models

The models this workload names.

  • Time series
  • SQL
  • Events
The layer

One path from a request to the data it names.

  • Planning Predicates narrow the work before it runs
  • Execution Rows, fields and windows answered together
  • Transactions Readers and writers do not block each other
Surfaces

How the workload reaches the layer.

  • SQL surface The query language the layer is documented in
  • Applications Services and jobs reading and writing as they run
  • Analytics & AI clients The same layer, the same access path
Workload map

The work, stage by stage.

Choose a stage to read what happens there, or a shape to see every stage that handles it. Nothing on the map is a private interface.

PLOMID · Real-time Analytics ingest · aggregate · alert · explore Select a stage
Stage

Ingest

Operational writes land in the layer

Time series · SQL

Workload map Real-time Analytics workload map. Each stage names the part of the operation it carries and the shapes present at that point.
01

Ingest

Operational writes land in the layer

  • Time series
  • SQL
02

Aggregate

Windows, rates and thresholds computed in place

  • Time series
03

Alert

Conditions evaluated against the same reads

  • Events
  • SQL
04

Explore

The result is queried like any other data

  • SQL
Outcomes

What changes when the data is in one place.

Stated as properties of the system rather than as results we cannot measure for you.

Nothing to fall behind

Freshness stops being a pipeline metric, because there is no pipeline.

Thresholds on live data

Conditions are evaluated against the data as written rather than against a copy.

One definition of a rate

The same window definition serves the dashboard and the report.

Results stay queryable

An alert is a record in the layer, so it can be read against what produced it.

Related industries

Where this shows up.

Derived from the industry pages that reference this solution, so the two directions of the relationship always agree.

All industries