Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann

Another area in which stream processing is used is for analytics on streams.

is more oriented toward aggregations and statistical metrics over a large number of events

However, it could still happen that some events were buffered on another machine somewhere, delayed due to a network interruption.

Assigning timestamps to events is even more difficult when events can be buffered at several points in the system.

However, the clock on a user-controlled device often cannot be trusted, as it may be accidentally or deliberately set to the wrong time

To adjust for incorrect device clocks, one approach is to log three timestamps

By subtracting the second timestamp from the third, you can estimate the offset between the device clock and the server clock

A hopping window also has a fixed length, but allows windows to overlap in order to provide some smoothing.

Unlike the other window types, a session window has no fixed duration.

so the stream processor’s local copy of the database needs to be kept up to date.

the stream processor can subscribe to a changelog of the user profile database as well as the stream of activity events.