The log as a data structure

A log is a list that only grows

When engineers talk about a log as a data structure, they mean an append-only, totally ordered sequence of records.

Picture a row of boxes. Each new record goes into the next empty box at the end.

Notice that the new record can only enter the empty box at the right end, so older records never move or change.

Nothing is squeezed into the middle. Nothing is rewritten.

The list simply grows over time, with the oldest record at one end and the newest at the other.

Append-only: write once, never edit

Append-only means the only way to change a log is to add a new record at the end.

You never update an old entry in place. If a fact changes, you append a new record describing the change.

Example: a customer moves house. The log does not edit the old address. It adds a record saying the address changed.

Old records stay exactly as written, so the full history is preserved.

Every record gets an offset

Each record receives a sequential number when it is appended. This number is called its offset.

Offsets count up one at a time: 0, 1, 2, and so on.

A small example: offset 0 is customer signs up, offset 1 is customer places order, offset 2 is customer changes address.

Every record has exactly one position, so the log is totally ordered. For any two records, you can always say which came first.

Offsets as logical timestamps

An offset works like a logical timestamp. It tells you the order things happened, without any clock.

If record A has offset 5 and record B has offset 9, A happened first.

Clocks on different machines can disagree. Offsets cannot, because the log itself assigned them.

The offset tells you before and after. It does not tell you how long passed between records.

Reading from the log

To read, you pick an offset and read forward, record by record, in order.

Reading does not remove anything. Records stay put, so many readers can share the same log.

Each reader remembers its own position: the offset of the next record it needs.

A reader that stops can resume from that offset later. One reader can be at offset 2 while another is at the very end.

Not your application's log file

The word log also names application logs: text lines like error messages and debug output written for humans.

Those are a different thing. You usually search them by keyword or time when something breaks.

The log as a data structure holds records that programs read in order, by offset, to learn what happened.

The overlap is real: both are append-only and ordered by time. Their purpose differs.

Key takeaways

A log is an append-only, totally ordered sequence of records.

Each record gets an offset, a sequential number that doubles as a logical timestamp.

Readers start at any offset and move forward. Reading never deletes.

When a colleague says log, ask: text messages for humans, or ordered records for programs?

Record A has offset 5 and record B has offset 9 in the same log. From the offsets alone, you can conclude _____

• that A was appended before B, but not how much time passed between them
• that A was appended before B, and that more time passed than between offsets 5 and 6

Answer: that A was appended before B, but not how much time passed between them

An offset is a logical timestamp: it tells you order (A came before B) but not how much time passed between records. The gap of 4 is a count of positions, not a duration.

A log is an _____ sequence, so changes are added only at the end. Because every record has exactly one position, the log is _____, and each record's _____ acts as a _____ showing before and after.

Answer: A log is an append-only sequence, so changes are added only at the end. Because every record has exactly one position, the log is totally ordered, and each record's offset acts as a logical timestamp showing before and after.

Append-only describes how the log changes (only at the end); totally ordered means any two records can be compared for which came first; the offset is the sequential number that serves as a logical timestamp, giving order without a clock.

A log stores customer records. A customer named Ana changes her email address. How should this change be recorded in an append-only log?

• Append a new record at the end stating that Ana's email changed
• Overwrite Ana's original record with the new email address
• Insert the new email right after Ana's original record
• Delete Ana's original record and append her updated record

Answer: Append a new record at the end stating that Ana's email changed

In an append-only log, the only allowed change is adding a record at the end, so the change is recorded as a new entry and the old record stays as written. Editing or inserting in the middle would break the append-only rule and lose history.