Workloads Outgrow Machines

What You Will Learn

Scale turns product ambition into infrastructure requirements, forcing engineers to design for growth in traffic, data scope, and algorithmic cost at the same time.

The Multiplicative Growth Problem

Scaling a system is more than just handling more users. Jeff Dean explains that demand grows through three factors multiplying together.

First, user traffic increases. Second, the amount of data you store grows. Third, the compute cost per request rises as algorithms improve.

In 2007, Google managed tens or hundreds of billions of web pages. Each page averaged 10 KB, creating hundreds of terabytes of raw data for search alone.

Why Machines Disappear

Infrastructure needs often outpace simple hardware additions. Better ranking algorithms require more CPU cycles for every single query.

Google observed that as the index grows, the work to search it increases. When you add more documents, you need more computers to maintain speed.

Traffic growth: 2x Corpus size growth: 5x Algorithmic cost: 3x Total resource demand: 30x

This compounding effect forces engineers to rethink how they build systems.

The Shared Platform Necessity

Different products create different data shapes. Search uses text indexes, while Maps handles geographic data and Video manages large media files.

Building custom failure handling for every product is inefficient. Small teams need shared abstractions to build quickly without reinventing storage.

Shared infrastructure allows one platform to support many distinct product needs simultaneously.

Designing for Growth

Infrastructure must support multiple data shapes rather than one specific product path. This prevents engineering bottlenecks as the company expands.

Do not assume scaling is only about traffic. Always account for the growth of your data corpus and the cost of your logic.

Abstraction reduces repeated work. It allows engineers to focus on product features instead of low level distributed systems failures.

Putting It All Together

Scaling a system effectively requires accounting for growth in traffic, data size, and _____ simultaneously.

• algorithmic cost
• hardware lifespan

Answer: algorithmic cost

Scaling isn't just about users; you must also account for the size of the data (corpus) and the computational cost of the algorithms used to process it.

Multiplicative growth means that if traffic increases _____ and the data corpus grows _____, the total resource demand becomes _____.

Answer: Multiplicative growth means that if traffic increases 2x and the data corpus grows 5x, the total resource demand becomes 10x.

Resource demands multiply rather than add. 2 (traffic) × 5 (corpus) = 10x total demand.

Why should a large-scale platform use shared infrastructure for different products like Search, Maps, and Video?

• To allow small teams to build quickly without reinventing failure handling.
• Because all products use the exact same data shapes and text indexes.
• To ensure that one flagship product dictates the entire system design.
• To reduce the total number of machines to a single physical server.

Answer: To allow small teams to build quickly without reinventing failure handling.

Shared abstractions allow small teams to build quickly by providing common solutions for complex problems like failure handling and storage, regardless of the data shape.