Free sample lesson · How to design WhatsApp
DynamoDB GSIs
How do you find all messages for a specific user in a split second among billions of records?
Lesson 1 of the Wondering course How to design WhatsApp
What You Will Learn
Global Secondary Indexes allow for flexible querying patterns beyond the primary key to ensure low-latency data retrieval.
The Limits of Primary Keys
In a high-scale chat app, your main table is likely partitioned by ChatID. This works perfectly for fetching a specific conversation's history.
But what if you need to find every message sent by a specific user across all chats?
Querying by UserID on a ChatID table requires a full table scan. This is slow, expensive, and impossible to scale with billions of records.
- Primary Key Query
- Full Table Scan
Enter Global Secondary Indexes
A Global Secondary Index (GSI) acts as a shadow copy of your data with a different partition key.
By setting the GSI partition key to UserID, you can query user activity with millisecond latency.
DynamoDB handles the heavy lifting. When you write to the base table, it automatically propagates the data to your index.
The Power of Index Projection
Index Projection is the process of choosing which attributes to copy from the base table into the GSI.
You don't have to copy everything. In fact, you shouldn't.
Projecting only essential fields—like Timestamp and MessagePreview—drastically reduces storage costs. It also minimizes the write throughput required to keep the index in sync.
Optimizing for Chat Performance
Mastering GSIs is about balancing read flexibility with write overhead.
For a seamless multi-device experience, use GSIs to track unread counts or message status across different platforms.
Always aim for minimal projections. Only include the data your specific query pattern needs to avoid unnecessary "shadow" data bloat.
Putting It All Together
- Identify Pattern: Determine the query attribute that isn't the base table's primary key.
- Define Projection: Select only the minimum required attributes to be copied to the index.
- Provision GSI: Create the index with the new partition key to enable low-latency lookups.
- Query Index: Perform queries against the GSI instead of scanning the base table.
Check yourself
In DynamoDB, _____ projection refers to copying only a subset of attributes from the base table into the GSI.
- Attribute
- Table
Answer: Attribute
Index projection is the specific process of choosing which attributes are duplicated into the index to save on storage and write costs.
To optimize a chat app, use a _____ to enable new query patterns and _____ to limit which data is copied.
Answer: To optimize a chat app, use a GSI to enable new query patterns and projection to limit which data is copied.
GSIs provide alternative lookup keys (like UserID), while projection ensures you only store what you actually need to query.
Why would a developer project only 'MessageText' and 'Timestamp' into a GSI instead of all attributes?
- To reduce storage costs and write throughput
- To increase the speed of the base table writes
- To allow the GSI to have a different Partition Key
Answer: To reduce storage costs and write throughput
Projecting fewer attributes reduces the amount of data written to the index, lowering both storage costs and the consumed Write Capacity Units (WCU).
Keep going
This is lesson 1 of 10 in How to design WhatsApp. Next up:
- Data Modeling
- Kafka Limits
- Redis PubSub
- ZooKeeper Role
- Consistent Hashing
Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.