The Frontend Engineer’s Guide to Data-Intensive Applications

What I Learned from Part 1: Foundations of Data Systems

A practical summary of “Designing Data-Intensive Applications” Part 1, translated for frontend engineers building production-grade, real-time experiences.

Introduction: Why This “Backend Book” Changed How I Build Frontend Apps

When I picked up Martin Kleppmann’s “Designing Data-Intensive Applications,” I expected a deep dive into databases, distributed systems, and backend architecture. What I didn’t expect was a complete paradigm shift in how I think about frontend development.

The revelation: Modern web applications—especially real-time, data-heavy frontend experiences—face the exact same fundamental challenges as backend distributed systems. We’re just solving them in a different context.

Part 1 of the book, “Foundations of Data Systems,” covers four critical chapters that every engineer should understand, regardless of whether they work on frontend, backend, or full-stack. Here’s what I learned and why it matters for building production-grade frontend applications.

Chapter 1: Reliable, Scalable, and Maintainable Applications

The Three Pillars

Kleppmann introduces three core concerns for any software system:

1. Reliability - “Continuing to work correctly, even when things go wrong”

• Handling hardware faults, software bugs, and human errors
• Graceful degradation and fault tolerance

2. Scalability - “Reasonable ways of dealing with growth”

• Handling increased data volume, traffic, and complexity
• Performance characteristics as load increases

3. Maintainability - “Many people working productively over time”

• Operability, simplicity, and evolvability
• Adapting to new use cases

The Frontend Translation

These aren’t just backend concerns. Let me show you:

Reliability in Frontend:

1
// Network failures → optimistic updates, retry logic
2
// Race conditions → request deduplication
3
// Partial failures → error boundaries, fallback UI
4
// State corruption → immutability, validation
5
// Preventing faults → ESLint, TypeScript strict mode

Scalability in Frontend:

1
// Bundle size → tree shaking, compression, code splitting
2
// State complexity → normalized state, memoization
3
// Real-time updates → efficient diffing, virtualization
4
// Asset optimization → CDN, caching strategies

Maintainability in Frontend:

1
// Code quality → ESLint, Prettier
2
// Type safety → TypeScript, Zod
3
// Component architecture → composition patterns
4
// Testing → unit, integration, E2E
5
// Documentation → Storybook, TSDoc

Key Insight: You’re a Data System Designer

When you build a modern frontend app, you’re composing multiple data systems:

• State managers (Redux, Zustand) - your “database”
• React Query / SWR - your async state and caching layer
• IndexedDB / LocalStorage - persistent storage
• WebSockets - message queues for real-time data
• Service Workers - background processing

Sound familiar? This is exactly the composite data system pattern Kleppmann describes. You’re not just an application developer—you’re a data system designer.

Chapter 2: Data Models and Query Languages

The Three Data Models

Chapter 2 explores how we structure and query data:

1. Relational Model (SQL)

• Data in tables
• Normalized - no duplication
• Requires joins for relationships

2. Document Model (NoSQL/JSON)

• Self-contained documents
• Better locality - everything in one place
• Natural for tree structures

3. Graph Model

• Nodes and edges
• Perfect for interconnected data
• Excels at relationship queries

The LinkedIn Profile Example

The book uses a brilliant example: representing a LinkedIn profile. This decision exists in frontend daily:

1
// Document style (nested)
2
const user = {
3
  id: 251,
4
  name: "Bill Gates",
5
  posts: [
6
    { id: 1, title: "Climate Change", comments: [...] }
7
  ]
8
}
9

10
// Relational style (normalized)
11
const state = {
12
  users: { 251: { id: 251, name: "Bill Gates" } },
13
  posts: { 1: { id: 1, userId: 251, title: "..." } },
14
  comments: { ... }
15
}
16

17
// Graph style (connections)
18
const graph = {
19
  nodes: { users: {...}, posts: {...} },
20
  edges: [
21
    { from: 'user:251', to: 'post:1', type: 'authored' },
22
    { from: 'user:301', to: 'post:1', type: 'liked' }
23
  ]
24
}

The Normalization Tradeoff

The book introduces a critical concept: normalization vs. duplication

Using IDs (normalized):

• No duplication, easy to update, consistency
• Requires lookups/joins

Storing data directly (denormalized):

• Better locality, one query
• Update overhead, inconsistency risk

This is EXACTLY the decision you face with Redux state structure or any similar state management tool:

1
// When to normalize?
2
// Frequent updates to individual entities
3
// Shared references (same user in multiple places)
4
// Many-to-many relationships
5

6
// When to nest?
7
// Read-heavy, display focus
8
// Tree structures
9
// Self-contained data

GraphQL and the Graph Model

GraphQL isn’t just a query language—it’s a graph traversal system:

1
query {
2
  user(id: "1") {
3
    friends {
4
      # Traverse edge
5
      posts {
6
        # Traverse another edge
7
        likedBy {
8
          # Complex graph query
9
          name
10
        }
11
      }
12
    }
13
  }
14
}

Apollo Client’s normalized cache is literally a graph database in your frontend!

Key Insight: State Management IS Data Modeling

Every time you design a Redux store or use the reducer state pattern or decide how to structure component state, you’re choosing a data model. Understanding relational vs. document vs. graph models helps you make that choice consciously, not by accident. Even in situation where you are working with an external API or backend, you face similar consideration of how to structure the data.

Chapter 3: Storage and Retrieval

How Databases Actually Work

Chapter 3 demystifies storage engines with a brilliantly simple example:

1
# The world's simplest database
2
db_set () {
3
  echo "$1,$2" >> database
4
}
5

6
db_get () {
7
  grep "^$1," database | sed -e "s/^$1,//" | tail -n 1
8
}

Write performance: O(1) - append is fast! Read performance: O(n) - scan entire file. Terrible!

This illustrates the fundamental tradeoff in storage systems.

The Write vs. Read Tradeoff

Indexes speed up reads but slow down writes.

• Index = Additional metadata to help locate data
• Every write must update both data AND indexes
• You choose indexes based on query patterns

Sound familiar?

1
// Frontend parallel: Memoization
2
const ExpensiveComponent = ({ data }) => {
3
  // Without memoization (no index) - recompute every render
4
  const processed = expensiveOperation(data);
5

6
  // With memoization (adding an index) - compute once, cache result
7
  const processed = useMemo(() => expensiveOperation(data), [data]);
8
  // Tradeoff: More memory for faster reads
9
};

Storage Engine Patterns in Frontend

1. Append-Only Logs

Redux DevTools, Reducer pattern or a WebSocket client is literally an append-only log!

1
const actions = [
2
  { type: "SET_USER", payload: { id: 1, name: "Alice" } },
3
  { type: "SET_USER", payload: { id: 1, name: "Alice Smith" } }, // Append!
4
  { type: "INCREMENT_COUNTER", payload: 42 },
5
];
6

7
// Time-travel works because we can replay the log

2. Hash Indexes

JavaScript Maps are hash indexes:

1
// O(1) lookup - exactly like database hash index
2
const index = new Map([
3
  ["user:1", { offset: 0, length: 50 }],
4
  ["user:2", { offset: 50, length: 45 }],
5
]);
6

7
const location = index.get("user:1"); // O(1)!

React’s component keys? Hash indexes for reconciliation.

3. SSTables (Sorted String Tables)

IndexedDB is built on LevelDB, which uses SSTables! This is why IndexedDB supports range queries:

1
// Range query - only possible with sorted data
2
const recentPosts = await db.getAllFromIndex(
3
  "posts",
4
  "timestamp",
5
  IDBKeyRange.bound(new Date("2025-01-01"), new Date("2025-12-31")),
6
);
7

8
// Binary search on sorted index - O(log n) instead of O(n)!

4. Segmentation and Compaction

Code splitting is segmentation:

1
// Instead of one giant bundle
2
// main.js (2MB)
3

4
// Split into segments
5
const Dashboard = lazy(() => import("./Dashboard")); // 500KB
6
const Admin = lazy(() => import("./Admin")); // 300KB
7

8
// Load segments on demand - just like storage engines!

Why Sequential Writes Matter

The book emphasizes: “Sequential writes are much faster than random writes.”

This applies to frontend too:

1
// SLOW: Random writes
2
for (let i = 0; i < 1000; i++) {
3
  await db.put("store", { id: i, data: Math.random() });
4
}
5

6
// FAST: Sequential write (batch)
7
const batch = [];
8
for (let i = 0; i < 1000; i++) {
9
  batch.push({ id: i, data: Math.random() });
10
}
11
await db.bulkAdd("store", batch);

Key Insight: Understanding Storage Explains Performance

When you understand how storage engines work, you understand:

• Why immutability simplifies things (append-only is crash-safe)
• Why batch operations are faster (sequential writes)
• Why indexes help some queries but not others (tradeoffs)
• Why Redux DevTools can time-travel (replay append-only log)

Chapter 4: Encoding and Evolution

The Core Challenge: Multiple Versions Running Simultaneously

Modern systems must handle different versions of code running at the same time:

• Backend: Rolling upgrades across servers
• Frontend: Users with old app versions still cached

The requirement: Backward and forward compatibility

Backward compatibility: New code can read old data Forward compatibility: Old code can read new data

The Problems with RPC (Remote Procedure Calls)

Kleppmann explains why trying to make network calls look like local function calls is fundamentally flawed:

Network requests are different from local calls:

• Unpredictable (can fail, timeout)
• Variable latency (milliseconds to seconds)
• May execute multiple times if retried
• Must encode everything as bytes
• Different programming languages

This is your daily reality as a frontend engineer:

1
// Treating network like local function (BAD)
2
function getUser(id) {
3
  return fetch(`/api/users/${id}`).then((r) => r.json());
4
  // What if timeout? Network down? Server error?
5
}
6

7
// Acknowledging network reality (GOOD)
8
function useUser(id) {
9
  return useQuery({
10
    queryKey: ["user", id],
11
    queryFn: () => fetchUser(id),
12
    retry: 3, // Network may fail
13
    timeout: 10000, // May timeout
14
    onError: handleError, // Handle gracefully
15
  });
16
}

REST vs. GraphQL

REST: Doesn’t hide that it’s a network protocol (good!) GraphQL: Learned from RPC’s mistakes—strongly typed, client-controlled, acknowledges network reality

Message Passing: The Better Abstraction

Message brokers (RabbitMQ, Kafka) provide asynchronous communication:

• Buffer messages if recipient unavailable
• Automatic redelivery on failure
• Decouple sender from recipient
• One message to many recipients

Frontend equivalents:

1
// WebSockets = Message broker
2
const ws = new WebSocket('wss://api.example.com')
3
ws.onmessage = (event) => {
4
  const update = JSON.parse(event.data)
5
  dispatch({ type: update.type, payload: update.data })
6
}
7

8
// BroadcastChannel = Pub/Sub between tabs
9
const channel = new BroadcastChannel('updates')
10
channel.postMessage({ type: 'USER_UPDATE', data: {...} })
11

12
// Service Workers = Message broker between tabs
13
navigator.serviceWorker.controller.postMessage({...})

Rolling Upgrades in Frontend

You face the same challenge as backend services:

The scenario:

• User hasn’t refreshed in 3 days (v1.0)
• You deployed v1.1 yesterday
• User’s old code hits new API
• Must maintain backward compatibility!

The solution:

1
// API v1
2
interface User {
3
  id: string;
4
  name: string;
5
}
6

7
// API v2 - Adding field safely
8
interface User {
9
  id: string;
10
  name: string;
11
  email?: string; // Optional! Old clients don't break
12
}
13

14
// Backend handles both versions
15
app.post("/api/users", (c) => {
16
  const body = c.req.json();
17

18
  const user = {
19
    id: generateId(),
20
    name: body.name,
21
    email: body.email || null, // Gracefully handle missing field
22
  };
23

24
  db.insert(user);
25
  c.json(user);
26
});

Key Insight: Frontend Apps Are Distributed Systems

Modern web apps face the same distributed systems challenges:

• Network unpredictability
• Multiple versions running simultaneously
• Need for backward/forward compatibility
• Async communication patterns

Understanding these fundamentals makes you a better frontend engineer.

The Bigger Picture: How It All Connects

Part 1 builds a foundation by progressing through layers:

Chapter 1: What are we trying to achieve?

• Reliable, scalable, maintainable systems

Chapter 2: How do we structure data?

• Choose the right model for your use case

Chapter 3: How do we store and retrieve data?

• Understand the tradeoffs between write and read performance

Chapter 4: How do data and code evolve over time?

• Design for compatibility and change

The Frontend Journey

As a frontend engineer, this foundation transforms how you think about:

Architecture Decisions:

• Normalized vs. nested state (Chapter 2)
• When to add indexes/memoization or caching (Chapter 3)
• API versioning strategies (Chapter 4)

Performance Optimization:

• Bundle splitting = segmentation (Chapter 3)
• Batch operations = sequential writes (Chapter 3)
• Optimistic updates = async messaging (Chapter 4)

Reliability Patterns:

• Error boundaries = fault tolerance (Chapter 1)
• Retry logic = handling network unpredictability (Chapter 4)
• Immutability = crash safety (Chapter 3)

Practical Takeaways: Decision Frameworks

When to Normalize State?

Dos: Normalize when:

• Frequent updates to individual entities
• Shared references (same data in multiple places)
• Many-to-many relationships

Don’ts Keep nested when:

• Read-heavy, display-focused
• Tree structures (one-to-many)
• Self-contained data

When to Add Indexes (Memoization)?

Dos Add indexes when:

• Expensive computations
• Frequently accessed
• Stable inputs (high cache hit rate)

Don’ts Skip indexes when:

• Simple, fast operations
• Constantly changing inputs
• Memory constrained

When to Use Message Passing vs. Request/Response?

Dos Message passing (WebSockets) when:

• Real-time updates
• Server-initiated updates
• Multiple subscribers
• Fire-and-forget communication

Dos Request/Response (REST/GraphQL) when:

• Client-initiated queries
• Need immediate response
• Request/response pattern
• Cacheable data

How to Handle API Evolution?

The Golden Rules:

1. Add fields as optional (backward compatible)
2. Never remove required fields (breaks old clients)
3. Version your APIs (URL, header, or content negotiation)
4. Use feature detection when possible
5. Test with old clients before deploying

Key Patterns to Apply Immediately

1. The Append-Only Log Pattern

1
// Use for: Undo/redo, time-travel debugging, audit trails
2
const eventLog = [];
3

4
function dispatch(action) {
5
  eventLog.push(action); // Append only!
6
  currentState = reducer(currentState, action);
7
}
8

9
function undo() {
10
  // Replay all but last action
11
  const newLog = eventLog.slice(0, -1);
12
  return newLog.reduce(reducer, initialState);
13
}

2. The Normalized Cache Pattern

1
// Use for: Shared data, frequent updates
2
const cache = {
3
  users: { 1: {...}, 2: {...} },
4
  posts: { 1: { userId: 1, ...}, 2: {...} }
5
}
6

7
// Update user once, reflected everywhere
8
cache.users[1].name = "New Name"

3. The Optimistic Update Pattern

// Use for: Variable network latency
3 collapsed lines
const { mutate } = useMutation({
  mutationFn: likePost,
  onMutate: async (postId) => {
    // Update immediately
    queryClient.setQueryData(["post", postId], (old) => ({
      ...old,
      liked: true,
    }));
  },
  onError: (err, postId, context) => {
    // Rollback on failure
    queryClient.setQueryData(["post", postId], context.previous);
  },
});

4. The Compatibility Pattern

1
// Use for: API evolution
2
interface Config {
2 collapsed lines
3
  version: number;
4
  features: string[];
5
  // Add new fields as optional
6
  newFeature?: boolean;
7
}
8

9
function handleConfig(config: Config) {
10
  // Feature detection
11
  if (config.features?.includes("real-time")) {
12
    enableWebSocket();
13
  }
14

15
  // Gracefully handle new fields
16
  if (config.newFeature !== undefined) {
17
    useNewFeature(config.newFeature);
18
  }
19
}

Questions to Ask When Designing Systems

Reliability

• What happens when the network fails?
• How do we handle partial failures?
• What’s our error recovery strategy?
• How do we prevent data corruption?

Scalability

• How does performance change with more data?
• What’s our bundle size strategy?
• How do we handle growing state complexity?
• Where are the bottlenecks?

Maintainability

• Can new team members understand this?
• How do we test this effectively?
• How easy is it to change?
• What’s our documentation strategy?

Data Modeling

• Is this data naturally nested or interconnected?
• How often do we update vs. read?
• Do we have many-to-many relationships?
• What are the query patterns?

Storage

• What’s the read/write ratio?
• Do we need range queries?
• How important is locality?
• What’s the data volume?

Evolution

• How do we version this?
• What’s our compatibility strategy?
• How do we handle rolling upgrades?
• Can old clients still work?

Conclusion: The Foundation That Changes Everything

Part 1 of “Designing Data-Intensive Applications” isn’t just about databases and backend systems. It’s about fundamental principles that apply to any system that stores, processes, and communicates data.

As frontend engineers building production-grade, real-time experiences, we face the same challenges:

• Reliability: Networks fail, users make mistakes, bugs happen
• Scalability: Apps grow in complexity, data volume, and traffic
• Maintainability: Teams change, requirements evolve, systems must adapt

Understanding these foundations transforms how you:

• Design state management (Chapter 2: data models)
• Optimize performance (Chapter 3: storage patterns)
• Handle network communication (Chapter 4: distributed systems)
• Build for change (All chapters: evolution and compatibility)

What’s Next?

Part 2 of the book dives into Distributed Data:

• Chapter 5: Replication
• Chapter 6: Partitioning
• Chapter 7: Transactions
• Chapter 8: The Trouble with Distributed Systems
• Chapter 9: Consistency and Consensus

These chapters become even more relevant as you build:

• Real-time collaborative features
• Offline-first applications
• Multi-region systems
• Complex state synchronization

The Mindset Shift

The biggest takeaway from Part 1? Stop thinking of frontend and backend as separate worlds. They’re both data systems facing the same fundamental challenges, just at different scales and contexts.

When you understand these foundations, you don’t just write code—you design systems. And that’s what separates good engineers from great ones.

Resources for Further Learning

For deeper understanding:

• Martin Kleppmann’s blog
• Jepsen analyses - distributed systems testing
• The Morning Paper - research paper summaries

For frontend-specific applications:

• Redux Toolkit Entity Adapter - normalized state
• TanStack Query - async state management
• Zustand - lightweight state
• IndexedDB API - browser storage

For building real-time systems:

• WebSocket API
• PartyKit - PartyServer, PartySocket, Y-PartyServer, partysub, partysync, partywhen
• Server-Sent Events
• BroadcastChannel API

This summary is based on “Designing Data-Intensive Applications” by Martin Kleppmann. All concepts and examples are translated and adapted for frontend engineering context. For the full treatment and complete understanding, I highly recommend reading the original book.

Happy building! 🚀

Update Changelog