Database and Storage

Databases, storage engines, and data access tradeoffs covering consistency, indexing, replication, durability, and system behaviour at scale.

7 Database Scaling Strategies

Seven database scaling strategies across replicas, sharding, caching, and partitioning.

Database OptimizationDatabase Scaling

ACID Transaction Guarantees

ACID transaction guarantees for atomicity, consistency, isolation, and durability.

AcidDatabases

Database Performance Cheat Sheet

Database performance through indexing, query shape, caching, and contention control.

DatabasePerformance

Database Lock Types

Database locks compared by compatibility, granularity, and concurrency control.

Concurrency ControlDatabase Locking

Database Sharding Concepts

Database sharding through shard keys, fan-out queries, and rebalancing.

Database DesignDatabase Sharding

Types of Databases

Database types compared by data model, query pattern, and consistency needs.

DatabaseNosql

Consistent Hashing

Consistent hashing for stable key placement as nodes join, leave, or fail.

Consistent HashingDistributed Systems

CAP Theorem Misconceptions

CAP theorem under network partitions, and why pick-two thinking is misleading.

Cap TheoremDistributed Systems

Message Queue Evolution: IBM MQ, RabbitMQ, Kafka, and Pulsar

Message queue evolution from brokered delivery to partitioned event logs.

Message QueuesSystems Atlas

Database Sharding Overview

Database sharding through partition keys, routing, rebalancing, and cross-shard tradeoffs.

DatabaseSharding

Read Replica Pattern Implementation

Read replica routing through database middleware, lag control, and failover.

DatabaseRead Replicas

8 Data Structures Used in Databases

Eight database data structures for indexing, storage layout, caching, and query access.

Data StructuresDatabases

iQIYI Database Selection Tree

Database selection through workload-driven choices across relational and NoSQL.

Database SelectionIqiyi

PostgreSQL as a General-Purpose Database

PostgreSQL adoption through transactional breadth, extensions, and consolidation.

DatabasesPostgreSQL

PostgreSQL's Enduring Popularity

PostgreSQL combines reliable transactions, rich features, and room to grow.

DatabasePostgreSQL

Serverless Databases

Serverless databases scale managed compute and storage with demand.

DatabaseServerless

SQL Statement Execution in a Database

SQL execution through parsing, optimisation, storage engines, and concurrency control.

Database InternalsSQL

6 Database Models

Database models for flat, hierarchical, network, relational, object, and document data.

Data StructuresDatabase Models

4 Data Sharding Algorithms

Data sharding algorithms for key placement, hotspot control, and shard rebalancing.

AlgorithmsData Sharding

Types of Memory

Memory hierarchy from registers to storage, with speed, size, and cost tradeoffs.

Memory ManagementSystem Architecture

Cloud Database Cheat Sheet

Cloud database types mapped to workload shape, query model, and operations.

Cloud ComputingDatabases

Database Type Selection

Database selection through workload shape, consistency needs, and access patterns.

Database SelectionDatabase Types

B-Tree vs. LSM-Tree

B-tree and LSM-tree tradeoffs for reads, writes, compaction, and storage layout.

Data StructuresDatabases

Eventual Consistency Patterns

Eventual consistency patterns for asynchronous propagation, sync, and replica convergence.

ConsistencyDatabases

Six Object Storage Use Cases

Six common object storage workloads, from archives and backups to data lakes.

Data ManagementObject Storage

Storage Systems Overview

Block, file, and object storage compared through access model and system tradeoffs.

Data StorageStorage Systems

Kafka Message Loss Scenarios

Kafka message loss scenarios across producers, brokers, replication, and consumers.

KafkaMessage Loss

Change Data Capture for Real-Time Data

Change Data Capture from database logs to downstream streams and derived systems.

Data StreamingData Synchronisation

Message Delivery Semantics

Delivery semantics across loss, duplication, retries, and processing guarantees.

Delivery SemanticsMessage Queues

Erasure Coding

Erasure coding for durable storage with parity fragments and repair trade-offs.

Data RedundancyData Storage

Event Sourcing System Design

Event sourcing as append-only state, replay, projections, and audit history.

Event SourcingSystems Atlas

Time-Series Database (TSDB) in 20 Lines

Time-series database structure for timestamped writes, tags, retention, and queries.

DatabaseTimeseries

Four Queue Types

Four queue structures and the insertion and removal rules that define them.

Data StructuresQueues

Kafka Performance Architecture

Kafka stays fast through append-only logs, batching, partitions, and page cache.

KafkaPerformance

Six Data Management Patterns

Data management patterns for caches, replicas, indexes, and analytical stores.

Data ManagementData Patterns

Database Isolation Levels

Isolation levels explained through read anomalies, locking, and concurrency tradeoffs.

DatabasesTransactions

Kafka Fundamentals

Kafka fundamentals for logs, brokers, partitions, producers, and consumers.

Distributed SystemsKafka

Data Migration with Avro

Apache Avro schema evolution for compatible data migration between systems.

Apache AvroData Migration

Types of Message Queues

Message queue types compared by delivery model, retention, and broker behaviour.

Message QueueMessaging Systems

Large File Uploads to S3

Large S3 uploads through multipart transfers, parallel parts, and retries.

Object StorageS3

Pessimistic vs. Optimistic Locking

Pessimistic and optimistic locking compared by contention, retries, and consistency.

Concurrency ControlDatabase Transactions

Read Replica Pattern

Read replica architecture for scaling queries away from the primary database.

Database ReplicationRead Scalability