What is a Database
A database is an organized collection of data, stored and accessed electronically, managed by a Database Management System (DBMS). The DBMS handles storage, retrieval, concurrency, integrity, and security so applications don't have to.
Why Use a Database (not files)
- Querying — ask complex questions of the data (SQL or query APIs).
- Concurrency — many users read/write safely at once.
- Integrity — constraints and transactions keep data valid.
- Durability & recovery — survive crashes; back up and restore.
- Scale & performance — indexes, caching, replication, sharding.
Database Types
Databases come in families, each optimized for a data shape and access pattern.
| Type | Stores data as | Examples | Best for |
|---|---|---|---|
| Relational (SQL) | Tables, rows, columns | MySQL, PostgreSQL | Transactions, joins, structured data |
| Document | JSON-like documents | MongoDB, Couchbase | Flexible schemas, nested data |
| Key-Value | Key → value | DynamoDB, Redis | Ultra-fast lookups, caching |
| Wide-Column | Rows with dynamic columns | Cassandra, HBase | Huge write volume, time-series |
| Graph | Nodes & edges | Neo4j, Neptune | Relationships, recommendations |
| Search | Inverted index | Elasticsearch | Full-text search, analytics |
| Time-series | Timestamped points | InfluxDB, Timestream | Metrics, IoT |
SQL vs NoSQL
The fundamental database decision. SQL (relational) databases use fixed schemas, tables, and strong consistency; NoSQL databases trade some of that for flexibility and horizontal scale.
Side by Side — Visual
Comparison
| SQL | NoSQL | |
|---|---|---|
| Schema | Fixed, predefined | Flexible / schema-less |
| Scaling | Vertical (mostly) | Horizontal (built-in) |
| Consistency | Strong (ACID) | Often eventual (BASE) |
| Joins | Yes | No (denormalize/embed) |
| Query | SQL (standard) | Per-database APIs |
| Best for | Transactions, complex queries | Scale, flexible/large data |
ACID vs BASE
Two consistency philosophies. ACID (relational) prioritizes correctness; BASE (many NoSQL) prioritizes availability and scale.
| ACID | BASE |
|---|---|
| Atomicity — all or nothing | BAsically Available — responds always |
| Consistency — always valid state | Soft state — may change over time |
| Isolation — txns don't interfere | Eventual consistency — converges later |
| Durability — survives crashes | — |
CAP Theorem
A distributed database can guarantee at most two of: Consistency, Availability, Partition tolerance. Since network partitions are inevitable, the real choice under a partition is C vs A.
| Choice | Behaviour | Databases |
|---|---|---|
| CP | Consistent; may reject requests during partition | MongoDB (default), HBase |
| AP | Available; may return stale data | Cassandra, DynamoDB (eventual) |
| CA | Only without partitions | Single-node MySQL |
Data Modeling Basics
Data modeling is deciding how to structure data for your access patterns. The approach flips between SQL and NoSQL.
| SQL (model the data) | NoSQL (model the queries) | |
|---|---|---|
| Start from | Entities & relationships | Access patterns / queries |
| Goal | Normalize (avoid redundancy) | Denormalize (avoid joins) |
| Relationships | Foreign keys + joins | Embed or duplicate data |
Which Database to Use
The most important database skill: matching the engine to your workload. Here's a practical decision guide for our four databases.
Decision Flow — Visual
Pick By Need
- MySQL — you need ACID transactions, joins, strong consistency, structured/relational data (orders, payments, users). Default for most apps.
- MongoDB — flexible/evolving schemas, nested/hierarchical documents, fast iteration (catalogs, content, profiles).
- Cassandra — write-heavy at massive scale, multi-datacenter, always-on availability, time-series/event data (sensor logs, messaging).
- DynamoDB — serverless key-value/document at any scale, predictable single-digit-ms latency, deep AWS integration, minimal ops.
Comparison Matrix
A head-to-head of the four databases across the dimensions that matter most.
| MySQL | MongoDB | Cassandra | DynamoDB | |
|---|---|---|---|---|
| Model | Relational | Document | Wide-column | Key-value/doc |
| Schema | Fixed | Flexible | Flexible (per-table) | Flexible |
| Scaling | Vertical + read replicas | Horizontal (sharding) | Horizontal (masterless) | Horizontal (managed) |
| Consistency | Strong (ACID) | Strong (CP-leaning) | Tunable (AP) | Eventual + strong opt |
| Transactions | Full ACID | Multi-doc (4.0+) | Limited (LWT) | Limited (per-region) |
| Query | SQL (joins) | Rich query + aggregation | CQL (no joins) | Key lookups (no joins) |
| Writes | Good | Good | Excellent (write-optimized) | Excellent |
| Ops | Self/managed (RDS) | Self/Atlas | Heavy self-managed | Fully serverless |
| Sweet spot | OLTP, complex queries | Flexible app data | Massive writes, multi-DC | Serverless scale on AWS |
Polyglot Persistence
Polyglot persistence means using multiple databases in one system, each for what it does best — rather than forcing everything into one engine.
MySQL Overview
MySQL is the world's most popular open-source relational database — tables, SQL, and ACID transactions (via InnoDB). The default choice for structured, transactional data.
- Relational — data in tables linked by keys; queried with SQL.
- ACID via InnoDB — reliable transactions, foreign keys, row-level locking.
- Mature ecosystem — replication, tooling, cloud (RDS/Aurora), Spring Data JPA.
When to Use MySQL
✓ Use MySQL when
- You need ACID transactions (payments, orders)
- Data is structured & relational
- You need complex queries & joins
- Strong consistency matters
- Team knows SQL (most do)
✗ Consider alternatives when
- Schema changes constantly → MongoDB
- Write volume exceeds one server → Cassandra
- You want zero DB ops on AWS → DynamoDB
- Data is graph/search/time-series shaped
Introduction to MongoDB
MongoDB is the leading document database. It stores data as flexible, JSON-like documents (BSON) instead of rows in tables — so a record and its nested data live together, with no fixed schema.
Relational vs Document — Visual
Why MongoDB
- Flexible schema — documents in a collection can differ; evolve without migrations.
- Natural for objects — maps directly to app objects/JSON (no ORM impedance mismatch).
- Horizontal scale — built-in sharding & replica sets.
- Rich queries — secondary indexes, aggregation pipeline, geospatial, text search.
Documents & Collections
A document is a BSON (binary JSON) object; a collection is a group of documents (like a table without a fixed schema).
| SQL | MongoDB |
|---|---|
| Database | Database |
| Table | Collection |
| Row | Document |
| Column | Field |
_id (auto-generated ObjectId if you don't supply one). BSON adds types JSON lacks — dates, ObjectId, binary, decimal.CRUD Operations
$set, $inc, $push, $pull) modify specific fields without rewriting the whole document. upsert: true inserts if no match is found.Query Operators
| Category | Operators |
|---|---|
| Comparison | $eq $ne $gt $gte $lt $lte $in $nin |
| Logical | $and $or $not $nor |
| Element/Array | $exists $type $all $elemMatch $size |
{ field: 1 }) to fetch only needed fields — like avoiding SELECT *.Indexing
Indexes (B-tree) speed up queries — same principle as SQL. Without one, MongoDB does a full collection scan.
explain() to verify usage. Over-indexing slows writes.Aggregation Pipeline
The aggregation pipeline processes documents through a sequence of stages, each transforming the stream — MongoDB's equivalent of SQL GROUP BY + analytics, but far more powerful.
Pipeline Flow — Visual
| Stage | Does (SQL analogue) |
|---|---|
$match | Filter (WHERE) |
$group | Aggregate (GROUP BY + aggregates) |
$sort / $limit | ORDER BY / LIMIT |
$project | Select/reshape columns |
$lookup | Left join to another collection |
$unwind | Flatten an array into rows |
$match (and $sort on indexed fields) early so the pipeline uses indexes and processes fewer documents. $lookup gives you joins when you really need them — but heavy use suggests the data should have been embedded.Schema Design (Embedding vs Referencing)
The central MongoDB design decision: embed related data inside a document, or reference it in another collection. Get this right and MongoDB flies; get it wrong and you fight the database.
Embed vs Reference — Visual
Which to Choose
| Embed when… | Reference when… |
|---|---|
| Data is read together | Data is large or accessed independently |
| 1-to-few relationship | 1-to-many / many-to-many (unbounded) |
| Child has no life of its own | Child is shared across parents |
| You want atomic updates | Document would grow without bound |
Replica Sets
A replica set is a group of MongoDB nodes holding the same data — one primary (writes) and several secondaries (replicate & serve reads). Provides high availability and read scaling.
Sharding
Sharding partitions a collection across multiple machines (shards) by a shard key — enabling horizontal scale beyond one server for huge datasets/throughput.
- Shard key — determines how documents distribute; choose one with high cardinality and even access (avoid hotspots).
- mongos router — routes queries to the right shard(s).
- Config servers — store cluster metadata.
- Each shard is itself a replica set (HA + scale).
Transactions
Since v4.0, MongoDB supports multi-document ACID transactions — though for many apps a well-embedded document makes them unnecessary (single-document writes are already atomic).
When to Use MongoDB
✓ Use MongoDB when
- Schema evolves rapidly / varies per record
- Data is hierarchical / nested (documents)
- You want fast iteration without migrations
- Read patterns fit "fetch one rich document"
- You need easy horizontal scaling
✗ Avoid when
- Heavy multi-entity transactions & joins → MySQL
- Highly relational data with many links
- You need strict, enforced schema constraints
- Extreme write scale across DCs → Cassandra
Introduction to Cassandra
Apache Cassandra is a distributed wide-column NoSQL database built for massive write throughput, linear horizontal scale, and no single point of failure. Originally from Facebook; now widely used at internet scale (Netflix, Apple, Discord).
- Masterless — every node is equal (peer-to-peer ring); no primary bottleneck.
- Always-on — survives node/datacenter failures; multi-DC native.
- Write-optimized — extremely fast writes (append-only log + memtable).
- AP (CAP) with tunable consistency per query.
Cassandra Architecture
Cassandra runs as a masterless ring of equal nodes. Data is distributed by hashing the partition key; each node owns a range of the hash space. Any node can serve any request (acting as coordinator).
- Partitioner hashes the partition key → places data on a node (and its replicas).
- Replication factor (RF) — how many nodes hold each row's copy.
- Gossip protocol — nodes share state; no leader election needed.
- Storage: commit log → memtable → SSTables (append-only; compaction merges them).
Wide-Column Data Model
Data lives in tables grouped into keyspaces. Rows are grouped into partitions; within a partition, rows can have many columns. You design one table per query — denormalization and duplication are normal.
| Cassandra | Roughly like |
|---|---|
| Keyspace | Database (sets replication) |
| Table | Table (but query-specific) |
| Partition | Group of rows on one node |
| Row / Column | Row / Column |
Partition & Clustering Keys
The primary key drives everything in Cassandra — data placement and sort order. It splits into a partition key (which node) and optional clustering columns (sort within the partition).
CQL (Cassandra Query Language)
CQL looks like SQL but is intentionally limited — no joins, no arbitrary WHERE, no aggregation across partitions.
JOIN or GROUP BY across partitions, and ALLOW FILTERING (full scan) is a red flag in production. Model tables so every query targets one partition.Tunable Consistency
Cassandra lets you choose the consistency level per query — how many replicas must respond before success. You tune the consistency/availability/latency trade-off.
| Level | Meaning |
|---|---|
| ONE | 1 replica responds (fast, weak) |
| QUORUM | Majority of replicas (balanced) |
| LOCAL_QUORUM | Majority in the local DC (common) |
| ALL | Every replica (strong, slow) |
LOCAL_QUORUM is the typical multi-DC default.Replication
Each keyspace sets a replication factor (RF) — how many copies of each row exist across the ring — and a strategy for placing them (often per datacenter).
NetworkTopologyStrategy spreads replicas across datacenters for multi-region resilience. Replication + tunable consistency together deliver "always-on" with controllable correctness.When to Use Cassandra
✓ Use Cassandra when
- Write volume is huge & constant
- You need linear horizontal scale
- Always-on / multi-datacenter availability
- Time-series, logs, events, messaging, IoT
- Access patterns are known & query-shaped
✗ Avoid when
- You need joins / ad-hoc queries → MySQL
- Strong transactions across rows
- Access patterns are unknown/changing
- Small data or a small team (ops-heavy)
Introduction to DynamoDB
Amazon DynamoDB is a fully managed, serverless key-value & document NoSQL database. It delivers single-digit-millisecond latency at any scale with zero servers to manage — you just create tables and AWS handles everything else.
- Serverless — no provisioning, patching, or scaling ops; pay per use.
- Predictable performance — consistent low latency regardless of size.
- Scales automatically — from a few requests to millions/sec.
- AWS-native — IAM, Lambda, Streams, global tables.
Partition & Sort Keys
Every DynamoDB item is located by its primary key: a partition key (which physical partition) and an optional sort key (order within that partition). Key design is the data model.
| Key type | Use |
|---|---|
| Partition key only | Simple key-value lookup (one item per key) |
| Partition + Sort key | Many items per partition, queryable by range/prefix |
Query within a partition key (+ sort key conditions). A full-table Scan is slow and costly. Choose a high-cardinality, evenly-accessed partition key to avoid "hot partitions". This mirrors Cassandra's partition/clustering keys.Capacity Modes
DynamoDB bills throughput in two modes:
| Mode | How it works | Best for |
|---|---|---|
| On-Demand | Pay per request; auto-scales instantly | Spiky/unknown traffic, simplicity |
| Provisioned | Set RCU/WCU (+ auto-scaling) | Predictable traffic, lowest cost |
Secondary Indexes (GSI & LSI)
Secondary indexes let you query by attributes other than the primary key — essential since base-table queries are limited to the key.
| GSI (Global) | LSI (Local) | |
|---|---|---|
| Partition key | Different from base table | Same as base table |
| Created | Anytime | Only at table creation |
| Consistency | Eventual only | Strong possible |
| Use | Query by any attribute | Alternate sort key |
Single-Table Design
DynamoDB's signature (and counter-intuitive) pattern: store multiple entity types in one table, using generic key names and key prefixes — so related items share a partition and one query fetches them together (since there are no joins).
DynamoDB Streams
Streams capture an ordered log of item-level changes (insert/modify/delete), which a Lambda can consume in near-real-time — enabling event-driven architectures.
Consistency
DynamoDB offers two read consistency models per request:
| Model | Behaviour | Cost |
|---|---|---|
| Eventually consistent (default) | May briefly return stale data | Cheaper, faster |
| Strongly consistent | Always latest committed value | 2× read cost, same region |
When to Use DynamoDB
✓ Use DynamoDB when
- You're on AWS & want zero DB ops
- Serverless apps (Lambda) needing scale
- Predictable single-digit-ms latency at scale
- Key-value / known access patterns
- Spiky traffic (on-demand auto-scales)
✗ Avoid when
- You need joins / complex ad-hoc queries → MySQL
- Access patterns are unknown / will change a lot
- Not on AWS (it's AWS-only)
- Rich analytics / reporting (use a warehouse)