Why I Keep My Data in Text Files (And You Should Too)

Databases have become a cult, and nobody in the congregation is allowed to ask the obvious question out loud: do you actually have the traffic to justify any of this? Somewhere along the way, "cloud-hosted, multi-region, horizontally-sharded SQL cluster" stopped being an answer to a scaling problem and became the default starting point, the thing you reach for on day one, before you've shipped a single feature, before you have a single user, before you have any evidence at all that a single Postgres instance on a single machine would have buckled under your load. It won't. For the overwhelming majority of apps that will ever be built, it never will.

The Single Point of Failure You Already Have

Here's the sales pitch for the distributed database: resilience. No single point of failure. Automatic failover. Multi-region replication so a data center fire in one continent doesn't take down your app in another. It's a genuinely compelling pitch, right up until you notice that you've traded one single point of failure for a dozen new ones you don't control and can't debug at 2am.

Your managed database cluster is a single point of failure sitting behind an API you don't own, maintained by an operations team you'll never speak to, with an incident response time measured against their SLA, not your users' patience. When it goes down, and managed cloud databases do go down, entire regions of them, on schedules nobody announces in advance, you get to sit in a status page refresh loop with everyone else who trusted the same abstraction, unable to do anything but wait. That's not resilience. That's outsourced fragility with better marketing.

A single SQLite file sitting on disk has exactly one failure mode: the disk fails. You know exactly what that failure mode looks like, you can test it, you can back up against it trivially, and you are never, under any circumstances, waiting on someone else's incident response team to fix your own database.

The Math Nobody Runs Before Choosing Postgres-as-a-Service

Modern SQLite, properly indexed, on a single machine with a reasonable SSD, handles tens of thousands of reads per second and comfortably into the thousands of writes per second for the overwhelming majority of app workloads, single-writer constraints and all. That's not a cute benchmark, that's a number most side projects, most internal tools, and most early-stage products will never come close to touching in their entire lifetime.

Ask yourself honestly what your actual concurrent write load looks like. Not your aspirational, Series-B-pitch-deck, hockey-stick-growth-chart write load. Your real one, this month, with your real user count. For a huge share of apps that answer is somewhere between "a few dozen writes a minute" and "a few hundred a day," a load that a laptop from four years ago running SQLite off a local SSD wouldn't even register as work. You bolted on a distributed cluster to solve a scaling problem you don't have, and in exchange you paid a very real cost: network round trips on every single query, a whole new class of connection-pool exhaustion bugs, a monthly bill, and an entire operational surface area (backups, replicas, failover, migrations across a live cluster) that exists purely to serve traffic you were never going to see.

Backups Are Where the Cult Really Falls Apart

Ask a team running a managed multi-region cluster to walk you through their actual disaster recovery process, step by step, from "the primary region is gone" to "the app is serving correct data again." Watch how long the explanation takes. Watch how many acronyms show up. Watch how much of it depends on a vendor dashboard staying reachable during the exact outage you're trying to recover from.

Now ask what it takes to back up a flat file or a single SQLite database. It's a file. You copy the file. cp data.sqlite backup/data-$(date +%s).sqlite, cron it, ship the copy somewhere durable, done. Restoring is copying it back. There is no replication lag to reason about, no eventual-consistency window where a read might return stale data depending on which replica answered, no multi-step runbook with steps that can silently fail out of order. The entire backup and restore story fits in one line of shell and is fully understandable by a single engineer with zero specialized distributed-systems training.

This isn't a toy-scale argument either. Companies running SQLite in production at meaningful scale routinely point to exactly this property: a database you can literally hand someone as a single file is a database whose failure modes you can actually reason about, instead of trusting to a vendor's internal architecture you'll never get to inspect.

Where the Cult Has a Point (And Where It Stops)

None of this is an argument that distributed databases are useless. If you are actually running multi-region traffic in the millions of requests per day, with genuine concurrent-write contention from thousands of simultaneous users hammering the same rows, a single-file database is the wrong tool, and pretending otherwise is its own kind of cult behavior in reverse. Real scale is real, and when it shows up, the tradeoffs flip: network overhead becomes worth it, operational complexity becomes worth it, because the alternative is an actual outage caused by an actual bottleneck.

The problem was never that distributed databases are bad engineering. The problem is that "distributed database" became the reflexive answer to "how should I store data," asked and answered before anyone checked whether the question needed that answer. It's cargo cult architecture: copying the infrastructure choices of companies operating at a scale you don't have, because their blog posts are the ones that get shared, not because your actual traffic graph has anything in common with theirs.

What This Looks Like in Practice

For a genuinely large share of apps, especially the solo-developer, early-stage, "I have an idea and I'm building it" category, the honest architecture looks something like this: SQLite as the primary datastore, WAL mode enabled for better concurrent read/write behavior, a properly designed schema with real indexes (the "properly indexed" part isn't optional, an unindexed flat file is genuinely slow, an indexed one is not), and a simple, boring, cron-driven backup script shipping copies to durable object storage on a schedule.

That setup is faster than a network-hop-per-query cloud database for the vast majority of real-world query patterns, because there's no network hop at all, the query executes on the same machine as the code calling it. It's cheaper, because you're paying for a disk instead of a managed service markup. It's easier to reason about during an incident, because the entire failure surface is a single file on a single disk instead of a distributed system's worth of moving parts you didn't build and can't fully see into. And it's easier to migrate away from later, if and when you actually hit the scale that justifies something bigger, because a well-structured SQLite schema translates to Postgres in an afternoon, not a rewrite.

The distributed cluster isn't wrong. It's premature, in the overwhelming majority of cases where teams reach for it, applied to a scale problem that exists only in the pitch deck, not in the traffic logs. Reach for the flat file first. Reach for SQLite first. Let real, measured, undeniable load be the thing that eventually forces your hand toward something more complex, instead of letting a blog post from a company processing a billion requests a day make that decision for you before you've shipped anything at all.