PhaseLock

Introducing PhaseLockDB, part I

PhaseLock is a sync engine built around a simple idea: save your events as a log, which is easy to replicate, and write a reducer that builds state from those events, which can be run anywhere. The result is one sync engine for your web app, your mobile app, and even your backend.

We're one week past our launch and there's so much to do.

The PhaseLock sync engine was developed under Kurrent's banner, and so originally intended to use KurrentDB as the database. Now that we're independent from Kurrent, we think a custom database would suit our sync engine perfectly.

A custom database is a natural fit for how PhaseLock works. On the client side, PhaseLock builds local state by feeding events to reducer functions. Our custom database will do the exact same thing with the exact same reducer functions.

"But wait," you say, "PhaseLock does all replication by streaming events from server to client, so why build state on the server if you only send the events anyway?"

Great question, glad you asked. Building state in the server has a few huge implications. Today we'll discuss consistency.

When the event log lived in a separate database, you were unable to make accept/reject decisions on incoming commands within the transaction loop. Whatever service checked commands (basically, incoming user requests) and wrote events had to live outside the database. As soon as you had two replicas, each checker lost the ability to know the "current" state, as there was a short delay between when one replica wrote an event and when the other replica received it.

But with a custom db, state can be maintained directly in the database, and a user-defined precommit function can do the same checks synchronously. This turns a thorny multi-server async problem into a single TypeScript function.

So one precommit function in the right place replaces a whole replicated service layer in the KurrentDB-based architecture.

"But wait," you say, "won't a JavaScript function running serially inside the database cause a massive slowdown?"

Great question, glad you asked. It turns out that the precommit function only has to run as-if it were serialized. We can run precommit on each incoming command in parallel and commit them in order, with one rule: we must track which keys of state each precommit() call read, and rerun it if we find that it depended on a key that was updated by any events that landed before it. This is roughly what Postgres SERIALIZABLE isolation level does, only our retries are handled within our database rather than sent back to a client.

Based on some exploratory benchmarks, and assuming conflicts are sparse, we expect the parallelized CPU time per event to be less than the storage write time per event, meaning you get a single-threaded TypeScript experience with no cost to throughput.

For apps built around the PhaseLock sync engine, this custom db means the whole stack gets even simpler. You'll have fewer services to run, less code to write, and a simpler mental model.