PhaseLock

Introducing PhaseLockDB, part II

Syncing as permissions change

PhaseLock is an open-source backend for live apps that still work offline. You declare your types and write your business logic once, then PhaseLock's sync engine powers your web app, your mobile app, and even your backend workers.

Last week we introduced PhaseLockDB and discussed how state inside the database and a user-defined decide() function can replace an entire replicated service in front of your database.

Today we're going to talk about the most fundamental problem in sync: How does the server choose what updates to send to a client?

"The new ones," you say, "duh." Ok, but hold on... what if the client's permissions are also changing over time?

This problem comes in a couple of cases: the reconnecting case and the online case. Let's imagine a chat application to illustrate:

Once, while trying to bolt sync onto an existing CRUD app, we ran into this problem of dynamically recalculating permissions for long-lived connections. The only solution we trusted ourselves to build was what we called the "just boot'em strategy": close every connection on any change to the permissions tables (we had 5!) and recalculate client subscriptions when they reconnect.

But PhaseLockDB has one simple solution for both cases: your data is organized into slices and permissions are one stateless grant() function.

PhaseLockDB defines a "slice" of data one key prefix in your store. Your global data model is composed of one or more non-overlapping slices. For the example of a chat application, you might have one slice per channel, plus one user-private slice for each user.

Then any client's view of the global state is just a composition of one or more slices, as defined by your grant() function.

The grant() function is written by you and runs in the database. It has access to global state and it returns a list of slices or slice prefixes that a given user is allowed to read. For our chat example, it could look like this:

grant.ts
function* grant(txn: ChatTxn, userId: string) {
  const user = txn.get.user(userId);

  // admins can see anything (the bare "*" stream name glob)
  if (user.isAdmin) {
    return ["*"];
  }

  // every user can see their own user slice
  // (the final "!" character indicates an exact stream name)
  const result = [`user:${userId}!`];

  // every user can see the private channels they are members of
  for (const {channelId} of txn.scan.user_channels(userId, {keys: true})) {
    result.push(`channel:${channelId}!`);
  }

  return result;
}

The result of that grant() function serves multiple purposes:

Additionally, the keys read by grant() are tracked, and any time a dependent key is modified, the function is rerun and the new result takes effect. So tracking permissions changes on a long-lived connection is cheap and correct.

If you've been reading our examples, you'll see that the grant() function is actually just a normal PhaseLock query. It's the exact same mechanism that frontend code uses to feed a reactive UI, only this query runs in the server.

It's worth pointing out the drawback of slices too, which is that when decide() processes a client mutation and needs to make that mutation's effect known to multiple clients who aren't granted any slices in common, the decider has to write the effect into multiple slices so each client sees the result. All of those edits land in the same transaction automatically, so everything stays consistent, but it is a cost.

But what slices buy you is big: one stateless function fully defines your read permissions and every client view is a simple composition of slices. They add sync-friendly boundaries to your data, offering a foundation for PhaseLock's sync protocl between PhaseLockDB and our sync engine. More on that next time.