PhaseLock

Introducing PhaseLockDB, part II

Syncing as permissions change

PhaseLock is a sync engine built around a simple idea: save your events as a log, which is easy to replicate, and write a reducer that builds state from those events, which can be run anywhere. The result is one sync engine for your web app, your mobile app, and even your backend.

Last week we introduced PhaseLockDB and discussed how state inside the database and a user-defined precommit() function can replace an entire replicated service in front of your event log.

Today we're going to talk about the most fundamental problem in sync: How does the server choose what updates to send to a client?

"The new ones," you say, "duh." Ok, but hold on... what if the client's permissions are also changing over time?

This problem comes in a couple of cases: the reconnecting case and the online case. Let's imagine a chat application to illustrate:

Once, while trying to bolt sync onto an existing CRUD app, we ran into this problem of dynamically recalculating permissions for long-lived connections. The only solution we trusted ourselves to build was what we called the "just boot'em strategy": close every connection on any change to the permissions tables (we had 5!) and recalculate client subscriptions when they reconnect.

But PhaseLockDB has one simple solution for both cases: your data is organized into slices and permissions are one stateless maySee() function.

PhaseLockDB defines a "slice" of data as: one stream of events, one reducer for those events, and one key prefix populated by that reducer. Your global data model is composed of one or more non-overlapping slices. For the example of a chat application, you might have one slice per channel and a directory slice for all public channels, plus per-user data slices.

Then any client's view of the global state is just a composition of one or more slices, as defined by your maySee() function.

The maySee() function is written by you and runs in the database. It has access to global state and it returns a list of slices or slice prefixes that a given user is allowed to read. For our chat example, it could look like this:

maySee.ts
function* maySee(qx: ChatQueryContext, userId: string, traits: string[]) {
  // admins can see anything
  if (traits.includes("admin")) {
    return ["*"];
  }

  // every user can see their own user slice and the directory slice
  const result = [`user.${userId}/`, `directory/`];

  // every user can see public channels listed in the directory
  for (const ch of (yield* qx.get.directory())) {
    result.push(`channel.${ch}/`);
  }

  // every user can see the private channels they are members of
  const user = yield* qx.get.users(userId);
  for (const ch of user.memberships) {
    result.push(`channel.${ch}/`);
  }

  return result;
}

The result of that maySee() function serves multiple purposes:

Additionally, the keys read by maySee() are tracked, and any time a dependent key is modified, the function is rerun and the new result takes effect. So tracking permissions changes on a long-lived connection is cheap and correct.

If you've been reading our examples, you'll see that the maySee() function is actually just a normal PhaseLock query. It's the exact same mechanism that frontend code uses to feed a reactive UI, only this query runs in the server.

It's worth pointing out the drawback of slices too, which is that when precommit() processes a client command that affects multiple slices, it must then emit separate events to each slice. They all land in the same transaction, so everything stays consistent, but it is a cost.

But what slices buy you is big: one stateless function fully defines your permissions, every client view is a simple composition of slices, and most importantly: slices answer all the common problems that event-log-based solutions face. More on that next week.