PhaseLock

Getting offline right

PhaseLock is a sync engine built around a simple idea: save your events as a log, which is easy to replicate, and write a reducer that builds state from those events, which can be run anywhere. The result is one sync engine for your web app, your mobile app, and even your backend.

We just launched a hosted demo of PhaseLock: a simulated library catalog with complex consistency rules and three interacting clients and offline-mode switches to play with. The app seems boring because it does the obvious thing: if an offline patron tries to hold a book, they see their hold as pending until they are online again, and if the book is then already on hold they see an error message.

But that simple sequence turns out to be a real struggle for many sync engines, and often app authors are caught by surprise.

More and more apps are designed to work locally, and devs are asking, "what do I do with conflicts?" When two offline devices store user operations locally and then try to sync to each other, conflict handling becomes a core pillar of the application.

And we would add a related UX wrinkle: "how do you show the user what state is synced and what is not?" That sets user expectations around how durable their data is (if I lose my phone do I lose my work?) and whether it has been confirmed (can my edit still conflict with another user's edit?). The latter is especially important in collaborative applications.

Let's dig deeper into these challenges and how PhaseLock solves them.

Handling offline conflicts

With a normal online-only CRUD app, conflicts were always possible but didn't degrade user experience so badly. First, they weren't so bad because detecting them was easy to implement; all you needed was a simple transaction that checks some SELECTs before doing UPDATEs. Second, they weren't so bad because the divergence between two online clients was basically limited to a single POST request that delivered instant user feedback.

On the first point, PhaseLock actually does make conflict detection and handling as easy as with an online CRUD app. As a refresher, the write path to PhaseLock looks like:

So conflicts are handled in one function, precommit, which runs with a read-only transaction of server state. Your precommit can read current state to check the validity of a command before emitting results, which is exactly analogous to the CRUD pattern of a transaction checking SELECTs before running UPDATE.

That means implementing conflict detection feels easy and natural. Your precommit doesn't have to think about outboxes or offline mode. It just decides one at a time how each command should be processed, exactly as a POST handler in a CRUD app works. But unlike a POST handler, that logic can be reused to give your app free optimistic updates, as we wrote about last week.

On the second point, the degree of divergence allowed by an offline app, there's unfortunately no fix for that. Two offline apps can accumulate chains of mutations where one early conflict causes cascading invalidations of user work.

Since PhaseLock can't magically prevent conflicts, the best option is to have application code detect and handle them in one place. There's no last-write-wins behavior hard-coded deep in a sync algorithm. There's just your precommit that reads current state, decides if a command is accepted, and emits events accordingly.

In the sync apps we've built, we've found that last-write-wins is great for most conflicts, but for certain very important user edits, recording both sides of an edit and/or notifying the user of a conflict is desirable. Which conflicts deserve which treatment is specific to the app domain, and a sync engine needs to let application code decide.

You can see the notification feature in action in our hosted library catalog demo by switching both patrons to offline, requesting holds for the same book, and switching them back to online. The first patron switched online will get the hold and the second will see an error message. The code that creates that error message is the rejection logic used by precommit here.

Setting user expectations

Offline apps are great for users because they're always available, even if the network isn't. But the added network complexity can't be hidden from the user. A user needs to know if their edits have been sent to their coworkers or not. They need to know if their photos are synced safely to the cloud or not. The Outlook desktop app has always supported this richer UX: emails written locally stay in the outbox until they are sent to the server.

So a sync engine for offline apps must surface information for the app to distinguish "pending" local-only state vs "confirmed" server state. Ideally, the granularity of what is "pending" is arbitrarily fine; if you have one in-flight entity rename command, you should be able to tell that the name of the entity is pending but its other fields are confirmed.

In PhaseLock, pending state can be arbitrarily fine. It isn't automatic, but your reducer gets an extra pending flag set to true when reducing events that were optimistically generated by a forecast function (as described last week), rather than originating from the server's event log.

You can see the pending state in action in our hosted library catalog demo by switching a patron offline and requesting a hold on any book. The pending flag for a hold is declared on the type model here, populated in the reducer here, and rendered in the UI here.

There's no code to clear the pending flag because it clears automatically: when the real event arrives from the server, it is reduced with pending set to false and the optimistic state is replaced by confirmed state.

Most sync engines, even those designed for offline mode, get pending state wrong. Some can't tell you what state is pending at all, and most don't have a way to surface it at the field level of granularity.

Conclusion

Here at PhaseLock, we're thrilled to see more offline and local-first applications.

No longer is the server the One True Home of logic and data. Now it may merely be a sync point between devices.

Our phones are more capable than our computers used to be. So why should an application grind to a halt over a flaky network?