Getting offline right
PhaseLock is a sync engine built around a simple idea: save your events as a log, which is easy to replicate, and write a reducer that builds state from those events, which can be run anywhere. The result is one sync engine for your web app, your mobile app, and even your backend.
We just launched a hosted demo of PhaseLock: a simulated library catalog with complex consistency rules and three interacting clients and offline-mode switches to play with. The app seems boring because it does the obvious thing: if an offline patron tries to hold a book, they see their hold as pending until they are online again, and if the book is then already on hold they see an error message.
But that simple sequence turns out to be a real struggle for many sync engines, and often app authors are caught by surprise.
More and more apps are designed to work locally, and devs are asking, "what do I do with conflicts?" When two offline devices store user operations locally and then try to sync to each other, conflict handling becomes a core pillar of the application.
And we would add a related UX wrinkle: "how do you show the user what state is synced and what is not?" That sets user expectations around how durable their data is (if I lose my phone do I lose my work?) and whether it has been confirmed (can my edit still conflict with another user's edit?). The latter is especially important in collaborative applications.
Let's dig deeper into these challenges and how PhaseLock solves them.
Handling offline conflicts
With a normal online-only CRUD app, conflicts were always possible but didn't degrade user experience so badly. First, they weren't so bad because detecting them was easy to implement; all you needed was a simple transaction that checks some SELECTs before doing UPDATEs. Second, they weren't so bad because the divergence between two online clients was basically limited to a single POST request that delivered instant user feedback.
On the first point, PhaseLock actually does make conflict detection and handling as easy as with an online CRUD app. As a refresher, the write path to PhaseLock looks like:
- A user takes an action in the UI
- The UI encodes that action in a command
- The command may sit in an outbox until it can connect to the server
- The command is processed by a precommit function, which produces events
- Resulting events are committed to the log, and also reduced to update the server state
So conflicts are handled in one function, precommit, which
runs with a read-only transaction of server state. Your
precommit can read current state to check the validity of a
command before emitting results, which is exactly analogous to the CRUD
pattern of a transaction checking SELECTs before running UPDATE.
That means implementing conflict detection feels easy and natural. Your
precommit doesn't have to think about outboxes or offline
mode. It just decides one at a time how each command should be
processed, exactly as a POST handler in a CRUD app works. But unlike a
POST handler, that logic can be reused to give your app free optimistic
updates, as we wrote about last week.
On the second point, the degree of divergence allowed by an offline app, there's unfortunately no fix for that. Two offline apps can accumulate chains of mutations where one early conflict causes cascading invalidations of user work.
Since PhaseLock can't magically prevent conflicts, the best option is to
have application code detect and handle them in one place. There's no
last-write-wins behavior hard-coded deep in a sync algorithm. There's
just your precommit that reads current state, decides if a
command is accepted, and emits events accordingly.
- If you want to use last-write-wins, great.
- If you want to import a CRDT library to resolve conflicts, go for it.
- If you want to record both sides of a conflicting edit, do it.
- If you want to emit a notification to the author of a losing edit, you can.
In the sync apps we've built, we've found that last-write-wins is great for most conflicts, but for certain very important user edits, recording both sides of an edit and/or notifying the user of a conflict is desirable. Which conflicts deserve which treatment is specific to the app domain, and a sync engine needs to let application code decide.
You can see the notification feature in action in our hosted library catalog demo by
switching both patrons to offline, requesting holds for the same book,
and switching them back to online. The first patron switched online
will get the hold and the second will see an error message. The code
that creates that error message is the rejection logic used by
precommit here.
Setting user expectations
Offline apps are great for users because they're always available, even if the network isn't. But the added network complexity can't be hidden from the user. A user needs to know if their edits have been sent to their coworkers or not. They need to know if their photos are synced safely to the cloud or not. The Outlook desktop app has always supported this richer UX: emails written locally stay in the outbox until they are sent to the server.
So a sync engine for offline apps must surface information for the app to distinguish "pending" local-only state vs "confirmed" server state. Ideally, the granularity of what is "pending" is arbitrarily fine; if you have one in-flight entity rename command, you should be able to tell that the name of the entity is pending but its other fields are confirmed.
In PhaseLock, pending state can be arbitrarily fine. It isn't
automatic, but your reducer gets an extra pending flag set
to true when reducing events that were optimistically
generated by a forecast function (as described last week), rather
than originating from the server's event log.
You can see the pending state in action in our hosted library catalog demo by switching a patron offline and requesting a hold on any book. The pending flag for a hold is declared on the type model here, populated in the reducer here, and rendered in the UI here.
There's no code to clear the pending flag because it clears
automatically: when the real event arrives from the server, it is
reduced with pending set to false and the
optimistic state is replaced by confirmed state.
Most sync engines, even those designed for offline mode, get pending state wrong. Some can't tell you what state is pending at all, and most don't have a way to surface it at the field level of granularity.
Conclusion
Here at PhaseLock, we're thrilled to see more offline and local-first applications.
No longer is the server the One True Home of logic and data. Now it may merely be a sync point between devices.
Our phones are more capable than our computers used to be. So why should an application grind to a halt over a flaky network?