json-patch

Immutable JSON Patch implementation based on RFC 6902 which adds operational transformation (OT) and last-writer-wins (LWW) support for syncing between client and server. Does not support the full OT algorithm because copy and move operations cannot be transformed correctly in all cases, so operations must always be applied in correct order. This means a central server is required to determine order.

The JSON Patch implementation was originally from https://github.com/mohayonao/json-touch-patch which is no longer supported. It was refactored heavily and converted to TypeScript.

Features

• Immutable: The original JSON is not changed and data is shared as much as possible.
• Rollback: If an error occurs, all patches are rejected unless requested. Return the original JSON.
• Customizable: You can add custom operators.
• Patch API: A JSONPatch object to simplify the creation and transformation of patches.
• Multiplayer: You can transform patches against each other for collaborative systems using Operational Transformation (OT).
• Syncable: You can sync objects across server-clients using last-writer-wins (LWW) at the field level.

Installation

$ npm install --save @typewriter/json-patch

Quick Start

The easiest way to use json-patch is with the JSONPatch API.

1import { JSONPatch } from '@typewriter/json-patch';
2
3const prevObject = { baz: 'qux', foo: 'bar' };
4
5const patch = new JSONPatch();
6patch.replace('/baz', 'boo');
7
8const nextObject = patch.apply(prevObject);
9// → { baz: "boo", foo: "bar" }
10//              |
11//             replaced
12
13console.log(prevObject);
14// → { baz: "qux", foo: "bar" }
15//              |
16//

Operational Transformation Quick Start

Using OT with JSON Patch requires operations to be applied in the same order on the server and across clients. This requires clients to keep a last-known-server version of the object in memory or storage as well as a current-local-state version of the object in memory or storage. The first is for applying changes in order and the second is for the app to have the current local state. A version/revision number should be used to track what version of the data a change was applied to in order to know what changes to transform it against, if any. As this is an advanced topic, a bare minimum is provided here to display usage of the API.

1// client.js
2import { JSONPatch } from '@typewriter/json-patch';
3
4// The latest version synced from the server
5let committedObject = { baz: 'qux', foo: 'bar' };
6let rev = 1;
7
8// Start off using this version in our app
9let localObject = committedObject;
10
11const localChange = new JSONPatch();
12localChange.replace('/baz', 'boo');
13
14// Update app data immediately
15localObject = patch.apply(committedObject);
16
17// Receive a change patch from the server
18const { patch: serverChange, rev: latestRev } = getChangeFromServer();
19
20// Apply server changes to our committed version
21committedObject = serverChange.apply(committedObject);
22rev = latestRev; // Keep track of the revsion so the server knows whether to transform incoming changes
23
24// Transform local changes against committed server changes
25const localChangeTransformed = serverChange.transform(committedObject, localChange);
26
27// Re-apply local changes to get the new version
28localObject = localChangeTransformed.apply(committedObject);
29
30// Send local change to server with the revision it was applied at
31sendChange(localChangeTransformed, rev);

Low-level API

If you don't want to use JSONPatch you can use these methods on plain JSON Patch objects.

• applyPatch(prevObject: object, patches: object[], [ opts: object ]): object
  • opts.custom: object custom operator definition.
  • opts.partial: boolean not reject patches if error occurs (partial patching)
  • opts.strict: boolean throw an exception if error occurs
  • opts.error: object point to a cause patch if error occurs
  • returns nextObject: object

Quick example

1import { applyPatch } from '@typewriter/json-patch';
2
3const prevObject = { baz: 'qux', foo: 'bar' };
4const patches = [
5  { op: 'replace', path: '/baz', value: 'boo' },
6];
7const nextObject = applyPatch(prevObject, patches);
8// → { baz: "boo", foo: "bar" }
9//              |
10//             replaced
11
12console.log(prevObject);
13// → { baz: "qux", foo: "bar" }
14//              |
15//             not changed

How to apply patches

add

1const patches = [
2  { op: "add", path: "/matrix/1/-", value: 9 },
3];

Return a new JSON. It contains shallow-copied elements that have some changes into child elements. And it contains original elements that were not updated.

1assert(prevObject.matrix[0] === nextObject.matrix[0]);
2assert(prevObject.matrix[1] !== nextObject.matrix[1]);
3assert(prevObject.matrix[2] === nextObject.matrix[2]);

remove

1const patches = [
2  { op: "remove", path: "/matrix/1" },
3];

Return a new JSON. It contains shallow-copied elements that have some changes into child elements. And it contains original elements that are not updated any.

1assert(prevObject.matrix[0] === nextObject.matrix[0]);
2assert(prevObject.matrix[1] !== nextObject.martix[1]);
3assert(prevObject.matrix[2] === nextObject.matrix[1]);

replace

1const patches = [
2  { op: "replace", path: "/matrix/1/1", value: 9 },
3];

Return a new JSON. It contains shallow-copied elements that have some changes into child elements. And it contains original elements that are not updated any.

1assert(prevObject.matrix[0] === nextObject.matrix[0]);
2assert(prevObject.matrix[1] !== nextObject.matrix[1]);
3assert(prevObject.matrix[2] === nextObject.matrix[2]);

replace (no changes)

1const patches = [
2  { op: "replace", path: "/matrix/1/1", value: 4 },
3];

Return the original JSON. Because all elements are not changed.

prevObject.matrix[1][1] is already 4. So, this patch is need not to update any.

1assert(prevObject === nextObject);

move

1const patches = [
2  { op: "move", from: "/matrix/1", path: "/matrix/2" },
3];

Return a new JSON. [op:move] works as [op:get(from)] -> [op:remove(from)] -> [op:add(path)].

1assert(prevObject.matrix[0] === nextObject.matrix[0]);
2assert(prevObject.matrix[1] === nextObject.martix[2]);
3assert(prevObject.matrix[2] === nextObject.matrix[1]);

copy

1const patches = [
2  { op: "copy", from: "/matrix/1", path: "/matrix/1" },
3];

Return a new JSON. [op:copy] works as [op:get(from)] -> [op:add(path)].

1assert(prevObject.matrix[0] === nextObject.matrix[0]);
2assert(prevObject.matrix[1] === nextObject.martix[1]);
3assert(prevObject.matrix[1] === nextObject.martix[2]);
4assert(prevObject.matrix[2] === nextObject.matrix[3]);

test failed

1const patch = [
2  { op: "add" , path: "/matrix/1/-", value: 9 },
3  { op: "test", path: "/matrix/1/1", value: 0 },
4];

Return the original JSON. Because a test op is failed. All patches are rejected.

prevObject.matrix[1][1] is not 0 but 4. So, this test is failed.

1assert(prevObject === nextObject);

invalid patch

1const json = [
2  { op: "replace", path: "/matrix/1/100", value: 9 },
3];

Return the original JSON. Because all patches are rejected when error occurs.

prevObject.matrix[1][100] is not defined. So, this patch is invalid.

1assert(prevObject === nextObject);

Syncable Object Store

json-patch provides a utility that will help sync an object field-by-field using the Last-Writer-Wins (LWW) algorithm. This sync method is not as robust as operational transformation, but it only stores a little data in addition to the object and is much simpler. It does not handle adding/removing array items, though entire arrays can be set. It should work great for documents that don't need merging text like Figma describes in https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ and for objects like user preferences.

It works by using metadata to track the current revision of the object, any outstanding changes needing to be sent to the server from the client, and the revisions of each added value on the server so that one may get all changes since the last revision was synced. The metadata will be minuscule on the client, and small-ish on the server. The metadata must be stored with the rest of the object to work. This is a tool to help with the harder part of LWW syncing.

Syncable will auto-create objects in paths that need them. This helps with preventing data from being overwritten during merging that shouldn't be.

It should work with offline, though clients will "win" when they come back online, even after days/weeks being offline. If offline is not desired, send the complete data from the server down when first connecting and then receive changes. If offline is desired but not allowed to "win" when coming online with changes that occurred while offline, you may use changesSince(rev) on the server and receive(patch, serverRev, true /* overwrite local changes */) to ensure local changes while offline do not win over changes made online on the server.

Use whitelist and blacklist options to prevent property changes from being set by the client, only set by the server. This allows one-way syncable objects such as global configs, plans, billing information, etc. that can be set by trusted sources using receive(patch, null, true /* ignoreLists */) on the server.

Example usage on the client:

1import { syncable } from '@typewriter/json-patch';
2
3// Create a new syncable object
4const newObject = syncable({ baz: 'qux', foo: 'bar' });
5
6// Send the initial object to the server
7newObject.send(async patch => {
8  // A function you define using fetch, websockets, etc
9  return await sendJSONPatchChangesToServer(patch);
10});
11
12// Or load a syncable object from storage (or from the server)
13const { data, metadata } = JSON.parse(localStorage.getItem('my-object-key'));
14const object = syncable(data, metadata);
15
16// Automatically send changes when changes happen.
17// This will be called immediately if there are outstanding changes needing to be sent.
18object.subscribe((data, meta, hasUnsentChanges) => {
19  if (hasUnsentChanges) {
20    object.send(async patch => {
21      // A function you define using fetch, websockets, etc. Be sure to use await/promises to know when it is complete
22      // or errored. Place the try/catch around send, not inside
23      await sendJSONPatchChangesToServer(patch);
24    });
25  }
26});
27
28// Get changes since last synced after sending any outstanding changes
29const response = await getJSONPatchChangesFromServer(object.getRev());
30if (response.patch && response.rev) {
31  object.receive(response.patch, response.rev);
32}
33
34// When receiving a change from the server, call receive
35// (`onReceiveChanges` is a method created by you, could use websockets or polling, etc)
36onReceiveChanges((patch, rev) => {
37  object.receive(patch, rev);
38});
39
40// persist to storage for offline use if desired. Will persist unsynced changes made offline.
41object.subscribe((data, metadata) => {
42  localStorage.setItem('my-object-key', JSON.stringify({
43    data, metadata,
44  }));
45});
46
47
48// Auto-create empty objects
49object.change(new JSONPatch().add(`/docs/${docId}/prefs/color`, 'blue'))

On the server:

1import { syncable } from '@typewriter/json-patch';
2
3// Create a new syncable object
4const newObject = syncable({ baz: 'qux', foo: 'bar' }, undefined, { server: true });
5
6// Or load syncable object from storage or from the server
7const { data, metadata } = db.loadObject('my-object');
8const object = syncable(data, metadata, { server: true });
9
10// Get changes from a client
11const [ returnPatch, rev, patch ] = object.receive(request.body.patch);
12
13// Automatically send changes to clients when changes happen
14object.onPatch((patch, rev) => {
15  clients.forEach(client => {
16    client.send({ patch, rev });
17  });
18});
19
20// Auto merge received changes from the client
21onReceiveChanges((clientSocket, patch) => {
22  // Notice this is different than the client. No rev is provided. The server sets the next rev
23  const [ returnPatch, rev, broadcastPatch ] = object.receive(patch);
24  storeObject();
25  sendToClient(clientSocket, [ returnPatch, rev ]);
26  sendToClientsExcept(clientSocket, [ broadcastPatch, rev ]);
27});
28
29// persist to storage
30function storeObject() {
31  db.put('my-object-key', {
32    data: object.get(),
33    metadata: object.getMeta(),
34  });
35}

License

MIT