Joseph Naberhaus


Go Delta Sync (WIP)

Written in December 2020
Due to other obligations I have put this project temporarily on hold.

While working on another project I needed to keep a tree of nested objects in sync between a client and server. This tree could be modified multiple times a second, therefore, rather than send the entire tree I wanted to only send the delta of what had changed. Further complicating things, this tree would be constantly revised as I worked on the project. My solution would need to be easy to add more objects to.

This was a perfect use case for code generation. I began working on a tool that I originally called the operations generator. This would give me a mechanism for modifying the contents of a model. The modification would produce a delta object. The delta would then be sent to the clients which would update their local state to match the server. Below is a diagram showing how deltas are applied to a model.

The mechanism for creating the deltas is called an operation. An operation is another model that is written by the code generator. There is a conversational style API to configure the operation with what should be changed. Then the operation is applied to a model which modifies the original model and produces a delta. Below is a diagram showing the full process.

While working on this tool I realized the need for another tool to generate the client-side code to apply the deltas. For this, I'm working on a simple code generation library that can target any arbitrary language. Information about this project can be found here