gitsync-nncp over Filespooler

You can use gitsync-nncp (a tool for Asynchronous syncing of git repositories) atop Filespooler. This page shows how. Please consult the links in this paragraph for background on gitsync-nncp and Filespooler.

Basics

How you will set up Filespooler depends, of course, on what kind of transport you are using. These examples assume a directory synchronization as with, for instance, Using Filespooler over Syncthing. Of course, other transports are available.

There are two things to consider with gitsync-nncp: sending and receiving. Let’s discuss sending first.

Sending

gitsync-nncp commands like push_full take a command (with optional parameters) to use to send data. After that command (and parameters), gitsync-nncp adds its own additional parameter: a branch name. This is a perfect fit for Filespooler. Here’s an example script you might use. Let’s call it gitsync-send-fspl:

#!/usr/bin/env bash

set -euo pipefail

fspl prepare -s ~/statefile -i - -- "$@" | \
    fspl queue-write -q ~/queuedir

Now, call gitsync-nncp like this:

gitsync-nncp push ~/bin/gitsync-send-fspl

Easy! Of course, your send-fspl script could just as easily run over some other transport, add compression or encryption, etc.

Receiving

Now, the counterpart: receiving. We can use the loosely-ordered mode here to allow us to receive from many machines in a Many-To-One with Filespooler or many-to-many configuration.

fspl queue-process -q ~/queuedir --allow-job-params --order-by=Timestamp \
  --on-error=Leave gitsync-nncp -- receive ~/repopath

This will cause the parameter given to prepare – the branch name – to be appended after gitsync-nncp receive ~/repopath.

That’s it!

gitsync-nncp is a tool for using Asynchronous Communication tools such as NNCP or Filespooler, or even (with some more work) Syncthing to synchronize git repositories.

Since Filespooler is an ordered queue processor by default, it normally insists on a tight mapping between the sequence numbers in job files and execution order in a queue.

In some cases, you may want to use Filespooler to send the data from one machine to many others. An example of this could be using gitsync-nncp over Filespooler where you would like to propagate the changes to many computers.

Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication.