Hydra
Docs

index

UP  
DN   explorer,  ssl,  outline,  portmgr,  about
 
Fig. 1 Explorer
 

Overview

Hydra is a networked file sync program. There is a gui viewer, shown in Fig. 1, for visualizing the underlying xml commands. Hydra is used in automating such tasks as simple code updates or archiving live data.

documentation and download

       explorer  managing sync joblist
           ssl   adding ssl to hydra
        outline  built in text editor
        portmgr  hydra's port options
          legal  license

       download

hydra design

By conscious choice, Hydra supports 1-way sync and not 2-way. 1-way only exhibits robustness in the face of adverse network events.

A 1-way sync operates at any given time on an activation scale running from quiescent to flood. In the quiescent periods, Hydra transfers a fingerprint of an entire directory tree in a single tcp request. In a flood region, hydra avoids taking 100% of cpu by taking short naps in 100's of msec time periods after comparable cpu usage by itself. During the file transfer period, we see best case transfer of very small files rate limiting at 1/second. Restated, no more than 3600 files per hour can be synced.

There is a problem with FAT file systems: they only store a file modify time to the nearest +/-2 seconds. Hydra requires precision to the nearest second in making comparisons over large spans. In a quiescent region, two machines will exchange a checksum of entire directory tree including file times in order to reduce the http/https transaction to a single tcp/ip exchange. This does not work if each file time matches only within nearest 2 seconds. What might work is substitute file sizes for file dates in comparisons. See the file_diff_method in the hydra_jobs.py file shared by any network to select. The default setting is time-based and by default FAT is not supported.