Wormhole

While developing tools for StratOS, I wrote a nifty little program that I call “Wormhole,” which uses the Linux kernel’s inotify functionality to respond to filesystem events. More specifically, it watches a single directory and performs specified actions on any files that are moved into or written to the directory. Once the files are processed, the results are stored in another directory. The syntax is like this:

$ wormhole action entrance_directory exit_directory
where the action is either a program name or one of the built-in commands: --zip or --unzip. The program that is used as the action, must take two input arguments: an input file and an output directory. Wormhole expects the action to remove the input file from the entrance_directory as its final step. The simplest (very silly) example is:
$ wormhole mv /directory1/  /directory2/
In this example, the action is the move command, mv. Whenever a file is added to /directory1, it will automatically be moved to /directory2.

A more realistic use case would involve a script that performs some sort of useful operations on the files that enter the entrance_directory. Here’s a slightly less silly example action:

#! /bin/bash

# pdf_append.sh - A command for adding new pages to a PDF file
# usage: pdf_append.sh input_image output_directory
# where input_image is an image file that can be processed with
# ImageMagick's convert command and output_directory contains
# exactly one PDF document.
# Note that this is a very fragile script for demonstration
# purposes only.

input_image=$1
output_dir=$2

# convert image to PDF 
# (this assumes the input isn't already a PDF)

input_pdf="$input_image.pdf"
convert $input_image $input_pdf

# get the name of the PDF being appended to.
# assumes there is exactly one PDF file in output_dir 

output_pdf="$(ls $output_dir/*.pdf)"

temp_pdf="${output_pdf/.pdf/-temp.pdf}"

# append the input_pdf to the end of the output_pdf

mv $output_pdf $temp_pdf

pdfunite $temp_pdf $input_pdf $output_pdf
# these tools would also work: 
# pdftk $temp_pdf $input_pdf cat output $output_pdf 
# pdfjoin $temp_pdf $input_pdf $output_pdf

# delete intermediate files:

rm $input_image $input_pdf $temp_pdf

Suppose I create directories in my current working direcotry, called in/ and out/. Inside of the out/ directory, I place a PDF document. I then execute wormhole, like this:
$ wormhole ./pdf_append.sh in out 
Now, if I save or move images into the in/ directory, they will automatically be appended to the PDF document in out/ as new pages.

Wormhole was initially developed to provide a means of automatically compressing and decompressing files before they are sent over a non-compressed network stream (specifically, NFS), so it has two built-in actions, --zip and --unzip. These actions currently use zlib to perform the compression, but they will eventually use the LZO compression library, due to its compression speed. The input and output directories are in RAM-backed, tmpfs file systems in order to minimize latency. The sender places files into one “mouth” of the wormhole and files are compressed and sent to the destination computer. The receiver watches the other mouth of the worm hole and decompresses files as they arrive.

The current version of Wormhole isn’t written to be a proper daemon, but it can be cheaply “daemonized” using nohup and output redirection. For example:

$ nohup wormhole --zip /path/to/in /path/to/out 1> /dev/null 2>/dev/null &
The source for the version of Wormhole described above can be found here.

318 Responses to “Wormhole”

  1. Lina Gutierrez Says:

    Boleh ga setuju, tapi ini keputusan yang terbaik

  2. Sam Zion Drake Pham Says:

    Share your favorite blog post in the comments below!

  3. ??? ??? Says:

    Consider the bonus offers available on different platforms and choose one that provides generous bonuses and favorable terms and conditions

  4. seosearchoptimizationpro Says:

    Iappreciatetheeffortputintoresearchingthistopic.

  5. Adelyn Sanford Says:

    This is my first time pay a quick visit at here and i am really happy to read everthing at one place

  6. Saniya Giles Says:

    I truly appreciate your technique of writing a blog. I added it to my bookmark site list and will

  7. Aiden Donovan Says:

    Your blog post was really enjoyable to read, and I appreciate the effort you put into creating such great content. Keep up the great work!

  8. Xavier Abdullah Wade Says:

    Every time I read one of your posts, I come away with something new and interesting to think about. Thanks for consistently putting out such great content!

  9. Yaritza Ochoa Says:

    Nice post. I learn something totally new and challenging on websites

  10. Kemal Çiftçi Says:

    AKP’nin T.B.M.M’ye sunmay? planlad??? 9. Yarg? Paketi’nde yer alan ve “etki ajanl???” düzenlemesi, sundu?u, Türk Silahl? Kuvvetleri (TSK) Personel Kanunu ile Baz? Kanunlarda De?i?iklik Yap?lmas?na Dair Kanun ile emekli komutanlar?n izin almaks?z?n aç?klama yapmalar?n?n suç say?lmas? ve Seferberlik Yönetmeli?i’nde yer alan “buhran”, “gerginlik”, “ayaklanma” veya “kalk??ma” gibi ifadelerdeki mu?lakl?klar tüm bu yasal düzenlemelerin amac?n?n toplum üzerinde yarg? bask?s? olu?turarak daha fazla susturmak ve korkutarak sindirmek oldu?una i?aret etmektedir.

  11. Tipobet Giri? Says:

    I really love to read such an excellent article. Helpful article. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  12. Tipobet Giri? Says:

    Thank you for great article. Hello Administ .Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  13. Tipobet Giri? Says:

    Thank you great post. Hello Administ .Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  14. Ariel K. Zavala Says:

    Thank you for the awesome content, it’s always appreciated

  15. Tipobet Giri? Says:

    Nice article inspiring thanks. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  16. vbet Giri? Says:

    Thank you great post. Hello Administ .Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  17. Tipobet Says:

    Thank you for great content. Hello Administ. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut

  18. Makenna Maryam Barrera Says:

    This is such an important and often overlooked topic Thank you for bringing attention to it and offering valuable advice

Leave a Reply