Antfarm - a simple file automation framework written in Node

Post Reply
User avatar
gabrielp
Advanced member
Posts: 645
Joined: Fri Aug 08, 2014 4:31 pm
Location: Boston
Contact:

Antfarm - a simple file automation framework written in Node

Post by gabrielp »

Hi guys,

For some time, I've been working on a little personal project to make file automation a little simpler. To summarize, you can use this to create hotfolders, download and move files to FTP or Amazon S3, and even trigger advanced workflows with HTTP requests. It's designed to be used for programmers but is very simple to get running and incorporate your own code.

As Switch users, especially those who script, Antfarm could be a useful tool to supplement Switch in your automation toolkit.

Antfarm and Switch
Even though Antfarm can natively do a few things that Switch can, it’s very different and aims to achieve a different goal:

- Antfarm has no admin UI. It’s designed for a developer who is comfortable with scripting/programming concepts. This also greatly simplifies Antfarm compared to the lifecycle of a job in Switch. For this reason, running application code on jobs via Antfarm is usually much faster than a similar workflow in Switch, since Switch has so much overhead (backing folders at every step, remote job ticket, etc…).

- Since Antfarm workflows are just code, they can be version controlled, tested, and continuously integrated. This makes collaborating and iterating workflows much easier than Switch currently. Unlike the current limitations of Switch Scripter, you can use whatever version of ECMAScript (or TypeScript!) you want, as well as your favorite IDE.

- Antfarm can more easily be run in the cloud and split amongst different servers. You could have a single monolithic Antfarm server or a separate one for each workflow. This is interesting when you consider docker workflows where you could scale workflow workers based on their load.

- Along with the normal file events that trigger automation (job arrived in hotfolder), you can also trigger logic in Antfarm with an HTTP request. Webhooks allow you to trigger automation logic from other APIs or events without having to generate dummy files as you would with Switch. A simple use-case for Antfarm is to create a workflow that responds to a webhook which generates one such dummy file to trigger further automation in a Switch workflow.

- Switch gives you a ton of helpful out-of-the-box functionality with its suite of configurators -- while Antfarm provides a bare minimum right now. In exchange, you get the entire NPM ecosystem. Since Antfarm workflows are just Node JavaScript code, you can easily pull in any Node library for additional functionality.

Example
To give you a feel of what it looks like, here is a workflow in Antfarm that sorts PDFs and others via a hotfolder into other folders:

Code: Select all

const   Antfarm = require('antfarm'),
        af = new Antfarm();

let hotfolder_a = af.createFolderNest("/var/hotfolders/a");
let pdf_folder = af.createFolderNest("/var/out/pdf");
let other_folder = af.createFolderNest("/var/out/others");

let tunnel = af.createTunnel("Simple pdf sorting workflow");

tunnel.watch(hotfolder_a);

tunnel.run((job, nest) => {
    if(job.extension == "pdf"){
        job.move(pdf_folder);
    } else {
        job.move(other_folder);
    }
});
This is comparible to the following Switch workflow:
Image

Links
Code: https://github.com/dominickp/antfarm
Wiki: https://github.com/dominickp/antfarm/wiki
API Reference: https://dominickp.github.io/antfarm
Free Switch scripts: open-automation @ GitHub
Free Switch apps: open-automation @ Enfocus appstore

Want to hire me? I'm looking for my next gig. Contact me on LinkedIn or via email.
Post Reply