Flow Auto Backup

Post Reply
User avatar
gabrielp
Advanced member
Posts: 645
Joined: Fri Aug 08, 2014 4:31 pm
Location: Boston
Contact:

Flow Auto Backup

Post by gabrielp »

Introduction
Continuing with my experiments relating to injecting Switch backing files, I now have been able to automatically backup flows. I have created a flow group which automatically backs up every flow on a Switch system, creating .sflow files which can be easily restored. By default, the flow is triggered every hour and archives the resulted flows to a network location, but since I used the Portal Callback pattern, you can change that without compromising the core functions.

Usage
Image

Image

You do not need to modify any of the 'FAB Functions' flows. The flow 'Auto Backup Program' is yours to mess around with.

You control the frequency of backups by means of the Dummy Job Clock which by default is set to 1 hour (3600 seconds).

You then are required to use the Flow Auto Backup (FAB) dataset which consists of a few Private Data keys:
  • FAB Portal Callback - The Portal channel where the packed flows will be delivered once the core functions are complete. You do not have to change this.
  • FAB Flows Path - The absolute path to your Switch Server/flows directory
  • FAB Manifest Path - The absolute path to your generic manifest.xml file. See below for more information.
Out of the Callback, .sflows will start flowing in. Each of these .sflows is ready for use and contains the following new FAB dataset fields:
  • FAB Flow Name - The name of the flow
  • FAB Flow Version - The current version of the flow
  • FAB Flow ID - The numerical ID of the flow
If any of your flows are password protected, these values will say 'Encrypted'.

By default, the 'Auto Backup Program' flow archives the flows into folders for each flow name. The version number is part of the .sflow name, so it will overwrite the same version, but if it's a new version, it will save a new copy in that folder.

Here is the result (Yes, this flow backs up itself. How meta ;) ):
Image

And encrypted flows all go together in the same folder:
Image

Manifest Setup
To reverse-engineer the .sflow filetype and reproduce its behavior within Switch, we need to introduce a simple XML file called a manifest. Luckily, Switch doesn't require there to be very much information in it, so we can simply use a static file. The best way to handle this is to export any flow, rename the .sflow to .zip, unzip it, and copy the manifest to a static location and use that absolute URL as the FAB Manifest Path.

For reference, here are the contents of my manifest.xml:

Code: Select all

<Manifest>
    <ProductInfo>Switch Version 13</ProductInfo>
    <ExportFormatVersion>1.0</ExportFormatVersion>
    <FlowFile>flow.xml</FlowFile>
    <SwitchFlavour>Switch</SwitchFlavour>
    <SwitchReleaseVersionNumber>13</SwitchReleaseVersionNumber>
    <SwitchReleaseType></SwitchReleaseType>
    <SwitchReleaseTypeNumber>0</SwitchReleaseTypeNumber>
    <SwitchUpdateVersionNumber>100</SwitchUpdateVersionNumber>
    <OperatingSystem>Mac OS X</OperatingSystem>
</Manifest>
Detailed Explanation
So now that you know how to use it, here is how it works. Everything is triggered from the Auto Backup Program flow. This flow is invoking FAB Get Flow Function via the Portal Callback pattern. The advantage of this is that you don't have to modify any of the core FAB flows which means if I ever update the FAB Functions, you can simply swap them out and everything will keep working. Said another way, the core flows are acting as a "black box": you don't know and don't need to know what's going on inside because it's abstracted away from you.

Within the FAB Get Flow Function, the triggering job is immediately asserted to ensure the required FAB dataset is set. If so, it then uses those values to inject a copy of your flows folder (being sure not to delete it after injection). The flows folder is split into individual files, where it then injects a copy of the manifest and the flow XML itself is consumed as a dataset. The flow and manifest are assembled into one, and then it writes the remaining fields of the FAB dataset before handing the job off to FAB Pack Flow Function.

Within FAB Pack Flow Function, the job goes through another assertion to verify the callback is set. The job is ungrouped, then renamed to match the .sflow file structure (flow.xml, manifest.xml), and then merged again. The job folder is now archived, but it can't sit within a top-level folder as the Archive configurator would do, so I built a script to archive the files at the top level called Flat Archive. A .zip comes out, which has its suffix replaced with .sflow and then sent to the callback.

Back in the Auto Backup Program flow, the .sflows stream out and the user can do what they want with them. In the default case, they get backed up onto the network separated by folders for each flow name.

What was used
Private data script: https://github.com/open-automation/Swit ... rivateData
Dummy job clock script: https://github.com/open-automation/DummyJobClock
Portals: https://github.com/open-automation/SwitchPortals
Variable Assert: https://github.com/open-automation/SwitchVariableAssert
Flat Archive: https://github.com/open-automation/switch-flat-archive

Download
All Flows: https://drive.google.com/file/d/0B9ciRz ... sp=sharing
FAB Get Flow Function Documentation: https://www.googledrive.com/host/0B9ciR ... mZRdExyekE
FAB Pack Function Documentation: https://www.googledrive.com/host/0B9ciR ... 3o5S0ZLUGc
Auto Backup Program Documentation: https://www.googledrive.com/host/0B9ciR ... jFwQ2YzWnc

Update 9/16/16
This has been updated. Download and new documentation can be found at this repo:
https://github.com/open-automation/swit ... uto-backup
Last edited by gabrielp on Fri Sep 16, 2016 5:23 pm, edited 2 times in total.
Free Switch scripts: open-automation @ GitHub
Free Switch apps: open-automation @ Enfocus appstore

Want to hire me? I'm looking for my next gig. Contact me on LinkedIn or via email.
lombert
Member
Posts: 167
Joined: Fri Feb 04, 2011 2:31 pm
Location: Sweden

Re: Flow Auto Backup

Post by lombert »

I'll be the first to say wow, and thanks to you to share it. I'll defently try it..

I just download the portal script today, and that is also great idéa..
Between jobs!
User avatar
gabrielp
Advanced member
Posts: 645
Joined: Fri Aug 08, 2014 4:31 pm
Location: Boston
Contact:

Re: Flow Auto Backup

Post by gabrielp »

Updated the flow group to utilize Portals v2 (with the Program scope).
Free Switch scripts: open-automation @ GitHub
Free Switch apps: open-automation @ Enfocus appstore

Want to hire me? I'm looking for my next gig. Contact me on LinkedIn or via email.
User avatar
gabrielp
Advanced member
Posts: 645
Joined: Fri Aug 08, 2014 4:31 pm
Location: Boston
Contact:

Re: Flow Auto Backup

Post by gabrielp »

I highly recommend everyone do this. It's saved me twice already.
Free Switch scripts: open-automation @ GitHub
Free Switch apps: open-automation @ Enfocus appstore

Want to hire me? I'm looking for my next gig. Contact me on LinkedIn or via email.
colins
Newbie
Posts: 5
Joined: Tue Jul 05, 2016 8:52 pm

Re: Flow Auto Backup

Post by colins »

We updated the flow to utilize Inject lite, Portals v3, Assert Not Blank, Private Data Write 25, and Flow Group Discovery scripts as well as added two new features. The first is a better way to structure the backup flows based on the group hierarchy. The second is a failure callback feature that passes all failure private data as well as the failed files out of the FAB functions.

We have setup a repository on Github for this as well: https://github.com/open-automation/swit ... uto-backup
Dave23
Member
Posts: 41
Joined: Thu Oct 12, 2017 4:42 pm

Re: Flow Auto Backup

Post by Dave23 »

Hi,

I've been trying to get this Auto Backup Flow to work however I have hit a bit of a snag. I am getting a script error on the Get Flow Function portal in the Auto Backup Program flow. Have you seen this error before?

AutoBackupProgramError.PNG
AutoBackupProgramError.PNG (31.99 KiB) Viewed 15815 times

Thanks

Dave
User avatar
gabrielp
Advanced member
Posts: 645
Joined: Fri Aug 08, 2014 4:31 pm
Location: Boston
Contact:

Re: Flow Auto Backup

Post by gabrielp »

Hey Dave,

Try remaking the portals with the latest version of the scripts or the Portals app from the app store. It may be a bit tedious but if you bring in new configurators and configure them manually you can be sure that nothing is corrupt. I bet one problem is that the versions of the Portals script in the example flows are very old.

You can grab the latest scripts from here https://github.com/open-automation/switch-portals and the app version from the appstore here https://www.enfocus.com/en/appstore/product/portals

Let me know if that works out.
Free Switch scripts: open-automation @ GitHub
Free Switch apps: open-automation @ Enfocus appstore

Want to hire me? I'm looking for my next gig. Contact me on LinkedIn or via email.
EDO
Newbie
Posts: 1
Joined: Wed Aug 14, 2019 11:03 am

Re: Flow Auto Backup

Post by EDO »

Thank you for this Flow Auto Backup, I added it to my Flows.

Also I expanded it a bit. Thought it could be useful to backup the ActionLists and Variable Sets as well.

Just route the Trigger to another flow. Use the tool to pick up jobs for example C:/Users/Druckserver/AppData/Roaming/Enfocus Prefs Folder/Variable Sets give it a hierarchy and put it to your Archiv.

Image
Attachments
Flow.png
Flow.png (12.68 KiB) Viewed 15306 times
Post Reply