Performance Switch flows vs. PitStop Server hotfolder

Post Reply
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

Hello together,

have you had the same experience with the performance of switch flows compared to clean PitStop Server workflows?

I have found that a switch-based PDF processing flow is slower than if the same functionality is only implemented with a PitStop Server hotfolder.

Example:
PitStop Server:

IN folder, OUT folder, ERROR folder
1 simple action list
Error-free documents are placed in folder OUT, incorrect documents in folder ERROR.
Processing of 500 PDFs: less than 1 minute

Switch-Flow (identical function as above):
IN folder, OUT folder, ERROR folder
PitStop server configurator, the same action list (external) selected.
Documents without errors are placed in the OUT folder, documents with errors in the ERROR folder.
Processing of the same 500 PDFs: approx. 7.5 min.

I know that there is still communication between Switch and PitStop Server, but that the processing takes 7 times as long is not so good.

I have a lot of simple PitStop Server workflows that I wanted to implement in Switch, but I don't know if the system will not perform well enough.
Have you had similar experiences? Thanks for feedback.

Many greetings,
Peter
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
Malcolm Mackenzie
Member
Posts: 121
Joined: Wed Mar 22, 2017 5:05 pm
Location: London, UK
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Malcolm Mackenzie »

Hi
Pitstop Server has no concurrent limit, only limited by the hardware.
Pitstop inside Switch is limited by your license.
What do you have, Core, Performance or unlimited?
Malcolm
Malcolm Mackenzie
Member
Posts: 121
Joined: Wed Mar 22, 2017 5:05 pm
Location: London, UK
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Malcolm Mackenzie »

Oh and what processor/cores for the hardware?
jan_suhr
Advanced member
Posts: 586
Joined: Fri Nov 04, 2011 1:12 pm
Location: Nyköping, Sweden

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by jan_suhr »

You have to turn on the flows property "Allow advanced performance tuning" for Switch to use all available processor cores for that flow. The default setting is "NO" which means that it only run one job at a time instead of parallell processes up to the number of processors you have licenses for. The standard Core license only uses four processor cores.
Jan Suhr
Color Consult AB
Sweden
=============
Check out my apps
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

We have a virtual machine (Win Server 2019) with Quad core and 16 GB RAM.

The Switch modules are Switch Core Engine, Configurator, Metadata and Switch Client Module.

I have changed the flows property "Allow advanced performance tuning" to "Yes" but with no effect.
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
Padawan
Advanced member
Posts: 358
Joined: Mon Jun 12, 2017 8:48 pm
Location: Belgium
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Padawan »

In the processing category of the Switch preferences, which values do you have for Concurrent processing tasks and default number of slots for concurrent elements?
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

We have the setting "0". That was the setting after installation. All available slots should be used.
Although in the manual the value "1" is given as "Default".
(see https://www.enfocus.com/manuals/UserGui ... ssing.html - Property: "Default number of slots for concurrent elements")

In our flow I set the flow default to "Allow advanced performance tuning = YES" as jan_suhr mentioned in his post.
In the PitStop Server Configurator element the "Number of Slots" is set to "Default". So the user preference 0 = "all available slots" (see above) should be used.
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

Sorry, here is the value for Concurrent processing (4). I post a screenshot of the complete settings.
Sorry for being in german. I hope you will understand. :?

I wonder why lots of values differ from the defaults in the manual. For example "Release acquired slots after (minutes). Default should be 20 minutes, in our settings value is 5 minutes. I have never changed this setting.

Switch_Preferences.PNG
Switch_Preferences.PNG (21.13 KiB) Viewed 14356 times
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
Padawan
Advanced member
Posts: 358
Joined: Mon Jun 12, 2017 8:48 pm
Location: Belgium
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Padawan »

The important settings for performance are:
Concurrent processing tasks
If you don't have a performance module, then you can set this maximally to 4. This is the case for you, so this is correct. This setting sets how many (heavy) tasks over all of Switch will run simultaneously.
Default number of slots for concurrent elements.
Using this setting you can set a limit to how many tasks each specific instance of a flow element in a flow has. You have this set to 0, which means there is no limit for for specific instances of flow elements, only the global limit set by the "Concurrent processing tasks" setting.

When you change the setting Jan referred to via Advanced Performance tuning on a specific element, then you overwrite the "Default number of slots for concurrent elements" setting for that specific element.

When you do the test in PitStop Server standaline, can you open the progress view (menu progress, show progress)? There you should see the active jobs. How many active jobs do you see at the same time?

PitStop Server standalone has no limit in the amount of concurrent jobs. Switch has a limit depending on your licenses.If you don't have a performance module, then the limit is 4.

If PitStop Server standalone is doing 6 tasks at the same time, then it will naturally be faster then PitStop Server via Switch with a limit of 4.
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

Hi Padawan,

thanks for your answer.

I have done a test in PitStop Server (standalone) and there are approx. 25 (!) concurrent processes.

Transferring it in a simple calculation:
25 processes (PitStop) : 4 processes (Switch) = Faktor 6,25
In my first testcase there are ca. 500 PDFs with a size of 100-150 kB. And when processing with Switch takes more than 7 Min. it is done by PitStop in less than 1 minute (7 : 6,25 = 1,12)

I have done a second test with only one PDF file with about 70 MB. There wasn't such a big difference. Switch needs 40-45 s for the complete processing (copy to hotfolder, pickup the job, doing the action list, putting it into out-folder). The PitStop standalone process needs 30-35 s.
So there will be a bit measurement accuracy and some communication between Switch and PitStop, so that the process times doesn't differ very much.

The result is for me, that there is no mis-configuration and if we want to increase the performance with lots of small files, we will have to change our license to performance or unlimited performance module. Right?
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
Padawan
Advanced member
Posts: 358
Joined: Mon Jun 12, 2017 8:48 pm
Location: Belgium
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Padawan »

Yes, you will certainly have to look into the performance module.

I know you can trial most modules, but I'm not sure if it works for the performance module. Can you try this?
- Open menu Help/Manage Licenses
- Click the performance module
- Click the trial button
- Follow the steps to activate the trial.
You now have a 30 day trial of the performance module.

- In the processing category of the Switch preferences, change the setting for concurrent processes. You should be able to set it to 12.
- Run the test again.

If the above method for a trial doesn't work, then you can also request a trial via your reseller. Note that a regular performance module adds 8 concurrent processes, but there are also licenses to add more licenses. You can get more info on that via your reseller.
Malcolm Mackenzie
Member
Posts: 121
Joined: Wed Mar 22, 2017 5:05 pm
Location: London, UK
Contact:

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by Malcolm Mackenzie »

Couple of points;
Are you on the latest Switch 19 Fall? (Whatever Fall is !)
There was some issue prior to that with Pitstop in switch not releasing the process.
As it is when Switch is doing the PitStop work in the Progress window you should see how many files are being processed.
See if closing the PitStop UI, if it's running, helps.

To try
Use the Pitstop queue you have and the hotfolders for it.
In Switch have two folders no connections.
One folder is linked to PitStop In the other Out.
Drop your files into the In and see if it takes more/less time.

Performance module relies on cores and there's 4.
4 cores is the minimum for Switch and try assigning 8 in the VM.

If you have no joy after this try support to remote on and look at the backend.
You can also contact me offline if you would like me to to do a speedtest comparison for you on a ramped PC.
abailescollins
Advanced member
Posts: 458
Joined: Wed Apr 22, 2015 4:28 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by abailescollins »

So you have probably seen the information about PitStop Server becoming Switch based in the future?
I only mention that because there will be some changes coming in the future regarding the way the two products work in the future, although I can't say too much at the moment.

Currently, If you are looking for simple Switch flows and using PitStop Server for sheer volume, especially with lots of small files. You might consider using Switch with the hotfolder in/out tool, and feeding PitStop Server via the PitStop Server hotfolders.

Not using the PitStop Server configurator in Switch does limit what you can do with variables etc, but if you are looking for raw PDF processing, that might be of help. This means Switch works with 4 processes, PitStop Server standalone does not have a hard limit.

Of course you can also do a combination of hotfolder in/out and configurator if you want to as well.
It just depends what your goal is.
Head of Product Management @ Ultimate
abc@imposition.com
pfischer
Member
Posts: 43
Joined: Tue May 14, 2013 4:57 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by pfischer »

Malcolm Mackenzie wrote: Wed Mar 11, 2020 5:36 pm Couple of points;
Are you on the latest Switch 19 Fall? (Whatever Fall is !)
There was some issue prior to that with Pitstop in switch not releasing the process.
As it is when Switch is doing the PitStop work in the Progress window you should see how many files are being processed.
See if closing the PitStop UI, if it's running, helps.

To try
Use the Pitstop queue you have and the hotfolders for it.
In Switch have two folders no connections.
One folder is linked to PitStop In the other Out.
Drop your files into the In and see if it takes more/less time.

Performance module relies on cores and there's 4.
4 cores is the minimum for Switch and try assigning 8 in the VM.

If you have no joy after this try support to remote on and look at the backend.
You can also contact me offline if you would like me to to do a speedtest comparison for you on a ramped PC.
Hi Malcom,

yes, both applications are on the latest release "Switch 19 Fall" und "PitStop Server 19 Upd. 1".
I habe tried your test szenarios
1. Trigger the PitStop Hotfolders directly via 2 Switch folder (not using only the PitStop Server configurator element). Switch only makes the file handling.
Result: as fast as putting the files directly into the PitStop folders (less than 1 Min. / 25 parallel PitStop processes).
2. Closing PitStop UI.
Result: makes no real difference. It takes approx. 7 Min. to process the 500 files.
3. Number of parallel jobs in PitStop when configured as a Switch flow with PitStop configurator
Result: there are only 2 parallel jobs in the progress window.

@abailescollins:
yes, I've have seen the announcement for PitStop 2020 (with Switch). I am curious to see if a switch license will become superfluous for some customers.
Then I will build the workflows according to the required performance. It would be good if the performance would be the same for a switch-based solution. Not using metadata is a pity and limits the possibilities.

Thanks,
Peter
Media asset and production specialist
Group Communications / System Operations
Phoenix Contact GmbH & Co. KG
www.phoenixcontact.com
abailescollins
Advanced member
Posts: 458
Joined: Wed Apr 22, 2015 4:28 pm

Re: Performance Switch flows vs. PitStop Server hotfolder

Post by abailescollins »

No the Switch license will still be needed in the future.
There are some differences with the Switch Core supplied with PitStop Server. The main one being that it cannot be upgraded with extra Switch modules.
It's important to understand with PitStop Server that we are in a transition to a new generation of PitStop Server.
The reason we are making Switch Core available now is so that customers have 2-3 years to get used to it and transition their workflows.
It would make no sense for us to suddenly make a big change like that overnight.
The next generation PitStop Server will a very different product, but I can't say much more at the moment.
Head of Product Management @ Ultimate
abc@imposition.com
Post Reply