Ungroup / Assemble

[randomfactor]

I am trying to create a flow that will take in a bunch of jobs (thousands), crunch on them for a long while (hours?), then send out an email when ALL those jobs have finished.



To accomplish this, I'm packing a bunch of PDF files in a zip file, using the Uncompress element to get the files (a bunch of PDF files under a single subfolder), then use the Ungroup job element to treat each PDF as an individual job through the flow. These PDF files are sent through two parallel branches -- one creates a thumbnail, the other creates a low-res preview PDF.



The problem that I'm having is that I want to use the Assemble job element on _ONE_ of the branches just to gather things back up again before emitting an email to notify the submitter that the processing is complete. However, one branch has the original job numbers from the Ungroup operation and the other branch has _DUPLICATE_ job numbers that are not associated with the hierarchy information (subfolder name). All the (duplicated) jobs on the slower branch just hang on the input of the Assemble job element until the orphaned job timeout occurs.



How can I force a specific branch in my flow to get the original job number and not a duplicate job number?



Or, is there a better way to achieve the kind of synchronization (end of all processing for the original zip job) I need to emit a single email when all is complete?



Thanks,

Randall Krebs

[Danny]

The first thing I would try is duplicating the job folder before ungrouping it. That way you will have two ungroup and two assemble job elements. You might want to change the private data key for the ungroup elements so they are different.

There's probably better solutions but this should work.



Danny

[randomfactor]

I think your suggestion would probably work. However, I think I would have to duplicate some time-consuming processing nodes.



For example, I want to create a thumbnail only if the preflight checks pass. I also want to create a preview PDF only if the preflight checks pass. So, to implement your suggestion, I would have to duplicate the preflight check on each path out of the Ungroup Jobs node.



But, thank you for your suggestion and thank you for reading my lengthy problem description.