Assemble files from mutiple inputs

Post Reply
rhd_ole
Member
Posts: 92
Joined: Mon Jan 24, 2022 5:36 pm

Assemble files from mutiple inputs

Post by rhd_ole »

Hello Switch community!

I have an issue I'm struggling with for some reason and I feel like I'm missing something easy to solve this.

Run down:
  • I have a flow that starts from a submit point where 2-6 file could be submitted.
  • One is an excel that contains all the new info about the job.
  • I'm split that and converting it all to privateData ( product_id privateData is common ), and then creates a file that pushes to our MIS to create the job.
  • The MIS pushes out an XML to Switch that is picked up and combined with the 2-6 files that were submitted and assembled to combine all the privateData from the submitted file as well as the privateData fro the XML (job number and product_id specifically)
  • This worked great before they changed the number of files that could be submitted
My assemble was using "product_id' as the Job identifier ( all files have that PD) and then number of files I set as an inline since it was always 2 files ( the one submitted and the XML) now that has changed and I can't think of a way to turn the number of files to a variable that I could use for both to assemble based on number of files and/or set a complete condition.

Here is a simplified version of the flow.
Screenshot 2023-01-24 at 7.50.49 AM.png
Screenshot 2023-01-24 at 7.50.49 AM.png (231.21 KiB) Viewed 3851 times
Thanks,

-O
Color Science & Workflow Automation
freddyp
Advanced member
Posts: 1008
Joined: Thu Feb 09, 2012 3:53 pm

Re: Assemble files from mutiple inputs

Post by freddyp »

If I understand it correctly the number of files to be assembled will always be 1 more than the number of files in the folder that was submitted. In the flow you use "Ungroup job". That adds a few pieces of private data (check the documentation) and one of them is Ungroup.NumFiles. So what you need to define for "Number of files" is a Switch.Calculation of that private data tag + 1.
User avatar
magnussandstrom
Advanced member
Posts: 342
Joined: Thu Jul 30, 2020 6:34 pm
Location: Sweden
Contact:

Re: Assemble files from mutiple inputs

Post by magnussandstrom »

Feature request: It would be great to have 'mark-up tools' in Switch where you can draw boxes an write text (as Ole did in the screenshot) in Switch Designer, for self documentation and structure. I know this has been on the wish list for a long time.
rhd_ole
Member
Posts: 92
Joined: Mon Jan 24, 2022 5:36 pm

Re: Assemble files from mutiple inputs

Post by rhd_ole »

freddyp wrote: Wed Jan 25, 2023 8:49 am If I understand it correctly the number of files to be assembled will always be 1 more than the number of files in the folder that was submitted. In the flow you use "Ungroup job". That adds a few pieces of private data (check the documentation) and one of them is Ungroup.NumFiles. So what you need to define for "Number of files" is a Switch.Calculation of that private data tag + 1.
Thanks for the reply. I actually tried this two different ways, using Ungroup with the Ungroup.NumFiles, and as a Custom Assemble my own private data Nm_of_file = Job.FileCount.
This is what happens to both.

Ungroup Assemble
  • The first try the 2 submitted files would show as 3
  • The XML from the MIS would show 1
  • The submitted files would assemble but individually
  • The XML would fail
  • I realized the XML didn't have an Ungroup so I did a assemble/ungroup and tried again.
Ungroup.NumFile missing in the XML
Ungroup.NumFile missing in the XML
Screenshot 2023-01-25 at 6.28.28 AM.png (54.62 KiB) Viewed 3813 times
  • After adding the Assemble/Ungroup
  • All files show Ungroup.NumFiles
  • Result of Switch Calculation of Ungroup.NumFile +1 - Submitted files show 3, XML show 2
  • Assemble does not complete, or error till timeout.
Ungroup.NumFile shows but assemble will not complete
Ungroup.NumFile shows but assemble will not complete
Screenshot 2023-01-25 at 6.18.40 AM.png (55.62 KiB) Viewed 3813 times
Basically the same thing happens when I use my own private data, I could get two to complete but never all 3 in this case.

Any other suggestions? Thanks for taking the time to respond.
Color Science & Workflow Automation
freddyp
Advanced member
Posts: 1008
Joined: Thu Feb 09, 2012 3:53 pm

Re: Assemble files from mutiple inputs

Post by freddyp »

If you use "Scheme - Custom" in "Assemble job" you have to make sure that all files have the same "Job identifier" and for "Number of files" you can use Ungroup.NumFiles+1. I am sure there is some piece of information that is present for all the files involved that is identical and that can serve as the variable for "Job identifier".
rhd_ole
Member
Posts: 92
Joined: Mon Jan 24, 2022 5:36 pm

Re: Assemble files from multiple inputs

Post by rhd_ole »

freddyp wrote: Thu Jan 26, 2023 7:40 am If you use "Scheme - Custom" in "Assemble job" you have to make sure that all files have the same "Job identifier" and for "Number of files" you can use Ungroup.NumFiles+1. I am sure there is some piece of information that is present for all the files involved that is identical and that can serve as the variable for "Job identifier".
I tried a custom scheme using Ungroup and a version where I made my own private data (num_of_files=Job.FileCount)-

The Job identifier i'm using is 'product_id', that is the common private data on all the files.
The issue is the xml that comes in to Switch from our MIS, will never have the same 'Ungroup.NumFiles' as the submitted ones.

Example:

2 files submitted ( switch calculation Ungroup.NumFiles +1 ) = 3
1 XML File from MIS ( switch calculation Ungroup.NumFiles +1 ) = 2
Custom Assemble Scheme:
Job Identifier = 'product_id'
Number of files = switch calculation Ungroup.NumFiles +1

The result is the XML completes, since 1+1 is true, and leaves the submitted files, 2+1 is false and fails since there are only 2 files there.

I also tried the same thing but without an ungroup from the XML which makes the XML not have any private data for Ungroup.NumFile and the same thing happens except. I then tried to come up with a complete condition but failed there as well.

I've attached a sample flow in case this would help or anyone wanted to take a look( one using ungroup one using custom assemble) and 3 sample files (2 excel for the submitted files, and one XML for the file that comes out of the MIS. I set product_id private data as common from both input


Assemble files from multiple inputs.zip
Flow Plus Sample Files
(78.49 KiB) Downloaded 82 times

Flow Screenshot
Flow Screenshot
Screenshot 2023-01-26 at 6.27.43 AM copy.jpg (169.69 KiB) Viewed 3774 times
Color Science & Workflow Automation
freddyp
Advanced member
Posts: 1008
Joined: Thu Feb 09, 2012 3:53 pm

Re: Assemble files from mutiple inputs

Post by freddyp »

Right, so the XML from the MIS comes in separately and does not have the correct private data and the assembly fails because of that. I am starting to understand the problem at hand a bit better and this is what I think will solve it.

The files come in and get assembled into a folder. That is 1 job. The XML comes in from the MIS. That is also 1 job (do not assemble it!). If both jobs have the same id in private data you can assemble them with a custom scheme with the id as the job identifier and the number of jobs fixed at 2. The job name should probably not be automatic, but the value of the id. The result is a folder with a folder in it and an XML. Then use "Rename job" with "Files inside job folder - Flatten hierarchy (discarding folders)". The result will be a folder with all the files and the XML at the root level.
rhd_ole
Member
Posts: 92
Joined: Mon Jan 24, 2022 5:36 pm

Re: Assemble files from mutiple inputs

Post by rhd_ole »

freddyp wrote: Thu Jan 26, 2023 4:29 pm Right, so the XML from the MIS comes in separately and does not have the correct private data and the assembly fails because of that. I am starting to understand the problem at hand a bit better and this is what I think will solve it.

The files come in and get assembled into a folder. That is 1 job. The XML comes in from the MIS. That is also 1 job (do not assemble it!). If both jobs have the same id in private data you can assemble them with a custom scheme with the id as the job identifier and the number of jobs fixed at 2. The job name should probably not be automatic, but the value of the id. The result is a folder with a folder in it and an XML. Then use "Rename job" with "Files inside job folder - Flatten hierarchy (discarding folders)". The result will be a folder with all the files and the XML at the root level.
This works. So annoying, I did all at first before posting but I was ungrouping the submitted files and I shouldn't' have. Thank you so much.. Easy solutions are often overlooked it seems.
Color Science & Workflow Automation
Post Reply