Hello,
Recently after upgrading from Switch 9 to Switch 11, some of my scripts encounter a random error:
Communication was aborted: Failed to parse the received command data: expected command size xxxx, expected data size yyyy, received data size zzzz.
After some monitoring, I noticed that the error usually occurred after the processing of the script, at the moment Switch is dealing with moving jobs to output.
Prior Switch 10 / 11, it never happened.
As well, another problem related to upgrading from Switch 9 to Switch 11, even by exporting and re-importing flows, is that some of the HierarchySubmit elements stopped recognizing the path. I had to re-enter the path manually. It is also noted that some of those elements start to behave erroneously such as keep locking the file even after it failed to take the file in, in some cases lead to adding many unique identifiers to a file.
Please help!
Error: Communication was aborted...
-
marbledaemon
- Newbie
- Posts: 7
- Joined: Mon Sep 24, 2012 12:28 pm
Error: Communication was aborted...
Hi,
we've seen this error behavior with callas pdfToolbox configurators, too.
But I never saw this with one of our scripts.
How did you define the base path for the Submit hierarchy flow element. Did you use UNC paths? If yes, then this problem results from using backslashes instead of forward slashes inside the path definition.
Regards,
Robert
we've seen this error behavior with callas pdfToolbox configurators, too.
But I never saw this with one of our scripts.
How did you define the base path for the Submit hierarchy flow element. Did you use UNC paths? If yes, then this problem results from using backslashes instead of forward slashes inside the path definition.
Regards,
Robert
-
marbledaemon
- Newbie
- Posts: 7
- Joined: Mon Sep 24, 2012 12:28 pm
Error: Communication was aborted...
Hello Robert,
Thank you for replying. Most of the paths in our flows are network path. We did not have problems with slashes before because Switch 9 automatically converted them to /. It surprised us upon upgrading to Switch 11 and what we worry is whether we have to manually fix all the paths in our flows (more than 150 flows) just because up Switch 11. If it is the case, it would take a very high effort in both of time and risk management.
To be specific on the Communication aborted error, one of our scripts basically receives an xml (might weights up to 2 MB), parses it using Document object to get a list of files to be injected to the flow. Based on the list (in average 500 files), the script then creates a series of dummy jobs (0 byte-files containing the full file path in their private data) and send to the next element which is an Inject element.
During the monitoring, the error occurred out of the scope of the script process, during the handling of output jobs.
Marble,
Thank you for replying. Most of the paths in our flows are network path. We did not have problems with slashes before because Switch 9 automatically converted them to /. It surprised us upon upgrading to Switch 11 and what we worry is whether we have to manually fix all the paths in our flows (more than 150 flows) just because up Switch 11. If it is the case, it would take a very high effort in both of time and risk management.
To be specific on the Communication aborted error, one of our scripts basically receives an xml (might weights up to 2 MB), parses it using Document object to get a list of files to be injected to the flow. Based on the list (in average 500 files), the script then creates a series of dummy jobs (0 byte-files containing the full file path in their private data) and send to the next element which is an Inject element.
During the monitoring, the error occurred out of the scope of the script process, during the handling of output jobs.
Marble,
Error: Communication was aborted...
Hi Marble,
I was only speaking about UNC path (i.e. "servermapping") definitions. Normal network paths set up via drives names (if we're talking about MS Windows) are not affected by this.
How do you inject that 500+ files with your script code?
Is it a for/next loop injecting 500+ newly created output jobs? If yes, that's maybe not the best idea to do it.
Maybe it's better to create a temporary folder path, copy all files into that temporary folder and then move that folder to the outgoing connection.
On the other hand that way the main problem is only shifted to a later stage of processing because at any time you have to break up the folder into its content files anyway and then again you have 500+ new jobs.
Switch was never good in handling a high number of new jobs which arrive simulaneously.
Regards,
Robert
I was only speaking about UNC path (i.e. "servermapping") definitions. Normal network paths set up via drives names (if we're talking about MS Windows) are not affected by this.
How do you inject that 500+ files with your script code?
Is it a for/next loop injecting 500+ newly created output jobs? If yes, that's maybe not the best idea to do it.
Maybe it's better to create a temporary folder path, copy all files into that temporary folder and then move that folder to the outgoing connection.
On the other hand that way the main problem is only shifted to a later stage of processing because at any time you have to break up the folder into its content files anyway and then again you have 500+ new jobs.
Switch was never good in handling a high number of new jobs which arrive simulaneously.
Regards,
Robert
-
marbledaemon
- Newbie
- Posts: 7
- Joined: Mon Sep 24, 2012 12:28 pm
Error: Communication was aborted...
Hello,
Exactly almost all our input folders are UNC (intranet).
The injection of 500 files are done by Inject Job element that is position right after the XML parsing script. All that the script does is to create dummy job containing the full path of the file to inject. Then the Inject Job element will take care of the injection. The XML parsing script does not copy the files itself.
However, if it is confirmed that the 500+ new jobs (even dummy) are the cause of problem, then we should reconsider. Moving 500+ single files (which are not from a same source folder) into a temporary folder within a same script will likely to get timed out, that's why we separate them into single jobs. We might try to lower the number of jobs by grouping 5-10 files into one job but many files weighs several hundred MB and would take a long time to copy over the network.
Surprisingly we didn't have that very problem using Switch 9.
Thank you.
Exactly almost all our input folders are UNC (intranet).
The injection of 500 files are done by Inject Job element that is position right after the XML parsing script. All that the script does is to create dummy job containing the full path of the file to inject. Then the Inject Job element will take care of the injection. The XML parsing script does not copy the files itself.
However, if it is confirmed that the 500+ new jobs (even dummy) are the cause of problem, then we should reconsider. Moving 500+ single files (which are not from a same source folder) into a temporary folder within a same script will likely to get timed out, that's why we separate them into single jobs. We might try to lower the number of jobs by grouping 5-10 files into one job but many files weighs several hundred MB and would take a long time to copy over the network.
Surprisingly we didn't have that very problem using Switch 9.
Thank you.