Sending files to "in progress" folder before sending them to "final" folder

Post Reply
freakpazo
Newbie
Posts: 9
Joined: Fri Nov 03, 2023 3:46 pm

Sending files to "in progress" folder before sending them to "final" folder

Post by freakpazo »

Hello,

I am building a script that is taking images, getting data from them, sending the data to server and waiting for webhook to move them to specific folder. The script have one input connection and several output connections(depending on configuration).

Is there a way to first move the images to "working" folder(which is outgoing connection for the script) and at a later time(when the webhook arrives) to move them to another - "final" folder using connections and not hardcoding paths.

Ideal scenario will be as follows:
Script is connected to both the "working" and "final" folder with outgoing connections -> Inside jobArrived() use sendTo('working folder connection name') -> Inside timerFired() or when webhook arrives get the image from "working" folder with createJob('what is the path to working?')

The problem with this approach is that createJob() requires folder path and from what I saw there is no way to retrieve the folder path by the connection(maybe there is?). Possible solution will be to set the "working" folder path as flowElement property but my idea is that the script can be used on different machines and operate only with connections and connection names and not hard coding folder paths.

I am open to any solutions not only the suggestion provided above.
freddyp
Advanced member
Posts: 1023
Joined: Thu Feb 09, 2012 3:53 pm

Re: Sending files to "in progress" folder before sending them to "final" folder

Post by freddyp »

The "working" folder is your input folder. If you exit from the script without sending the input job it will stay in the input folder. This is usually combined with the use of timerFired in which you use getJobs() (see the sample code in the documentation) to get the list of waiting jobs and determine which ones are ready to be sent to output.

For the scenario that you describe you will use the httpRequestTriggeredAsync entry point to process the webhooks. You will see in the getJobs documentation that this call is not allowed in jobArrived and httpRequestTriggeredAsync within a concurrent script. This means you will have to define your script as a serialized one. Or in httpRequestTriggeredAsync you update the info about the waiting jobs in global data and you handle the sending to the output folder in timerFired.

You will need a timerFired entry point anyhow, because you do not want jobs to be stuck forever in the input folder, so having some sort of time-out feature is highly recommended.
freakpazo
Newbie
Posts: 9
Joined: Fri Nov 03, 2023 3:46 pm

Re: Sending files to "in progress" folder before sending them to "final" folder

Post by freakpazo »

Thank you I will try that.

A side question since it is related - The debugging of timerFired() function is really inconsistent for me.
I had no issues debugging inside jobArrived() with the following procces:
1. Start the script with debug mode true
2. Run the debugger in VS Code
3. Debugger hits line 1 in NodeScriptExecutor.js
4. Debugger hits line 1 in my main.ts
5. Debugger hits my breakpoint

Now when I try the same thing with timerFired()
- sometimes the debugger does not hit anything
- sometimes it hits only NodeScriptExecutor.js but does not hit main.ts
- sometimes it hits both NodeSCriptExecutor.js and main.ts on the first line but no my breakpoint
- sometimes it works as expected

The successes are maybe 1 in 10 tries or less. I notice in the messages the following appears:
The abort request to 'ws://127.0.0.1:55095/api/v1/process' has been sent successfully.
Process was aborted
Flow element timer was aborted because the operation timed out

I removed all my code from timerFired() and just set 1 line to log a message to make sure it is not related to my code and the problem still occurs
freddyp
Advanced member
Posts: 1023
Joined: Thu Feb 09, 2012 3:53 pm

Re: Sending files to "in progress" folder before sending them to "final" folder

Post by freddyp »

In the Debug settings you can define what entry point you want to debug. Make sure you only list timerFired and when you start the flow it will immediately start the debug session. I am not sure it is really necessary, but what I always do it to set the timer interval to a long period so Switch will certainly not start a new debug session when I am still busy debugging.
Post Reply