flowElement.getJobs() in flowStartTriggered entry point?

Post Reply
pcobee
Newbie
Posts: 15
Joined: Fri Apr 01, 2011 5:06 pm
Location: Greenville NC
Contact:

flowElement.getJobs() in flowStartTriggered entry point?

Post by pcobee » Mon Jan 17, 2022 11:01 pm

If I call flowElement.getJobs( ids ) in flowStartTriggered, the resulting jobs collection is empty. I've verified that I'm passing it a valid array of job ids. Has anyone gotten this to work? The documentation says...

The flowStartTriggered entry point is executed before any jobArrived and timerFired.

... so I'm starting to think that Switch hasn't internally created the collection of jobs yet as jobArrived is subsequently executed for all existing jobs on flow activation. I was hoping to use this to validate the actual jobs against my collection of job information kept in global data.

freddyp
Advanced member
Posts: 726
Joined: Thu Feb 09, 2012 3:53 pm

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by freddyp » Tue Jan 18, 2022 8:43 am

I do not see the added value of doing something with the list of waiting jobs in flowStartTriggered. For all the use cases I have seen so far flowElement.getJobs() belongs in the timerFired entry point.

pcobee
Newbie
Posts: 15
Joined: Fri Apr 01, 2011 5:06 pm
Location: Greenville NC
Contact:

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by pcobee » Tue Jan 18, 2022 5:14 pm

That was plan B. Since it would be adequate to make this check only at flow start up I was hoping to reduce the load on the server by not having to make the check every time timerFired was invoked. During our "peak" this app will have a number of instances running with around 20 to 30 thousand files in process at any point in time.

freddyp
Advanced member
Posts: 726
Joined: Thu Feb 09, 2012 3:53 pm

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by freddyp » Tue Jan 18, 2022 5:35 pm

In timerFired check if a certain piece of global data exists that defines that the waiting jobs have been checked. If it does not exist, define it and check the jobs. If it exists, set a very long timer interval and exit from timerFired. In flowStopTriggered remove that global data.

pcobee
Newbie
Posts: 15
Joined: Fri Apr 01, 2011 5:06 pm
Location: Greenville NC
Contact:

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by pcobee » Tue Jan 18, 2022 7:30 pm

I like it!

pcobee
Newbie
Posts: 15
Joined: Fri Apr 01, 2011 5:06 pm
Location: Greenville NC
Contact:

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by pcobee » Tue Jan 25, 2022 8:33 pm

When timerFired is called the very first time after a flow is restarted, if you call getJobs() you will get an empty list - even if there are actual jobs in the incoming connection. After timerFired is envoked, jobArrived is called for each job in the incoming connection which then re-populates the internal list of jobs and subsequent invocations of timerFired WILL return the list of jobs.

So, here is how I use timerFired to validate the job information in global data against the actual jobs waiting...

To facilitate identifying which jobs actually exist, I include a "valid" field in each job's global data entry

in timerFired
if control-switch does not exist (this will be on the fist invocation)
- setGlobalData(scope, control-switch, "run check") and exit

if control-switch = "run check" (this will be on the second invocation)
- get your job info list from global data
- build the filteredIds list of jobids and at the same time set each job's valid field to false
- call getJobs(filteredIds) to get the actual list of jobs
- loop through the list of actual jobs and use the id (job.getId()) to update each jobs valid field
- any jobs in the global data that don't exist will still have their valid field set to false
- loop through the jobs in global data and delete any job where valid = false
- resave job info to global data
- setGlobalData(scope, control-switch, "check complete")

if control-switch = "check complete" - resume normal processing

in flowStopTriggered
- call removeGlobalData(scope, control-switch) to delete your control-switch

freddyp
Advanced member
Posts: 726
Joined: Thu Feb 09, 2012 3:53 pm

Re: flowElement.getJobs() in flowStartTriggered entry point?

Post by freddyp » Wed Jan 26, 2022 11:24 am

Thanks a lot for sharing your experiences and insights!

It may not be necessary here but when I implement a use case where jobs are kept in the input folder I add a time-out property. In jobArrived I add the time stamp in Unix milliseconds (new Date().getTime()) to the data stored in global data. In timerFired that allows me to perform an additional check to see that a job has exceeded the user-defined time-out and send it to the error data connection.

Post Reply