Enfocus Community

Posted: **Tue Jun 29, 2021 9:48 am**

Hi,
I'm trying to use the new flowElement.getJobs() method, so I have code very similar to the one in the documentation (jobsIds are stored in globaldata during jobArrived, I'm calling getJobs() in timerFired() entry point).

My problem is : how to deal with invalid jobID ?

Freddy said during a recent webinar that this should not happen, otherwise it's is a logic error in our code.

But in fact, I'm exactly in this situation after that an user delete a job in the input folder, from the Switch Designer.

I have no way to know that the job is not in the input folder anymore, may I?

So I can't remove this particular job from the globalData list.

Do you have a way to prevent such situation?

I don't know if it is possible, but I think that the getJobs() method should not throw an error if one jobID is invalid in the list.

Maybe it should answers by putting a "placeholder job" with a special "invalid job" state ?

Or at least, we should have information about the invalid jobID in the error object so we can act upon the globalData list ?

Posted: **Wed Jun 30, 2021 8:58 am**

I did not think of a user deleting a file behind your back because then indeed you will have an id in global data, but not in getJobs().

I tested it with the code from the documentation. The only thing I did was to reduce the timer interval to 5 seconds and the time-out period to 30 seconds and I added two lines of logging. It worked, also when I removed one of the input jobs. There was a warning message in that case, but that is only normal. Perhaps you could try this code unchanged to see what happens.

Code: Select all

async function jobArrived(s, flowElement, job) {
  let jobsInfo = (await s.getGlobalData(Scope.FlowElement, "jobsInfo")) || {};
  const jobId = job.getId();
  jobsInfo[jobId] = jobsInfo[jobId] ? jobsInfo[jobId] : new Date().getTime();
  await s.setGlobalData(Scope.FlowElement, "jobsInfo", jobsInfo); // collect jobs IDs to use them afterwards as parameter for the getJobs call
  await job.log(LogLevel.Debug, "Job arrived: " + JSON.stringify(jobsInfo));
}

async function timerFired(s, flowElement) {
  await flowElement.setTimerInterval(5); // interval for 'holding' jobs
  const jobsInfo = await s.getGlobalData(Scope.FlowElement, "jobsInfo");
  await flowElement.log(LogLevel.Debug, "Timer fired: " + JSON.stringify(jobsInfo));
  const currentTimestamp = new Date().getTime();
  let filteredIds = [];
  for (let id in jobsInfo) {
    // choose only jobs that have been staying in the flow element for at least 1 hour
    if (jobsInfo[id] + 30 * 1000 <= currentTimestamp && filteredIds.length <= 10000) {
      filteredIds.push(id);
      delete jobsInfo[id];
    }
  }
  if (filteredIds.length > 0) {
    const jobs = await flowElement.getJobs(filteredIds); // note: returns max 10000 jobs
    for (let job of jobs) {
      await job.sendToSingle(); // move each job to the outgoing connection
    }
    await s.setGlobalData(Scope.FlowElement, "jobsInfo", jobsInfo);
  }
}

Posted: **Fri Jul 02, 2021 10:20 am**

I did not think of a user deleting a file behind your back because then indeed you will have an id in global data, but not in getJobs().

I didn't think of it too, but indeed, it happens pretty soon when my first script with getJobs() goes live ...

If I understand the changes you made in the exemple code, you just reduced the window of opportunity for the problem to occur... or maybe I miss something??

In my opinion, it's a real and non trivial potential issue : let's take an example, something like Enfocus's own Pdf review module ...
In this kind of script/app, the files can stay in inputs for hours or days...

I'm sure that pretty soon, a user will delete a job pending approval because it's no longer relevant or because it's en error, they don't submit the right file... or just by mistake

Users are human and make bad decisions, you have to deal with that.

It's not a big deal to have an exception when calling getJobs()... the problem imho is that it's an "all or nothing" error, we know that we have an invalid ID but we don't know which one(s) ...

In my case, even if I have hundred files in input, my getJobs() call is only on 2 or 3 ids each time... so my workaround was, if i encounter an error, to test each jobid one by one on the 2 or 3 next timerFired events ... it's a bit slow and cumbersome but that doesn't happen every day either ...

But I can't reasonably do that if my jobID's list contains 10 items or more...

It'll be better to know which jobsID are invalid in the error.

Enfocus Community

flowElement.getJobs() and invalid job ID

flowElement.getJobs() and invalid job ID

Re: flowElement.getJobs() and invalid job ID

Re: flowElement.getJobs() and invalid job ID