How to use flowElement.createJob to save a writeStream

Post Reply
bkromer
Member
Posts: 99
Joined: Thu Jul 11, 2019 10:41 am

How to use flowElement.createJob to save a writeStream

Post by bkromer »

Hey I want to download a file with axios how can I create a tempfile for sendToSingle?

Code: Select all

 ...const response = await axios({
      method: 'GET',
      url: fileUrl,
      responseType: 'stream',
    });
    
    const writer = response.data.pipe(fs.createWriteStream(tempFilePath));
   
I read in the docs that createPathWithName is not supported anymore and now I have to use flowElement.createJob but cant get it to work.
What do I need to pass to the function?
Can somebody provide a code snippet? I have a URL and need to download a pdf file.

Thanks in Advance
Benjamin
mkayyyy
Member
Posts: 75
Joined: Mon Nov 21, 2016 6:31 pm
Location: UK

Re: How to use flowElement.createJob to save a writeStream

Post by mkayyyy »

You'll need to use a library like https://www.npmjs.com/package/tmp to create a temp file to write to

Below is a code example:

Code: Select all

import * as tmp from "tmp";
import * as fs from "fs";
import axios from "axios";

async function jobArrived(s: Switch, flowElement: FlowElement, job: Job) {
    const fileUrl = "http://www...";

    // Create temp file
    const tempFile = tmp.fileSync({ postfix: ".pdf" });

    const response = await axios({
        method: 'GET',
        url: fileUrl,
        responseType: 'stream',
    });
        
    response.data.pipe(fs.createWriteStream(tempFile.name));

    const newJob = await flowElement.createJob(tempFile.name);
    await newJob.sendToSingle();

    // Remove temp file
    tempFile.removeCallback();
}
bkromer
Member
Posts: 99
Joined: Thu Jul 11, 2019 10:41 am

Re: How to use flowElement.createJob to save a writeStream

Post by bkromer »

mkayyyy wrote: Tue Jan 04, 2022 9:53 am You'll need to use a library like https://www.npmjs.com/package/tmp to create a temp file to write to

.....
Thanks for the snippet, works perfectly! :)
Benjamin
bkromer
Member
Posts: 99
Joined: Thu Jul 11, 2019 10:41 am

Re: How to use flowElement.createJob to save a writeStream

Post by bkromer »

I tried to use this code to download single files from dropbox via API. But there it doesn't work!? I get an empty PDF file.
But when I instead save the file locally it works

Code: Select all

await response.data.pipe(fs.createWriteStream(path));
I guess I need to use some sort of temporary file to create a new job? Why not just save locally and pass this as new job?

I have the following code:

Code: Select all

const fs = require('fs');
const axios = require('axios');

async function jobArrived(s, flowElement, job) {

  const jsonFile = await job.get(AccessLevel.ReadWrite);
  const json = JSON.parse(fs.readFileSync(jsonFile));
  let {
    fileName,
    downloadUrl,
    ninoxId,
    databaseId,
    teamId,
    tableId
  } = json;

  if (fileName, downloadUrl, ninoxId, databaseId, teamId, tableId) {
    try{
      let config = {
        method: 'post',
        url: 'https://content.dropboxapi.com/2/files/download',
        headers: {
          'Content-Type':'application/octet-stream; charset=utf-8',
          'Dropbox-API-Select-User': 'MYID',
          'Dropbox-API-Path-Root': '{".tag": "namespace_id", "namespace_id": "MYNAMESPACE"}',
          'Dropbox-API-Arg': `{"path":"/MyFolder/${fileName}"}`,
          'Authorization': 'Bearer TOKEN'
        },
        responseType: 'stream',
        data:null
      };
      const response = await axios(config);
      await job.log(LogLevel.Info, `API CALL RESPONSE STATUS CODE: ${response.status}`);
      let myPath = `C:\\Users\\MYFOLDER\\${fileName}`;
      await response.data.pipe(fs.createWriteStream(myPath));
      const newJob = await flowElement.createJob(`${myPath}`);
      await newJob.setPrivateData('pdNINOXID', ninoxId);
      await newJob.setPrivateData('pdDATABASEID', databaseId);
      await newJob.setPrivateData('pdTEAMID', teamId);
      await newJob.setPrivateData('pdTABLEID', tableId);
      await newJob.setPrivateData('pdFILENAME', fileName);
      await newJob.sendToSingle(fileName);
      await job.sendToNull(jsonFile);
  }catch (err){
    await job.log(LogLevel.Info, `error: ${err}`);
    await job.sendToNull(jsonFile);
 }
  } else {
    await job.sendToNull(jsonFile);
    await job.log(LogLevel.Error, `PRIVATE DATA IS MISSING! One of downloadUrl, ninoxId, databaseId, teamId, tableId! File: ${job.getName()}` );
    await job.fail(`Something went wrong with the job${job.getName()}` );
  }
}
Still sometime this doesnt work :oops:
Benjamin
Post Reply