EMFILE: too many open files

Post Reply
borisCM
Member
Posts: 37
Joined: Mon May 06, 2019 12:23 pm
Location: Netherlands

EMFILE: too many open files

Post by borisCM »

Hi,

i think i am doing something wrong in my node.js projects.
after a while i get a lot of
"EMFILE: too many open files, open" errors in switch

i'm working on windows

i am using tmp module to create the tmp files

Code: Select all

const tmp = import("tmp")
let orderXMLdocTMP = (await tmp).fileSync();
and fs to delete them when i'm done with the file.

Code: Select all

try {
      (await fs).unlinkSync(orderXMLdocTMP.name); //delete temp file
      await job.log(LogLevel.Info, `unlinkSync ${orderXMLdocTMP.name} succes`);
    } catch (err) {
      await job.log(LogLevel.Error, err);
    }
i don't get any errors back. but the file is not deleted in the temp folder.
so after a day i have a 1000 temp files in that folder. and when i try to open them they all have "access denied".

can anyone help?
what am i doing wrong?
freddyp
Advanced member
Posts: 1008
Joined: Thu Feb 09, 2012 3:53 pm

Re: EMFILE: too many open files

Post by freddyp »

There is no need to use await with the synchronous methods of tmp.

Check the user access rights of the file that were created. If they are not OK, I suggest you try adding an access mode at the moment of creating the temporary file:

Code: Select all

let orderXMLdocTMP = tmp.fileSync( { mode: 0o777 } );
The documentation of tmp uses this variant as well. I do not know if both work or not:

Code: Select all

let orderXMLdocTMP = tmp.fileSync( { mode: 0777 } );
777 means read/write/execute for everybody.You can find more info here: https://en.wikipedia.org/wiki/File-system_permissions
borisCM
Member
Posts: 37
Joined: Mon May 06, 2019 12:23 pm
Location: Netherlands

Re: EMFILE: too many open files

Post by borisCM »

thanks for the quick response.

the await gets added automatically while typing and if i dont place it, visual studio code gives a type error.
Property 'fileSync' does not exist on type 'Promise<{ default: typeof import(.../DEV/ImpPPXML/node_modules/@types/tmp/index"); file(options: FileOptions, cb: FileCallback): void; file(cb: FileCallback): void; ... 6 more ...; tmpdir: string; }>'.ts(2339)
main.ts(1115, 34): Did you forget to use 'await'?
i also can't transpile without the (await tmp)
Info: The script package from the folder 'C:/Users/switch/Dropbox/Switch/BackupSwitch/Scripts-XSLT/NodeJS/DEV/ImpPPXML' is packing
Error: Failed to transpile the code: TypeScript failed with exit code 2
Error: TypeScript standard output: ../../../../../../../AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML/main.ts(1284,30): error TS2339: Property 'fileSync' does not exist on type 'Promise<{ default: typeof import(/AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML/node_modules/@types/tmp/index"); file(options: FileOptionsDiscardFd, cb: FileCallbackNoFd): void; file(options: FileOptions, cb: FileCallback): void; file(cb: FileCallback): void; ... 6 more ...; tmpdir: string; }>'.

Error: Failed to encrypt the script package 'C:/Users/switch/AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML'
ps:

changing the mode Oo777 does not help. will get in touch with our IT partner.
i would expext a fs.unlinksync err when deleting is not possible?
Last edited by borisCM on Tue Feb 01, 2022 5:41 pm, edited 1 time in total.
freddyp
Advanced member
Posts: 1008
Joined: Thu Feb 09, 2012 3:53 pm

Re: EMFILE: too many open files

Post by freddyp »

Did you add the typings for tmp?

Code: Select all

npm i @types/tmp --save-dev
borisCM
Member
Posts: 37
Joined: Mon May 06, 2019 12:23 pm
Location: Netherlands

Re: EMFILE: too many open files

Post by borisCM »

jes jes.

the standard fs has the same behavior and there i don't need to install the types separatly normally

i need to add it like this:

Code: Select all

(await fs).writeFileSync(tmpBoundCover.name, xmlStringBoundCover, "utf-8");
(await fs).unlinkSync(tmpBoundCover.name);
borisCM
Member
Posts: 37
Joined: Mon May 06, 2019 12:23 pm
Location: Netherlands

Re: EMFILE: too many open files

Post by borisCM »

this fixed the type issue

Code: Select all

import *  as fs from "fs"; 
import * as tmp from "tmp"; 
instead of

Code: Select all

const fs = import("fs");
borisCM
Member
Posts: 37
Joined: Mon May 06, 2019 12:23 pm
Location: Netherlands

Re: EMFILE: too many open files

Post by borisCM »

think i found a solution

https://raszi.github.io/node-tmp/

Code: Select all

const tmp = require('tmp');

const tmpobj = tmp.fileSync();
console.log('File: ', tmpobj.name);
console.log('Filedescriptor: ', tmpobj.fd);
  
// If we don't need the file anymore we could manually call the removeCallback
// But that is not necessary if we didn't pass the keep option because the library
// will clean after itself.
tmpobj.removeCallback();
Post Reply