Page 1 of 1
EMFILE: too many open files
Posted: Tue Feb 01, 2022 2:16 pm
by borisCM
Hi,
i think i am doing something wrong in my node.js projects.
after a while i get a lot of
"EMFILE: too many open files, open" errors in switch
i'm working on windows
i am using tmp module to create the tmp files
Code: Select all
const tmp = import("tmp")
let orderXMLdocTMP = (await tmp).fileSync();
and fs to delete them when i'm done with the file.
Code: Select all
try {
(await fs).unlinkSync(orderXMLdocTMP.name); //delete temp file
await job.log(LogLevel.Info, `unlinkSync ${orderXMLdocTMP.name} succes`);
} catch (err) {
await job.log(LogLevel.Error, err);
}
i don't get any errors back. but the file is not deleted in the temp folder.
so after a day i have a 1000 temp files in that folder. and when i try to open them they all have "access denied".
can anyone help?
what am i doing wrong?
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 5:15 pm
by freddyp
There is no need to use await with the synchronous methods of tmp.
Check the user access rights of the file that were created. If they are not OK, I suggest you try adding an access mode at the moment of creating the temporary file:
Code: Select all
let orderXMLdocTMP = tmp.fileSync( { mode: 0o777 } );
The documentation of tmp uses this variant as well. I do not know if both work or not:
Code: Select all
let orderXMLdocTMP = tmp.fileSync( { mode: 0777 } );
777 means read/write/execute for everybody.You can find more info here:
https://en.wikipedia.org/wiki/File-system_permissions
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 5:25 pm
by borisCM
thanks for the quick response.
the await gets added automatically while typing and if i dont place it, visual studio code gives a type error.
Property 'fileSync' does not exist on type 'Promise<{ default: typeof import(.../DEV/ImpPPXML/node_modules/@types/tmp/index"); file(options: FileOptions, cb: FileCallback): void; file(cb: FileCallback): void; ... 6 more ...; tmpdir: string; }>'.ts(2339)
main.ts(1115, 34): Did you forget to use 'await'?
i also can't transpile without the (await tmp)
Info: The script package from the folder 'C:/Users/switch/Dropbox/Switch/BackupSwitch/Scripts-XSLT/NodeJS/DEV/ImpPPXML' is packing
Error: Failed to transpile the code: TypeScript failed with exit code 2
Error: TypeScript standard output: ../../../../../../../AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML/main.ts(1284,30): error TS2339: Property 'fileSync' does not exist on type 'Promise<{ default: typeof import(/AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML/node_modules/@types/tmp/index"); file(options: FileOptionsDiscardFd, cb: FileCallbackNoFd): void; file(options: FileOptions, cb: FileCallback): void; file(cb: FileCallback): void; ... 6 more ...; tmpdir: string; }>'.
Error: Failed to encrypt the script package 'C:/Users/switch/AppData/Local/Temp/2/ImpPPXMLMvZEUq/ImpPPXML'
ps:
changing the mode Oo777 does not help. will get in touch with our IT partner.
i would expext a fs.unlinksync err when deleting is not possible?
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 5:41 pm
by freddyp
Did you add the typings for tmp?
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 5:46 pm
by borisCM
jes jes.
the standard fs has the same behavior and there i don't need to install the types separatly normally
i need to add it like this:
Code: Select all
(await fs).writeFileSync(tmpBoundCover.name, xmlStringBoundCover, "utf-8");
(await fs).unlinkSync(tmpBoundCover.name);
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 5:54 pm
by borisCM
this fixed the type issue
Code: Select all
import * as fs from "fs";
import * as tmp from "tmp";
instead of
Re: EMFILE: too many open files
Posted: Tue Feb 01, 2022 6:22 pm
by borisCM
think i found a solution
https://raszi.github.io/node-tmp/
Code: Select all
const tmp = require('tmp');
const tmpobj = tmp.fileSync();
console.log('File: ', tmpobj.name);
console.log('Filedescriptor: ', tmpobj.fd);
// If we don't need the file anymore we could manually call the removeCallback
// But that is not necessary if we didn't pass the keep option because the library
// will clean after itself.
tmpobj.removeCallback();