Privatedata from filename

[Aristide]

Hello,

and thank you for this forum.

I want to detect some characters at the beginning of the file name in the flow to create a “myVariable/MyPrivateData” variable, with a regex :
([0-9]{4})(-123)(.*) = "toto"
([0-9]{4})(-456)(.*) = "titi"
([0-9]{4})(-789)(.*) = "tata"
etc

Then I can use these variables as private data in the flow arrows or variable paths.

Where can I find documentation or examples about this kind of private data creation ?
I have the Scripting module but i'm not a scripting expert... Someone has examples to share ?

Or : should it be better to write an xml file with all rules to be read for each jobs ?
<rules>
<rule pattern="([0-9]{4})(-123)(.*)" myVariable="toto" />
<rule pattern="([0-9]{4})(-456)(.*)" myVariable="titi" />
<rule pattern="([0-9]{4})(-789)(.*)" myVariable="tata" />
</rules>
In this case i'm completly lost ! Possible to do this technic too ?

Thank you very much for your help

AF

[jan_suhr]

Since you have the Scripting module you can use this app to add Private data to the job with scripting.
https://www0.enfocus.com/en/appstore/pr ... ivate-data

To do what you want is quite simple with scripting.

[Aristide]

Thank you Jan,

yes this is always the same problem ! I have the Scripting Module but not enough knowledge to use it well !
I still need to learn how to script. I can't find many sources for training...
Someone knows a site for training or to download examples ?

[jan_suhr]

The documentation shows some examples and give you instructions on the special functions Switch have.

There are some videos in the Learning section on the support pages on Enfocus.com

And then there is ChatGPT that can help you too.
Mention that you use Enfocus Switch in the question to ChatGPT and it knows what you are doing.

[loicaigon]

First of all, you could achieve this without scripting to the expense of more branches inside your flow.
But as you have the scripting module, you could indeed achieve your goal.

Another approach is to use this app:
https://www0.enfocus.com/en/appstore/pr ... ivate-data
But scripting is certainly needed for the logic if you want to avoid multiple branches.
But there ChatGPT provided me a script that worked in a blink and I just use it as script expression with the mentioned app.

I leave it to you to see what works the best for you.

Loic

[Aristide]

Merci Loïc !

I am always surprised as we often need to use third parties applications in Enfocus Switch.
Can't these apps disappear or not be updated?

Thanks to ChatGPT, i finally wrote a script and it is working very well.

Code: Select all

// Is invoked each time a new job arrives in one of the input folders for the flow element.
// The newly arrived job is passed as the second parameter.
function jobArrived(s, job) {
    var filename = job.getName();
    s.log(1, "Nom du fichier : " + filename);

// Define regex patterns and corresponding values
	var patterns = [
        { regex: /^([0-9]{4})-123(.*)/, value: "toto" },
        { regex: /^([0-9]{4})-456(.*)/, value: "titi" },
        { regex: /^([0-9]{4})-789(.*)/, value: "tata" }
    ];

	// Check the filename against each pattern
    var matched = false;

    for (var i = 0; i < patterns.length; i++) {
        var pattern = patterns[i];
        if (filename.match(pattern.regex)) {
            job.setPrivateData("myKey", pattern.value);
            s.log(1, "Correspondance trouvée : " + pattern.value);
            matched = true;
            break;
        }
    }

    if (!matched) {
        s.log(2, "Aucune correspondance trouvée pour : " + filename);
    }
	
//important pour avancer le job
    SendFileToConnection(s, job);
}

// Is invoked at regular intervals regardless of whether a new job arrived or not.
// The interval can be modified with s.setTimerInterval().
function timerFired(s) {
}

function SendFileToConnection(s, job, theFilePath) {
    var theOutConnections = s.getOutConnections();    
    for (var i = 0; i < theOutConnections.length; ++i) {
        job.sendTo(theOutConnections.at(i), job.getPath());
    }
}

So proud to share it here, this is my first script ! ChatGPT was very patient with me.
Don't hesitate to correct/improve the script please !
Again, if you have sources to learn like "scripting for dummies... please share the links.
And it seems this is an old way of scripting non ? Now we must use "node.js" ?
I hope Enfocus will maintain this way of scripting...


Loïc, can you send me how you use the Add Private Data application with regular expression please?

Thank you all and for this forum

AF

[tdeschampsBluewest]

Hi!
The script you're currently using has been written in a legacy format that will soon no longer be supported by Enfocus.

I strongly advise rewriting the script in Node.js/TypeScript. Unfortunately, GPT-based assistants won't provide much support for Enfocus-specific functions.

However, if you want better results you can usse an LLM with Retrieval-Augmented Generation (RAG) and upload the Enfocus documentation into it.
I personally tried this with Mistral, and the results were significantly better.


Here is the revised script, with added functionallity.

Code: Select all

async function getPatterns(flowElement: FlowElement): Promise<{ regex: RegExp; value: string }[]> {
    const patternProperty = await flowElement.getPropertyAsString("patterns");
    let array = patternProperty instanceof Array ? patternProperty : [patternProperty]; //put it in an array if only one line provided
    array = array.filter((i) => i.trim() !== ""); // Remove empty lines

    const patterns: { regex: RegExp; value: string }[] = [];
    for (let line of array) {
        line.split("="); // split the line into two parts, the first part is the regex and the second part is the value
        const regex = new RegExp(line[0]); // first part is the regex
        const value = line.slice(1).join("="); // second part is the value, the join("=") is to keep the separator in case of multiple "=" in the value
        patterns.push({ regex, value });
    }
    return patterns;
}

async function jobArrived(s: Switch, flowElement: FlowElement, job: Job) {
    const fileName = job.getName();
    await job.log(LogLevel.Info, `Nom du fichier : ${fileName}`);

    const patterns = await getPatterns(flowElement); //get the patterns from the property, should be presented like this in switch :
    /*
    ^([0-9]{4})-123(.*)=toto
    ^([0-9]{4})-123(.*)=tata
    ^([0-9]{4})-123(.*)=titi
    */
    const myKey = await flowElement.getVariableAsString("myKey"); //set the key as a variable in the sscript
    let value = "";
    for (let pattern of patterns) {
        if (fileName.match(pattern.regex)) {
            value = pattern.value;
            await job.log(LogLevel.Info, `Correspondance trouvée : ${pattern.value}`);
            break;
        }
    }
    if (value !== "") {
        await job.setPrivateData(myKey, value);
        await job.sendToData(Connection.Level.Success);
    } else {
        await job.log(LogLevel.Error, `Aucune correspondance trouvée pour le fichier : ${fileName}`);
        await job.sendToData(Connection.Level.Error);
    }
}

[JimmyHartington]

Hi Thomas

Can I ask on a low practical method which documentation you uploaded.
HTML-files, pdf or other?

[magnussandstrom]

However, if you want better results you can usse an LLM with Retrieval-Augmented Generation (RAG) and upload the Enfocus documentation into it.
I personally tried this with Mistral, and the results were significantly better.

I really hope Enfocus build an VS Code extension with pretrained a RAG-model (or similar). This would make writing Switch scripts really fast!

[tdeschampsBluewest]

Hi Thomas

Can I ask on a low practical method which documentation you uploaded.
HTML-files, pdf or other?

Hi Jimmy, i uploaded the documentation as a PDF, but i also uploaded some snippet i use for coding.

It's a beta feature, so i'm just toying with it right now and with agent (since it's free with Mistral).


here is an example of prompt :

Code: Select all

Create me a script for switch which will parse my filename against a set of regex/replacement value
If a regex is valid, it must create a privata data key with the replacement value, and the file trough Success connection, otherwhile in the error.

Here is the result (no change from me) :

Code: Select all

async function jobArrived(s: Switch, flowElement: FlowElement, job: Job) {
    try {
        // Define your regex/replacement pairs here
        const regexReplacements: { regex: RegExp; replacement: string }[] = [
            { regex: /pattern1/, replacement: 'replacement1' },
            { regex: /pattern2/, replacement: 'replacement2' },
            // Add more patterns as needed
        ];

        // Get the job's filename
        const jobName = await job.getName();
        let modifiedName = jobName;

        // Check each regex/replacement pair
        for (const { regex, replacement } of regexReplacements) {
            if (regex.test(jobName)) {
                modifiedName = jobName.replace(regex, replacement);
                // Create a private data key with the replacement value
                await job.setPrivateData('ModifiedFilename', modifiedName);
                break; // Stop after the first match
            }
        }

        // Log the result
        await job.log(LogLevel.Info, `Original filename: ${jobName}, Modified filename: ${modifiedName}`);

        // Send the job to the appropriate connection based on whether a regex matched
        if (modifiedName !== jobName) {
            await job.sendToSingle('Success'); // Send to Success connection
        } else {
            await job.sendToSingle('Error'); // Send to Error connection
        }
    } catch (error) {
        await job.log(LogLevel.Error, `Error processing job: ${error.message}`);
        await job.sendToSingle('Error'); // Send to Error connection on error
    }
}

The code itself won't work (sendToSingle are wrongly written), but it's impressive!

[loicaigon]

Hi Aristide,

You asked for how I used the "Add private data" and scripting. As a disclaimer, this solution is not that meaningful given that you end up with a script expression anyway. As you have the scripting module, just go for a complete script.

That said, the script ChatGPT wrote for me was:

Code: Select all

// Your imports here

// Do not remove calculateScriptExpression function. It must return a promise to the property value.
async function calculateScriptExpression(s: Switch, flowElement: FlowElement, job: Job): Promise<string> {
    // Get the job name
var jobName = job.getName();

// Define your regex patterns and return values
var rules = [
    { regex: /^([0-9]{4})(-123)(.*)/, value: "toto" },
    { regex: /^([0-9]{4})(-456)(.*)/, value: "titi" },
    { regex: /^([0-9]{4})(-789)(.*)/, value: "tata" }
];

// Check the job name against each pattern
for (var i = 0; i < rules.length; i++) {
    if (rules[i].regex.test(jobName)) {
        // Match found – return corresponding value
        return rules[i].value;
    }
}

// Default return if no pattern matched
return "unknown";
}

Just used with the mentioned app:

[Aristide]

Thank you for the details Loïc
Yes and using a script will allow me to modify it once for all flows.


Hi Thomas
About the legacy / node.js mode : did Enfocus sent a date before we must have rewrite all scripts ?

I will try to rewrite my script with node.js.
But without a very low knowledge for scripting and at level 0 in node.js concept in Switch or elsewhere...
It will take time...

Any help would be welcome !