Custom Script Basics

Charles Jackson / May 23, 2022

Instalink offers an extensive library of custom data transformation functions that can be used to change input data from one format to another. There is thorough documentation of these transforms on Instalink.io. Be sure to check out the transform documentation if you have any questions about the specifics of the behavior of the transforms.

These transforms can also be used within custom transform scripts to create custom behavior. This post is intended to be a quick overview of how scripts work in Instalink and how to create custom behaviors for your transformations.

It is important to note that the scripting syntax isn't actually a full-fledged programming language. So it's not analogous to ECMA design patterns like javascript or C. It is intended to be more similar to Excel spreadsheet functions. So Instalink scripting follows a similar design pattern to how someone would write transforms on table data in a spreadsheet as that was initially our target audience for this software.

Because of the intended audience, there are certain concepts like "if / else" conditionals, custom methods, variable declarations, loops, or in script references that don't exist in the Instalink scripts. Basically we didn't want to give someone the opportunity to write something like "while(true)" in a script and accidentally take down their services. Everything is tightly controlled to ensure that the operations always complete efficiently and safely.

Here are the basic tenants of the Instalink data transform design pattern...

► Every transform action has one or more transforms. These can be predefined or they can be custom transforms using the scripting syntax.

► The predefined transform functions are the same as the functions available to the custom script syntax.

► Every transform has four components:

INPUT VALUE
This can be derived from a lookup key (for example myOrders[].id) or by a literal value within the script (for example "This is the actual value"). Literals only work in the scripts. Predefined transforms will always expect an input key to look up the value.
TRANSFORM FUNCTION
This is the function that will operate on the input value. See Instalink Transform Documentation for more details.
OUTPUT KEY
The output lookup key determines where the data will be stored.
ARGUMENTS
These are the additional values that will affect how the transform function operates on the input value.

► The lookup syntax for input keys, output keys, and any other lookup key all use the same syntax patterns:

Generally a set of characters will look up that object in the current process data.
For example, "object" will look for a record called "object" within the current process data.
The dot notation syntax signifies the lookup to check a subrecord of a specified record.
"object.subrecord" will look for a record called "object" and then on that record it will try to find a record called "subrecord"
The bracket notation signifies that the transform will run on every record that is found in an array.
"myArray[].value" as an input key will run the transform function on every object within the "myArray" data set. So if there are 500 records in myArray the transform will run 500 times.
The bracket notation also applies to the output key. So if you want to maintain structural symmetry between input and outputs then the keys should have the same number brackets within them.
An input key of "myArray[].innerArray[].input" and an output key of "myArray[].innerArray[].output" will cause the output key to have the same number of records and array structure as the input record.

► Transform functions always return a value.

► The value returned by a transform function can be used as an input for another transform function when called in a custom script.

► There is no specific return declaration in the scripts. So the value that is returned from the function is always applied to the output. If the functions are nested, the order of operation will always start with the innermost function first. For example...

FUNC1(FUNC2(FUNC3(VALUE))) 
#FUNC3() will run first, then it's returned value will be passed to FUNC2(), then it's value will be passed to FUNC1() whose return value will be assigned to the OUTPUT KEY.

► You CAN NOT have more than one top level function in a transform script. So the following is illegal within a single script...

FUNC1(VALUE)
FUNC2(VALUE)

► There is no guaranteed order or processing for transforms within the transform action.

So it is not advisable to have multiple transforms on a single transform action reference the same exact output key. This is because it is not guaranteed which one will run first. If you need to run multiple transforms on a specific output key and they must be done in a specific order, you must use multiple transform actions and place those in the desired order of operation.

► If an output key references documents or subdocuments that do not exist in the current process data, the documents and all required sub documents will be created to ensure that the output data can be stored at the correct location.

► With few exceptions, you cannot define variables within the scripting syntax. All inputs are derived from the values contained in the current process data.

So the following is not possible...

VARIABLE = "something"
FUNC1(VARIABLE)

► "#" is used to denote comments which will not be processed.

► There are no loops in the scripting syntax.

All loops are handled either internally by specific functions that expect to operate on iterable values or by the bracket syntax in the input key.

► There is no if / else or switch statement logic in the scripting syntax.

There are several functions that can be analogous to such conventions such as MAP_GET. For example...

MAP_GET(
    [ TRUE => "This is true" ],   # the condition
    BOOLEAN(VALUE),   # the value to check
    "This is false"   # the value to default to if the check doesn't match
)

...is operationally the same as the following javascript...

if (VALUE == TRUE)
{
    return "This is true";
}
else
{
    return "This is false";
}

Also, MAP_GET can also be used in a similar functionality to a switch statement...

MAP_GET(
    [ "A" => 1, "B" => 2, "C" => 3 ],
    VALUE,
    4
)

...is the same as the javascript...

let output;
switch (VALUE)
{
    case "A":
    {
        output = 1;
        break;
    }
    case "B":
    {
        output = 2;
        break;
    }
    case "C":
    {
        output = 3;
        break;
    }
    default:
    {
        output = 4;
        break;
    }
}
return output;

► MAP_GET can be nested just like any other function with arguments which allows you to create complicated conditional statements.

Doing a MAP_GET with another MAP_GET as the default value is the same as doing an if / else if script.

► ${ ... } notation can be used to perform a statically defined lookup within a script.

For example...

 FUNC( VALUE, ${ myObject.value } )

...allows you to reference a second object in the script beyond the input value.

► The special function LOOKUP() can be used to perform dynamically defined lookups within a script.

So you could do something like...

FUNC( LOOKUP( VALUE, TEXT_CONCAT("myDynamicKey_", ${myValue} ) ) )

► Beyond the functions, there are special constants that may always be used.

NULL // a record that is defined but has no set value.
UNDEFINED // the value is neither set nor defined.
VALUE // the result of the input key lookup.
NAN // the value is not a number
TRUE // truthy value
FALSE // falsey value
PI // The radian value of half a circle.

► Square brackets are used to denote mapped objects as well as arrays

[ "A" => 1, "B" => 2 ]

The above example is generally the same as the following javascript style object...

{ A: 1, B: 2 }

The brackets can also be used to define arrays...

[ "A", "B" ]

...denotes an array where "A" is the first value and "B" is the second value. And the following...

[ [ "A" => 1 ], [ "A" => 2 ] ]

...is the same as the following javascript style array with objects as each item...

[ { A: 1 }, { A: 2 } ]

► Basic mathematical operations can be performed in the scripting syntax.

For example, the following are both valid...

1 + 1
25 / 5

► You CANNOT concat strings in the syntax

"text" + "text" 
# this is not allowed...

You have to use a function...

TEXT_CONCAT("text", "text")

These are the basic rules of the custom scripts. There are many more advanced features of the scripting engine which will be explained in more detail in future posts.