Scripting Reference

Overview

West Connectivity is an event-driven system that uses scripts to route data and perform application logic and rules. These scripts have a rich set of capabilities and are used to perform such actions as storing device data into a time series data store, offloading processing from your devices, and handling Application Solution API requests. These scripts have access to all of the West Connectivity services. A reference for each of these services and their functionality can be found here: Service Reference.

Scripts are written in Lua, on the LuaJIT VM, which is Lua 5.1 with some 5.2 features. For general information about Lua 5.1, please refer to the online Lua manual.

Scripts may be added to an Application by using the West Connectivity web UI

---

Service Call

The besides native Lua scripting, West Connectivity script offers direct access to high levels functionalities through Micro-Services integration. To used them, once you have the desired service enabled in your solution, you just need to make a script function call using the Capitalized service name as global reference and the operation name as function.

West Connectivity Services can also provide configuration parameters configurable from the West Connectivity User-Interface under the Services tab. Those parameters will then be used automatically for interacting with the service. This prevent for instance the need of passing secret credentials through the scripting environment.

The list of operations, arguments and response are defined for each services on West Connectivity Services page.

You can also call a service using the system.service_call() function.

Example

To send an email, the Email Service send operation can be called from the script with:

local result = Email.send({to="my@friend.com", subject="Hello", text="World"})

The Email service expose configuration parameters to set custom SMTP server. If configured those parameters will be automatically provided when calling the service.

Error handling

The service call functions are multi-return, with the first value being the result, and the second value being an error value. If the error value is nil, then the service call was successful, and the result can be used safely. If non-nil, the error value will be a Lua Table, as shown below.

If for any reason the service call failed, the following Lua Table is returned to the script as both the result and the error value:

{
  "error": "Error details, typically the Service response as a string",
  "status": 400, -- The status code
  "type": "QueryError" -- Either QueryError or ServerError
}

If the service call responds with a 204 HTTP status code, the following Lua Table is returned to the script as the result (with the error value being nil):

{
  "status":204
}

Service call costs

Service compute resource usage is aggregated to the global solution processing_time usage metric along with the Lua scripting CPU time.

You can profile your service compute usage with the system.runtime_stats() function.

Mind that many services also report other usage metrics such as the storage size.

---

Script Execution

West Connectivity Lua scripts are executed in reaction to system events, which are defined by West Connectivity services.

Example: The Device2 service emits the event event whenever a message is sent by a device to the Device API.

Service events will trigger the execution of the matching event handler script defined in your application script. Events and their payload are documented in their respective Micro-Service page.

To get a try, use the script editor under the Services tab of the West Connectivity Solution page to define event scripts (Eg. the Timer service timer event).

Cluster Behavior

Each events will trigger the related script execution in a unique separated Virtual Machine which ensure full isolation in the system. This means multiple scripts of your solution may be executed at the same time which enables your application to scale horizontally based on the system load.

---

Event Handler Context

During execution, event handler scripts are wrapped in a handler function corresponding to the West Connectivity Services event. The event handler function will expose arguments defined by the West Connectivity service event.

Note

West Connectivity event handler scripts share the same context within an application. Therefore, the use of the local keyword is highly recommended for every function and variable definitions to avoid potential name collision.

Example

Incoming data from an IoT device are triggered by the Device service datapoint event which defines an event parameter for the event handler context (note that for version 1 of this service the parameter was called data and had different properties). The wrapper function will therefore be:

function handle_device2_datapoint (event)
  -- Here goes your device data handler script
end

So when your event handler script defines:

print(event.type)

The final execution script will look like the following:

function handle_device_datapoint (event)
  print(event.type)
end

---

API Endpoint Scripts

For convenience, West Connectivity offers the option to build your custom HTTP API by defining endpoint scripts.

Endpoint scripts are automatically wrapped in a simple routing mechanism based on the Webservice West Connectivity service set within the event handler for the "request" event. As for a regular Service event handler, the script will receive the request event arguments containing the HTTP request data.

An extra response argument is provided to the endpoint script context allowing a compact response syntax.

The response object content:

Attribute	Type	Default Value	Description
code	integer	200	The response HTTP status code. If any exception occurs, a 500 is returned.
message	string or table	"Ok"	The HTTP response body. If a Lua table is given, the routing wrapper will automatically encode it as JSON object.
headers	Table of string	"content-type" = text/plan or application/json	The HTTP response headers depend on message type.

Example

response.headers = {} -- optional
response.code = 200 -- optional
response.message = "My response to endpoint " .. request.uri

or

return "My response to endpoint " .. request.uri

---

Endpoints Functions Context

Under the hood, endpoints scripts are stored in an _endpoints table used by the Webservice "request" event handler. So the final API script will be:

local _endpoints = {
    ["get_/myendpoint"] = function (request, response)
      -- endpoint script
      return "My response to endpoint " .. request.uri
    end
}
function handle_webservice_request(request)
  ... -- default routing mechanism
end

Websocket Endpoints

Websocket endpoints are handled in a similar manner as webservice endpoints based on Websocket West Connectivity service. The function context includes the websocketInfo as well as a response arguments. The response.message content will be automatically sent back to the websocket channel.

response.message = "Hello world"

Or you can directly pass the message as function result:

return "Hello world"

In addition, you can also interact with the websocket channel with the following functions:

websocketInfo.send("Hello") -- Send a message to the channel.
websocketInfo.send("world") -- Useful to send back multiple messages.
websocketInfo.close() -- Close the websocket connection

Websocket Endpoints Functions Context

Similar to the webservice endpoints, websockets are stored in the _ws_endpoints table. And final script at execution will be:

local _ws_endpoints = {
    ["/mywebsocketendpoint"] = function (websocketInfo, response)
      -- websocket endpoint script
      return "My response to endpoint " .. request.uri
    end
}
function handle_websocket_websocket_info(websocketInfo)
  ... -- default routing mechanism
end

---

Modules

West Connectivity recommends the use of a reusable block of Lua code. For this purpose, you can define Lua modules under the modules/ directory of your project.

West Connectivity modules follow Lua standard and should end with a return statement returning the module object. The module can then be accessed by other scripts using the lua require "moduleName" keyword.

Example

Module: "myModule"

local moduleObject = { variable = "World"}

function moduleObject.hello()
  return moduleObject.variable
end

return moduleObject

Usage

local moduleInstance = require "myModule"

moduleInstance.hello() -- -> "World"

To Keep in Mind About Modules

• Naming: To avoid confusion with West Connectivity services, module names need to start with a lowercase letter. Module names can contain '.' to represent the source file folder structure. The module can then be used with lua require "path.to.module"
• Context isolation: West Connectivity modules are executed in the same namespace as other script of the application. The use of local keyword is highly recommended for every function and variable definitions to avoid potential isolation issues.

Deprecated Behavior

For compatibility with earlier versions of West Connectivity, following non-standard is still supported but will get removed in a future release and should be avoided.

Module Constructor Call

Direct module constructor can be accessed directly without the lua require call.

Usage Example

local moduleInstance = myModule()

moduleInstance.hello() -- -> "World"

Module without return Statement

Legacy module definitions which omit the return statement are directly pre-pended to the application script which requires the following considerations:

• Isolation: As top-level script, the module script does not have any specific isolation.
• Loading order: The modules loading order is not ensured and cross-module calls should be avoided.
• Naming: The module name is irrelevant. Variable and function names will directly be accessible for other non-module scripts.
• Non-functional script: Module scripts which do not contain any functions are considered standard Lua modules with a lua return nil statement.

Deprecated module example:

myModule = { variable = "World"} -- This is a global definition

function myModule.hello()
  return myModule.variable
end

Usage

myModule.hello() -- -> "World"

---

Troubleshooting

The Lua script execution is recorded in the application logs and is accessible through the application management console under the Logs tab. To emit log from lua scripting see the log section.

The more detail description is available in the Solution Debug logs

---

Script Environment

Scripts are executed in their own sandboxed instance of the Lua VM to keep them isolated from each other. Each script has access to all West Connectivity Services, but access to those services is authenticated based on your Application.

Limits

Each script execution is resource constrained with following limitations: - 20 MB of memory per execution - 10 seconds of CPU time per event execution. This value is a component of your solution wide processing_time usage along with services usage. - There is no real time clock limitation

In cases where either limit is exceeded, your script will immediately fail with an error message explaining why.

You can profile your script resource usage with the system.runtime_stats() function.

If you are curious how Lua 5.1 memory management works, please see the following references:

• Garbage Collection
• Visibility Rules

---

Native Libraries

The following global Lua tables and functions are available to Lua scripts. They operate exactly as described in the Lua 5.1 reference manual.

• Basic Functions (Note: the dofile function is NOT available to scripts.)

• string (Note: the string.dump function are NOT available to scripts.)

• table

• math

• os (Note: Only os.difftime, os.date, os.time, os.clock, os.getenv function are available to scripts.)

Lua Number Type

Lua Number is following IEEE 754 64-bit double-precision floating point. Largest power of ten: a 64-bit double can represent all integers exactly, up to about 1,000,000,000,000,000 (actually - 2^52 ... 2^52 - 1).

---

West Connectivity libraries

In addition to the Lua system resources, the following global features are available to Lua scripts:

json

to_json

Converts a Lua table to a JSON string. This function is multi-return, the first value being the result, and the second value being an error value. If the error value is nil, then the conversion was successful, and the result can be used safely.

If non-nil, the error value will be a string.

local jsonString, err = to_json({})
if err ~= nil then
  print(err)
end

-- Or directly

local jsonString = to_json({})

Since items with nil values in a Lua table effectively don't exist, you should use json.null as a placeholder value if you need to preserve null indices in your JSON string.

local t = {
    name1 = "value1",
    name2 = json.null
}
local jsonString = to_json(t)
print(jsonString)
--> {"name1":"value1","name2":null}

from_json

Converts a JSON string to a Lua table. This function is multi-return, the first value being the result, and the second value being an error value. If the error value is nil, then the conversion was successful, and the result can be used safely.

If non-nil, the error value will be a string.

local luaTable, err = from_json("{}")
if err ~= nil then
  print(err)
end

-- Or directly

local luaTable = from_json("{}")

By default, JSON nulls are decoded to Lua nil and treated by Lua in the normal way. When the optional argument {decode_null=true} is used, null is interpreted as json.null within the table. This is useful if your data contains items which are "null" but you need to know of their existence (in Lua, table items with values of nil don't normally exist).

local luaTable = from_json("[1, 2, 3, null, 4, null]", {decode_null=true})
print(#luaTable)
--> 6

json.null

json.is_null()

Finds whether a variable is json.null

print(json.is_null(json.null))
--> true

print(json.is_null("test"))
--> false

json.stringify()

This functions the same as to_json

local jsonString, err = json.stringify({})
if err ~= nil then
  print(err)
end

-- Or directly

local jsonString = json.stringify({})

local t = {
    name1 = "value1",
    name2 = json.null
}
local jsonString = json.stringify(t)
print(jsonString)
--> {"name1":"value1","name2":null}

json.parse()

This functions the same as from_json

local luaTable, err = json.parse("{}")
if err ~= nil then
  print(err)
end

-- Or directly

local luaTable = json.parse("{}")

local luaTable = json.parse("[1, 2, 3, null, 4, null]", {decode_null=true})
print(#luaTable)
--> 6

---

bench

bench.measure()

Returns the elapsed time to nanosecond precision as a human readable string. It may be used to do optimization of application code. For example here's how to measure how long some code in a application endpoint took to run.

elapsed = bench.measure(function ()
  -- do a couple things
    end)
return elapsed   -- returns, e.g., "122.145329ms"

bench.measure() will also return any parameters the function returns before the elapsed time. For example:

a, b, elapsed = bench.measure(function ()
  return "foo", 2 -- you can return how ever many values you want, adjust the assignment accordingly
  end)
print({a=a, b=b, elapsed=elapsed}) -- results in printing `map[a:foo,b:2,elapsed:44.406µs]`

---

sync_call

This function enables a simple strategy for preventing the competition of critical section. A synchronized section will ensure no concurrent execution of that code are executed. An optional timeout (millisecond) argument is available to define how long to wait for previous execution of that section to be completed. (10000 ms by default and 30000ms maximum)

The signature is sync_call(lockId, [timeout,] function[, args, ..])

If there are no errors, sync_call returns true, plus any values returned by the call. Otherwise, it returns false, plus the error message table.

function transaction (key, name)
  local data = from_json(Keystore.get{key=key}.value)
  data.name = name
  Keystore.set{key = key, value = to_json(data)}
end

local arg1 = "userdata_123"
local arg2 = "Bortus"

local status, result_or_err = sync_call("locker_id", 1000, transaction, arg1, arg2)

---

context

Provides execution context informations:

print(context) -- Map containing the context information

Available keys * solution_id - The Id of the solution context of the script execution * service - The service Id triggering the execution * service_type - The type of triggering service. One of: core, West Connectivity, exchange, product, solution * script_key - The triggering service custom Key. Used as alias when the service Id is a generated string * event - Event name * caller_id - In case of solution-to-solution communication, the caller solution Id

---

log

This set of functions provides helpers to emit logs by severity level. Every severity function's behavior is similar are the print function.

log.emergency(msg1, msg2...) -- severity=0
log.alert(msg1, msg2...) -- severity=1
log.critical(msg1, msg2...) -- severity=2
log.error(msg1, msg2...) -- severity=3
log.warn(msg1, msg2...) -- severity=4
log.notice(msg1, msg2...) -- severity=5
log.info(msg1, msg2...) -- severity=6
log.debug(msg1, msg2...) -- severity=7
print(msg1, msg2...) -- severity=6

---

system

Namespace for system specific functions. Introduces two ways to access services from West Connectivity scripting to allow dynamic service call.

system.services

local service = "config"
local operation = "listService"

system.services[service][operation]({type = "core"})

-- or

system.service_call(service, operation, {type = "core"})

system.runtime_stats()

Get runtime metrics from current execution. This can be useful for development/debugging to see how close a script is to hit the CPU / Memory limits at any point. For more information about the limits, see Script Environment.

local stats = system.runtime_stats()
-- CPU time so far in microseconds.
local cpu_time = stats.cpu_time
-- Processing time so far in microseconds.
-- Processing time is aggregated by solution as metric for usage billing.
local processing_time = stats.processing_time
-- Current memory usage in bytes.
-- This includes all script variables and temporary objects not yet garbage collected.
local memory_usage = stats.memory_usage
-- Service call time so far in microseconds.
local service_call_time = stats.service_call_time
-- Script execution time so far in microseconds.
local clock_time = stats.clock_time

return to_json(stats) -- Return runtime stats as a json object.

Developers could also retrieve detailed service call time by adding service_calls parameter to system.runtime_stats().

Keystore.list()
Keystore.get({key="QAQ"})
local stats = system.runtime_stats({service_calls=true})

-- Service call count for Keystore.list
local listCount = stats["service_calls"]["Keystore.list"]["count"]
-- Total processing time for Keystore.list in microseconds.
local listTime = stats["service_calls"]["Keystore.list"]["time"]
-- Count of return code 200 for Keystore.list
-- Remark: 200 might not exist in stats["service_calls"] if Keystore.list failed.
local code200 = stats["service_calls"]["Keystore.list"]["200"]

-- Return runtime stats including statistics for service calls as a json object.
return to_json(stats)

---

crypto

For crypto services like base64 encoding or validating chain certificates there is a crypto module available. You need to import it as a classic module with require("mcrypto"). All functions are listed here:

Function	Arguments type needed
hash	data string, hashType int16
hmac	message string, key string, hashType int16
hmac_validate	message string, messageMAC string, key string, hashType int16
b64url_encode	data string
b64url_decode	data string
rand_bytes	data int16
verify_cert_chain	certificates array (starting from top hierarchy) [string]
encrypt_aes	message string, key string(32 bytes)
decrypt_aes	encryptedMessage string(hex), key string(32 bytes)
encrypt_rsa	message string, publicKey string, hashType int16
decrypt_rsa	encryptedMessage string(hex), privateKey string, hashType int16
sign_rsa	privateKey string, hash string, hashType int16
validate_sign_rsa	signature string(hex), publicKey string, hash string, hashType int16

Note

• the hashType need to be either 1 or 2, which respectively refers to SHA256 or SHA512.
• generated encrypted values and signature are in hexadecimal string format.
• decrypt_aes & encrypt_aes use only GCM operation mode for now. When decoding, make sure your data has been generated with same mode.
• sign_rsa & validate_sign_rsa uses both PSS padding which is less vulnerable than other implemention such as PKCS1 v1.5.
• Message to encrypt in encrypt_rsa must be no longer than the length of the public modulus minus twice the hash length, minus a further 2. source.

example

local crypto = require("mcrypto")
local string_to_encode = "This is a table"

return crypto.b64url_encode(string_to_encode)

---

platform.datetime

platform.datetime.parse

Converts a datetime string into timestamp This function uses the customized layout, datetime string, and timezone string to generate a corresponding timestamp.

For the layout format, datetime function follows golang native time library.

const (
    _                        = iota
    stdLongMonth             = iota + stdNeedDate  // "January"
    stdMonth                                       // "Jan"
    stdNumMonth                                    // "1"
    stdZeroMonth                                   // "01"
    stdLongWeekDay                                 // "Monday"
    stdWeekDay                                     // "Mon"
    stdDay                                         // "2"
    stdUnderDay                                    // "_2"
    stdZeroDay                                     // "02"
    stdUnderYearDay                                // "__2"
    stdZeroYearDay                                 // "002"
    stdHour                  = iota + stdNeedClock // "15"
    stdHour12                                      // "3"
    stdZeroHour12                                  // "03"
    stdMinute                                      // "4"
    stdZeroMinute                                  // "04"
    stdSecond                                      // "5"
    stdZeroSecond                                  // "05"
    stdLongYear              = iota + stdNeedDate  // "2006"
    stdYear                                        // "06"
    stdPM                    = iota + stdNeedClock // "PM"
    stdpm                                          // "pm"
    stdTZ                    = iota                // "MST"
    stdISO8601TZ                                   // "Z0700"  // prints Z for UTC
    stdISO8601SecondsTZ                            // "Z070000"
    stdISO8601ShortTZ                              // "Z07"
    stdISO8601ColonTZ                              // "Z07:00" // prints Z for UTC
    stdISO8601ColonSecondsTZ                       // "Z07:00:00"
    stdNumTZ                                       // "-0700"  // always numeric
    stdNumSecondsTz                                // "-070000"
    stdNumShortTZ                                  // "-07"    // always numeric
    stdNumColonTZ                                  // "-07:00" // always numeric
    stdNumColonSecondsTZ                           // "-07:00:00"
    stdFracSecond0                                 // ".0", ".00", ... , trailing zeros included
    stdFracSecond9                                 // ".9", ".99", ..., trailing zeros omitted

    stdNeedDate       = 1 << 8             // need month, day, year
    stdNeedClock      = 2 << 8             // need hour, minute, second
    stdArgShift       = 16                 // extra argument in high bits, above low stdArgShift
    stdSeparatorShift = 28                 // extra argument in high 4 bits for fractional second separators
    stdMask           = 1<<stdArgShift - 1 // mask out argument
)

For the customized layout to parse specific datetime string, such as '2022-10-01T23:12:34.123', reference the constants above. Use 2006 for 4 digit year(YYYY), 01 for month, 02 for date, 15 for 24-hour-format hours, 04 for 2 minute, 05 for seconds and so on. The layout should be similar to '2006-01-02T15:04:05.999'. The layout must be customized to parse the input datetime string. For example, if the input has - or : symbols, the layout should also have these symbols.

For the location string, please refer to timezone database name, such as Asia/Taipei

This function is multi-return, the first value being the result, and the second value being an error value. If the error value is nil, then the conversion was successful, and the result can be used safely.

  local datetime = require("platform.datetime")
  local res, err = datetime.parse('2006-01-02T15:04:05.999', '2022-10-01T23:12:34.123', 'America/New_York')
  if err ~= nil then
    print(err)
  end
  print(res)

If non-nil, the error value will be a string.

local datetime = require("platform.datetime")
local res, err = datetime.parse('2006-01-02T15:04:05.999', '2022-10-01T23:12:34.123', 'America/New_Yorker')
if err ~= nil then
  print(err)
end
--> "unknown time zone America/New_Yorker"

platform.datetime.format

Converts a timestamp into customized datetime string This function uses the customized layout, timestamp long integer, and timezone string to generate a corresponding timestamp. The parameters are similar to platform.datetime.parse, the only difference in on the second timestamp parameter, the expected timestamp should be in milliseconds.

This function is multi-return, the first value being the result, and the second value being an error value. If the error value is nil, then the conversion was successful, and the result can be used safely.

    local datetime = require("platform.datetime")
    local res, err = datetime.format('2006-01-02', 1650611617000, 'Asia/Taipei')
  if err ~= nil then
    print(err)
  end
  print(res)

If non-nil, the error value will be a string.

Hot VM

West Connectivity provides Hot Virtual Machine feature which enables re-using existing scripting environment.

What it is?

When enable, Scripting VMs are kept alive for later re-use When disabled, each script execution (= each event) creates a new VM. Hot VMs are kept in a pool by solution id.

Next event will take any available VMs from the pool or create new one if none are ready.

Why using it?

• Speed up by reducing execution time & latency.
• No need of VM creation at each script.
• No need of loading the solution script in VM every time.
• Cost down by keeping state & avoiding unnecessary data access.
• Global variables don’t die and can be re-used in next event.

How to enable it?

Every solution can enable this feature from West Connectivity UI by checking the reuse_vm setting under the solution->Services->Config settings.

Why not always ON? Existing solutions could face un-expected behavior.

What to mind?

1. Modules are loaded only 1 time at VMs start! Make sure you don’t expect it to be run at each event execution.
2. Beware of global variables leaking. To avoid Out-of-Memory issues, VMs are not re-usable if they reach 50% of the allowed memory.
3. The VM & cache values won’t be there all the time! Always check if the value is in cache or not.
4. Cache invalidation a hard problem. Make sure the cached values don’t change.
5. Your application will have many VMs in a point in time and is not a shared state.

Example

cachedvalue = cachedvalue or Keystore.get({key="value"}).value