Lab Testing

Running the tests

Testing in Logiclab is performed using coco labtest command. The command compiles a module to a manifest, creates a temporary environment for Logiclab, compiles the manifest and runs any test cases, specified in the .coco file.

Testing directives in the source code

Invoke a test and expect output

Test cases are specified using pairs of Logiclab commands and expected results like:

// < invoke TEST.MyFunction(name:"alice")
// > l: 5
endpoint invokable MyFunction(name String) -> (l U64):
    ...

Tests are run in a temporarily initialized Logiclab environment where the compiled logic is always called TEST, therefore we always invoke or deploy functions with invoke TEST.Function…

The tests can be written anywhere in the .coco file (though they are usually directly above functions for readability), but it’s mandatory they are always in pairs, every // < test_invocation must be followed by // > expected_outputs, if there’s no matching expected output for every input, the test will fail. If the test doesn’t produce any output (i.e. the function has no return parameters), it’s still necessary to write a directive that expects an empty return, i.e. // <

Expected outputs are pairs of return variables, e.g. x: 5 or names:[alice bob]. The order of return variables is not important, as long as the number of expected variables matches the actual output and all values are the same, the test will pass.

💡

Known limitation: we can’t test for output when a map is returned. Maps don’t have a deterministic order of elements, and labtest doesn’t implement expected maps yet. It also doesn’t implement “don’t care” values so we can’t yet write a test where we simply ignore some returned value. To test a function that returns a map, we can avoid this limitation by writing a test function, that only calls the function we want to test, but doesn’t return a map, e.g.:

// < invoke TEST.Helper(name:"alice")
// > l: 5
endpoint invokable Helper(name String) -> (l U64):
    m = make(Map[String]U64)
    l, m = (l, m) <- F(name)

// this is our actual function
func local F(name String) -> (l U64, m Map[String]U64):
    l = len(name)
    m[name] = 1

Testing for errors / exceptions

When we expect an exception, we use a // > ! exception_class::exception_error notation. We can specify only the exception class without a detailed error, but the class is mandatory. So both tests in the following example (taken from a complete example below) succeed:

// this test verifies exact match of exception (class::error)
// < invoke TEST.IncreaseNodeLimit!(kind: "wrong", delta: 1)
// > !string::invalid verification kind

// this test just checks for exception class without specific error string
// < invoke TEST.IncreaseNodeLimit!(kind: "wrong again", delta: 1)
// > !string
endpoint invokable IncreaseNodeLimit!(kind String, delta U64):
    ...

Tests modules, that include persistent state

Most modules include persistent state, Logiclab doesn’t allow invoking functions before the state is deployed. In such cases, our tests need to include the first test that is not “invoked” but “deployed”, like in the example below.

Example

The following is an example of a complete script that includes a persistent state, deployer, function that mutates the state and a reader that helps us check, if the state has the expected values after all the tests have run.

coco GuardianRegistry

state persistent:
    NodeLimitKYC U64
    NodeLimitKYB U64

// < deploy TEST.Init!()
// >
endpoint deployer Init!():
    mutate 0 -> GuardianRegistry.State.NodeLimitKYC

// < invoke TEST.IncreaseNodeLimit!(kind: "kyc", delta: 2)
// >
// < invoke TEST.IncreaseNodeLimit!(kind: "kyb", delta: 3)
// >
// < invoke TEST.IncreaseNodeLimit!(kind: "wrong", delta: 1)
// > !string::invalid verification kind
endpoint invokable IncreaseNodeLimit!(kind String, delta U64):
    if kind == "kyc":
        mutate delta -> GuardianRegistry.State.NodeLimitKYC

    elif kind == "kyb":
        mutate delta -> GuardianRegistry.State.NodeLimitKYB

    else:
        throw "invalid verification kind"

// < invoke TEST.Limits()
// > kyc: 2 kyb: 3
endpoint invokable Limits() -> (kyc U64, kyb U64):
    observe kyc <- GuardianRegistry.State.NodeLimitKYC
    observe kyb <- GuardianRegistry.State.NodeLimitKYB

The first “test” we run is a mandatory deploy statement that’s always needed when the module has a persistent state. Logiclab will fail to run any invoke statements before one of deployer functions are deployed, so to test a module with a state, a test with deploy statements needs to be present before any invoke tests. Even though the deployer Init! doesn’t return anything, we need to add an empty test so the test engine is certain that we get an empty output.

// < deploy TEST.Init!()
// >

The function IncreaseNodeLimit! that mutates the state is called three times during the test and changes the state (the end result we’re checking at the end of the test, as described above). As a successful call doesn’t return anything, the first two tests except an empty string. The third test on the other hands expects an error, so here we see we can run several tests on a single function.

At the end we invoke a function that helps us read the state and here we check whether the values in the persistent state are as expected:

// < invoke TEST.Limits()
// > kyc: 2 kyb: 3

We expect the execution output of two return variables, kyc and kyb with respective values. As the tests are run sequentially from the top to the bottom, the last test just verifies if the variables in the state are set as expected by the previous calls.

The above example has demonstrated the key issues with testing:

if we have a persistent state in our module, the first test must // < deploy a deployer
every // < test invocation needs to be followed by // > test of expected outputs, even if there are no outputs, we need to write an empty declaration
we can write as many tests as we want, if the tests mutate state, the results persist during the test
the tests are always executed sequentially from top to bottom
we can expect return values or exceptions