Skip to main content

Test Tool

Intro

The KCL Test tool and testing package provide a simple testing framework to test KCL code. All KCL files in each directory are a set of tests, and each schema starts with Test in each test.k is a test case.

How to use

There is a KCL file hello.k:

schema Person:
name: str = "kcl"
age: int = 1

hello = Person {
name = "hello kcl"
age = 102
}

Build a test file hello_test.k:

schema TestPerson:
a = Person{}
assert a.name == 'kcl'

schema TestPerson_age:
a = Person{}
assert a.age == 1

schema TestPerson_ok:
a = Person{}
assert a.name == "kcl"
assert a.age == 1

Execute the following command:

$ kcl-test
ok /pkg/to/app [365.154142ms]
$

Failed Test Case

Modify hello_test.k to the following code to build failed test case:

# Copyright 2021 The KCL Authors. All rights reserved.

import testing

schema TestPerson:
a = Person{}
assert a.name == 'kcl2'

schema TestPerson_age:
a = Person{}
assert a.age == 123

schema TestPerson_ok:
a = Person{}
assert a.name == "kcl2"
assert a.age == 1

Output:

$ kcl-test
FAIL /pkg/to/app [354.153775ms]
---- <TestPerson> failed [48.817552ms]
KCL Runtime Error: File /pkg/to/app/hello_test.k:7:
assert a.name == 'kcl2'
Assertion failure
---- <TestPerson_age> failed [47.515009ms]
KCL Runtime Error: File /pkg/to/app/hello_test.k:11:
assert a.age == 123
Assertion failure
---- <TestPerson_ok> failed [47.26677ms]
KCL Runtime Error: File /pkg/to/app/hello_test.k:15:
assert a.name == "kcl2"
Assertion failure
$

Option Args

Literal type command line arguments can be specified via the testing package:

schema TestOptions:
testing.arguments("name", "ktest")
testing.arguments("age", "123")
testing.arguments("int0", 0)
testing.arguments("float0", 0.0)
testing.arguments("bool-true", True)
testing.arguments("bool-false", False)

name = option("name")
assert name == "ktest"

age = option("age")
assert age == 123

assert option("int0") == 0
assert option("float0") == 0.0
assert option("bool-true") == True
assert option("bool-false") == False

testing.arguments defines a set of key-value arguments, valid only in the current test.

The option arguments can also be loaded from the settings.yaml. There is a file settings.yaml:

  - key: app-name
value: app
- key: env-type
value: prod
- key: image
value: reg.docker.inc.com/test-image

Parameters can then be configured via testing.setting_file("./settings.yaml"). At the same time, testing.arguments() is still supported to override the parameters in the configuration file:

schema TestOptions_setting:
testing.setting_file("./settings.yaml")
testing.arguments("file", "settings.yaml")

assert option("app-name") == "app"
assert option("file") == "settings.yaml"

Plugin Test

Automatically switch to plugin mode if the directory to be tested contains plugin.py and test files. Then set the environment variable KCL_PLUGINS_ROOT before the test (plugins in other directories can no longer be accessed) to test the current plugin, and restore the previous KCL_PLUGINS_ROOT after the test is completed.

Integration Test

Automatically execute integration tests when the directory contains *.k. If there is stdout.golden then verify the output. If there is stderr.golden then verify the error. Supports the settings.yaml file to define command line arguments.

Batch Test

  • kcl-test path Execute the test of the specified directory. It can be omitted if it's the same directory that the command is executed
  • kcl-test -run=regexp Execute the test which matches patterns
  • kcl-test ./... Execute unit tests recursively

Args

$ kcl-test -h
NAME:
kcl-go test - test packages

USAGE:
kcl-go test [command options] [packages]

OPTIONS:
--run value Run only those tests matching the regular expression.
--quiet, -q Set quiet mode (default: false)
--verbose, -v Log all tests as they are run (default: false)
--debug, -d Run in debug mode (for developers only) (default: false)
--help, -h show help (default: false)