The ceylon.test
module is a simple framework to write repeatable tests.
Tests execute the code of the module under test and can make assertions about what it does. For example,
Tests can be written as top level functions …
test void shouldAlwaysSucceed() {}
… or organized inside classes.
class YodaTest() { test void shouldBeJedi() { assert(yoda is Jedi); } test void shouldHavePower() { assert(yoda.midichloriansCount > 1k); } }
Notice the test annotation, which helps the framework to automatically discover tests.
The most convenient way how to run tests is to use IDE integration
or via command line tools ceylon test
and ceylon test-js
.
$ceylon test com.acme.mymodule
Tests can be also run programmatically, via interface TestRunner
and its factory method createTestRunner()
, but this API is usually
not necessary to use directly.
Assertions can be evaluated by using the language's assert
statement
or with the various assert...
functions, for example:
assert(is Hobbit frodo); assert(exists ring); assertNotEquals(frodo, sauron); assertThatException(() => gandalf.castLightnings()).hasType(`NotEnoughMagicPowerException`);
A test function which completes without propagating an exception is
classified as a success. A test function which propagates
an AssertionError
is classified as a failure. A test
function which propagates any other type of Exception
is classified as
an error.
Common initialization logic can be placed into separate functions, which run before or after each test.
class StarshipTest() { beforeTest void init() => starship.chargePhasers(); afterTest void dispose() => starship.shutdownSystems();
Or it is possible to execute custom code, which will be executed only once before or after whole test run. Such callbacks can be only top level functions.
beforeTestRun void createUniverse() { ... } afterTestRun void destroyUniverse() { ... }
Other options how to hook into tests execution, is to implement TestListener
and react on concrete events. Or if you have to go deeper, there are several
TestExtension
points.
Sometimes you want to temporarily disable a test or a group of tests, this can be done via the ignore annotation.
test ignore("still not implemented") void shouldBeFasterThanLight() { ... }
Sometimes the conditions, if the test can be reliable executed,
are know only in runtime, in that case one of the assume...
functions
can be used.
test void shouldBeFasterThanLight() { assumeTrue(isWarpDriveAvailable); ... }
Tests or its containers can be tagged with one or more tags. Those tags can later be used to filter which tests will be executed.
For example test, which is failing often, but from unknow reasons, can be marked as unstable …
test tag("unstable") shared void shouldSucceedWithLittleLuck() { ... }
… and then excluded from test execution
$ceylon test --tag=!unstable com.acme.mymodule
… or visa versa, we can execute only tests with this tag
$ceylon test --tag=unstable com.acme.mymodule
Test execution can be extended or completely changed by several extension points.
All extension points are child of marker interface TestExtension
and here is a list of currently available extension points:
TestListener
TestInstanceProvider
TestInstancePostProcessor
TestVariantProvider
ArgumentListResolver
Extensions can be registered declaratively on several places: on concreate test, on class which contains tests, on whole package or even module, with help of testExtension annotation, for example:
testExtension(`class DependencyInjectionInstancePostProcessor`, `class TransactionTestListener`) package com.acme;
It is possible to write parameterized tests. The responsibility for resolving argument lists
is on ArgumentListResolver
. It's default implementation find annotation
which satisfy ArgumentListProvider
or ArgumentProvider
interface,
collect values from them and prepare all possible combination.
Developers can easily implement their own argument providers, currently there exists basic implementation, parameters annotation.
Example:
shared {[Integer, Integer]*} fibonnaciNumbers => {[1, 1], [2, 1], [3, 2], [4, 3], [5, 5], [6, 8] ...}; test parameters(`value fibonnaciNumbers`) shared void shouldCalculateFibonacciNumber(Integer input, Integer result) { assert(fibonacciNumber(input) == result); }
Packages | |
ceylon.test | Contains API of the test module. |
ceylon.test.annotation | Contains annotations types of the test module. |
ceylon.test.engine | Contains test engine. |
ceylon.test.engine.spi | Contains SPI of test module. |
ceylon.test.event | Contains events of the test module. |
ceylon.test.reporter | Contains reporting tools. |
Dependencies | ||
ceylon.collection | 1.3.1 | |
ceylon.file (jvm) | 1.3.1 | |
ceylon.runtime (jvm) | 1.3.1 | |
java.base (jvm) | 7 | |
org.jboss.modules (jvm) | 1.4.4.Final |
Contains API of the test module.
Aliases | |
TestComparator | Source Codeshared TestComparator=> Comparison(TestDescription, TestDescription) Alias for functions which compare two tests, used for sorting tests in test plan. |
TestFilter | Source Codeshared TestFilter=> Boolean(TestDescription) Alias for functions which filter tests. Should return true if the given test should be run, or false if it should be excluded. |
TestSource | Source Codeshared TestSource=> Module|Package|ClassDeclaration|FunctionDeclaration|Class<Anything,Nothing>|FunctionModel<Anything,Nothing>|String Alias for program elements which can be used as a source for discovering tests. |
Annotations | |
afterTest | Source Codeshared AfterTestAnnotation afterTest() Marks a function which will be run after each test in its scope. It allow to place common initialization logic into separate place. class StarshipTest() { beforeTest void init() => starship.chargePhasers(); afterTest void dispose() => starship.shutdownSystems(); |
afterTestRun | Source Codeshared AfterTestRunAnnotation afterTestRun() Marks a toplevel function which will be executed once after all tests, after the test run is finished. afterTestRun void stopEmbeddedDatabase() { ... } |
beforeTest | Source Codeshared BeforeTestAnnotation beforeTest() Marks a function which will be run before each test in its scope. It allow to place common initialization logic into separate place. class StarshipTest() { beforeTest void init() => starship.chargePhasers(); afterTest void dispose() => starship.shutdownSystems(); |
beforeTestRun | Source Codeshared BeforeTestRunAnnotation beforeTestRun() Marks a toplevel function which will be executed once before all tests, before the test run starts. beforeTestRun void startEmbeddedDatabase() { ... } |
ignore | Source Codeshared IgnoreAnnotation ignore(String reason = "") Marks a test or group of tests which should not be executed, which will be skipped during test run. It can be set on several places: on concrete test, on class which contains tests, on whole package or even module. test ignore("still not implemented") void shouldBeFasterThanLight() { Parameters:
|
parameters | Source Codeshared ParametersAnnotation parameters(FunctionOrValueDeclaration source) Annotations to specify source of argument values for parameterized tests, can be used for whole function or individually for each parameter. As a source, can be used toplevel value or function, which type is compatible with parameters of test function. A test function may have multiple parameters, each with own value source. The test engine will execute it for each combination of provided values. For example, a function with one parameter whose argument provider yields two values, and second parameter whose argument provider yields three values, will be executed six times. Example: shared {[Integer, Integer]*} fibonnaciNumbers => {[1, 1], [2, 1], [3, 2], [4, 3], [5, 5], [6, 8] ...}; test parameters(`value fibonnaciNumbers`) shared void shouldCalculateFibonacciNumber(Integer input, Integer result) { assert(fibonacciNumber(input) == result); } Parameters:
|
tag | Source Codeshared TagAnnotation tag([String+] tags) Marks a test or group of tests with one or more tags, tags can be used for filtering, which tests will be executed. For example test, which is failing often, but from unknow reasons, can be marked as unstable … test tag("unstable") shared void shouldSucceedWithLittleLuck() { ... } … and then excluded from test execution $ceylon test --tag=!unstable com.acme.mymodule … or visa versa, we can execute only tests with this tag $ceylon test --tag=unstable com.acme.mymodule Parameters:
|
test | Source Codeshared TestAnnotation test() Marks a function as being a test. Example of simplest test: test shared void shouldAlwaysSucceed() {} |
testExecutor | Source Codeshared TestExecutorAnnotation testExecutor(ClassDeclaration executor) Annotation to specify custom testExecutor(`class ArquillianTestExecutor`) package com.acme; Parameters:
|
testExtension | Source Codeshared TestExtensionAnnotation testExtension([ClassDeclaration+] extensions) Annotation to specify various testExtension(`class DependencyInjectionInstancePostProcessor`, `class TransactionTestListener`) package com.acme; Parameters:
|
testSuite | Source Codeshared TestSuiteAnnotation testSuite({Declaration+} sources) Annotation to specify test suite, which allow combine several tests or test suites and run them together. testSuite({`class YodaTest`, `class DarthVaderTest`, `function starOfDeathTestSuite`}) shared void starwarsTestSuite() {} Parameters:
|
Functions | |
assertAll | Source Codeshared void assertAll(Anything()[] assertions, String? message = null) Verify all given assertions and any failures will be reported together. Example: assertAll([ () => assertEquals(agent.id, "007"), () => assertEquals(agent.firstName, "James"), () => assertEquals(agent.lastName, "Bond")]); Parameters:
Throws
|
assertEquals | Source Codeshared void assertEquals(Anything actual, Anything expected, String? message = null, Boolean compare(Anything val1, Anything val2) = ...) Fails the test if the given values are not equal according to the given compare function. Parameters: Throws
|
assertFalse | Source Codeshared void assertFalse(Boolean condition, String? message = null) Fails the test if the condition is true. Parameters:
Throws
|
assertNotEquals | Source Codeshared void assertNotEquals(Anything actual, Anything unexpected, String? message = null, Boolean compare(Anything val1, Anything val2) = ...) Fails the test if the given values are equal according to the given compare function. Parameters: Throws
|
assertNotNull | Source Codeshared void assertNotNull(Anything val, String? message = null) Fails the test if the given value is null. Parameters:
Throws
|
assertNull | Source Codeshared void assertNull(Anything val, String? message = null) Fails the test if the given value is not null. Parameters:
Throws
|
assertThatException | Source Codeshared ExceptionAssert assertThatException(Throwable|Anything() exceptionSource) Fails the test if expected exception isn't thrown. Example: assertThatException(() => gandalf.castLightnings()).hasType(`NotEnoughMagicPowerException`); Parameters:
Throws
|
assertTrue | Source Codeshared void assertTrue(Boolean condition, String? message = null) Fails the test if the condition is false. Parameters:
Throws
|
assumeFalse | Source Codeshared void assumeFalse(Boolean condition, String? message = null) Abort test execution if the assumption condition is true. Example: test shared void shouldUseNetwork() { assumeFalse(isDisconnected); ... } Parameters:
Throws
|
assumeTrue | Source Codeshared void assumeTrue(Boolean condition, String? message = null) Abort test execution if the assumption condition is false. Example: test shared void shouldUseDatabase() { assumeTrue(isDatabaseAvailable); ... } Parameters:
Throws
|
createTestRunner | Source Codeshared TestRunner createTestRunner(TestSource[] sources, TestExtension[] extensions = [], TestFilter filter = ..., TestComparator comparator = ...) Create a new Parameters:
|
defaultTestComparator | Source Codeshared Comparison defaultTestComparator(TestDescription description1, TestDescription description2) Default test comparator sort tests alphabetically. |
defaultTestFilter | Source Codeshared Boolean defaultTestFilter(TestDescription description) Default test filter, always return true. |
fail | Source Codeshared void fail(String? message = null) Throws an Parameters:
Throws
|
runTestTool | Source Codeshared void runTestTool() Run function used by |
Interfaces | |
TestListener | Source Codeshared TestListener Represents a listener which will be notified about events that occur during a test run. Example of simple listener, which triggers alarm whenever test fails. shared class RingingListener() satisfies TestListener { shared actual void testError(TestErrorEvent event) => alarm.ring(); } … such listener can be used directly when creating TestRunner runner = createTestRunner{ sources = [`module com.acme`]; extensions = [RingingListener()];}; … or better declaratively with usage of testExtension(`class RingingListener`) module com.acme; |
TestRunResult | Source Codeshared TestRunResult Represents a summary result of the test run. |
TestRunner | Source Codeshared TestRunner Represents a facade for running tests. Instances are usually created via the |
Classes | |
ExceptionAssert | Source Codeshared ExceptionAssert An assertions applicable to exceptions, see |
TestDescription | Source Codeshared TestDescription Describes a test, or a group of tests, can be arranged in a tree. |
TestResult | Source Codeshared TestResult Represents a detailed result of the execution of a particular test. |
TestState | Source Codeshared TestState The result state of test execution. |