Test rule provides a way to perform test initialization/disposing and a way to modify test behaviour.

Each rule is a top-level value or class attribute which satisfies some of the following interfaces:

  1. SuiteRule performs initialization or disposing respectively before or after execution of all tests in the scope (package for top-level value or class for attribute).
  2. TestRule performs initialization or disposing respectively before or after execution of each test in the scope (package for top-level value or class for attribute).
  3. TestStatement is indended to add success or failure report to the test results and is evaluated after each test is completed.

In order to apply a rule a top-level value or class attribute satisfied some of the rule interfaces has to be declared and marked with testRule() annotation.

Notes:

Example:

    testRule object myRule satisfies TestRule & SuiteRule & TestStatement {
        shared actual default void after(AsyncPrePostContext context) {
            context.proceed();
        }

        shared actual default void before(AsyncPrePostContext context) {
            context.proceed();
        }

        shared actual default void dispose(AsyncPrePostContext context) {
            context.proceed();
        }

        shared actual default void initialize(AsyncPrePostContext context) {
            context.proceed();
        }

        shared actual void apply(AsyncTestContext context) {
            context.complete();
        }
    }

    test async testFunction1(AsyncTestContext context) => ...
    test async testFunction2(AsyncTestContext context) => ...



    class MyTestClass() {
        shared testRule object myClassRule satisfies TestRule {
            shared actual void after(AsyncPrePostContext context) {
                context.proceed();
            }

            shared actual void before(AsyncPrePostContext context) {
                context.proceed();
            }
        }

        test async testMethod1(AsyncTestContext context) => ...
        test async testMethod2(AsyncTestContext context) => ...
    }

In the above example myRule is evaluated before / after execution of top-level function testFunction1 and testFunction2 while MyTestClass.myClassRule is evaluated before / after execution of methods testMethod1 and testMethod2.

Build-in test rules:

see, the package members.

Why every build-in test rule implements non-default rule methods? Since each method calls AsyncPrePostContext.proceed() or AsyncPrePostContext.abort() which complete initialization / disposing. Delegation should be used instead of extending.

Reminding: annotate the rule top-level value or attribute with testRule()!
Test statement may be applied to a given test function only (and not applied to the rest test functions in the scope):

  1. The statement top-level value or attribute should not be annotated with testRule().
  2. Test function should be annotated with applyStatement().
By: Lis
Since 0.6.0
Annotations
applyStatementshared ApplyStatementAnnotation applyStatement(ValueDeclaration* statements)

Indicates that the given test statements have to be applied to the annotated test function.
Each value declaration from statements list has to satisfy TestStatement interface.
The annotation may occur multiple times on the given function.

If statement from the given list is marked with testRule() annotation it will be executed twice!

Parameters:
  • statements

    Statement declarations. Each item has to satisfy TestStatement interface.

By: Lis
Since 0.7.0
testRuleshared TestRuleAnnotation testRule()

Indicates that the annotated value or attribute identifies a test rule.
The value declaration has to satisfy SuiteRule, TestRule or TestStatement interfaces.

By: Lis
Since 0.6.0
ApplyStatementAnnotationshared final ApplyStatementAnnotation

Indicates that the given test statements have to be applied to the annotated test function.
Each value declaration from ApplyStatementAnnotation.statements list has to satisfy TestStatement interface.
The annotation may occur multiple times on the given function.

If statement from the given list is marked with testRule() annotation it will be executed twice!

TestRuleAnnotationshared final TestRuleAnnotation

Indicates that the annotated value or attribute identifies a test rule.
The value declaration has to satisfy SuiteRule, TestRule or TestStatement interfaces.

Interfaces
SuiteRuleshared SuiteRule

Base interface for suite rule, which is initialized or disposed before or after execution of all tests in the scope (package for top-level value or class for attribute).

Usage: Declare attribute or top-level value satisfies the interface and mark it with testRule() annotation.

TestRuleshared TestRule

Base interface for test rule, which is initialized or disposed before or after execution of each test in the scope (package for top-level value or class for attribute).

Usage: Declare attribute or top-level value satisfies the interface and mark it with testRule() annotation.

TestStatementshared TestStatement

Base interface for test statement, which is applied after execution of each test in the scope (package for top-level value or class for attribute) and may report additional messages to the test results using AsyncTestContext submitted to TestStatement.apply().

Usage: Declare attribute or top-level value satisfies the interface and mark it with testRule() annotation.

Classes
AtomicValueRuleshared AtomicValueRule<Element>

Atomic reference on some value which is re-initialized to initial value each time the test is started.
If Element is mutable be careful with proper cleaning after the test - factory function is prefered in this case.

ContextualRuleshared ContextualRule<Element>

Test rule which stores values local to the current thread of execution meaning that each thread or process that accesses these values get to see their own copy.

The rule re-sets the stored value to the initial before each test.

The rule is like to ceylon.language.Contextual:

ContextualRule<Integer> intValue = ContextualRule<Integer>(10);

try (intValue.Using(100)) {
    value current = intValue.get();
}

Each time when Using.obtain is called the current value is stored and refreshed with newValue.
The stored value is retrieved with calling Using.release from common stack for all Using instances.

ContextualRule.get() may return value before ContextualRule.Using evaluation!

If Element is mutable be careful with proper cleaning after the test - factory function is prefered in this case.

CounterRuleshared CounterRule

Atomically counts something. Resets to initial value before each test.

CurrentTestStoreshared CurrentTestStore<Element>

The rule which returns an element only if it is requested from the same test which initializes the rule.
Otherwise TestAccessDenied is thrown.
Actually, a thread interrupted by timeout or by something else may still be alive and may modify a rule.
This way improperly completed test may modify the currently executed test data.
CurrentTestStore rule is indended to prevent such behaviour throwing TestAccessDenied exception when not owner tries to get stored element.

Owner here is the test for which initialization has been performed.
The stored value is reseted by extracted from source value each time the new test is started.

GaugeStatementshared GaugeStatement<Element>

Statement which verifies an element against given matcher each time when gauged, i.e. when GaugeStatement.gauge() method is called.
If matcher rejects verification the fail report is added to the final test report.

LockAccessRuleshared LockAccessRule<Element>

Tool for controlling access to a shared resource of Element type by multiple threads.

The resource value is re-initialized to initial before each test.

To acquire lock and get access to resource use nested class Lock which satisfies ceylon.language::Obtainable interface and is intented to be used within try block:

    shared testRule LockAccessRule<Element> lockRule = LockAccessRule<Element>(elem);
    ...
    value lock = lockRule.Lock();
    try(lock) {
        lock.element = modifyElement(lock.element);
    }
    ...
    try(lock) {
        lock.element = anotherModifyElement(lock.element);
    }

In order to acquire lock obtain has to be called.
In order to release lock release has to be called.
Actual value is stored in the resource only when release is called.

If Element is mutable be careful with proper cleaning after the test - factory function is prefered in this case.

MeterRuleshared MeterRule

Lock-free and thread-safely collects statistic data on an execution time and on a rate (per second) at which a set of events occur.
Statistic data is reset before each test.
To start recording call MeterRule.start(). To record time delta call MeterRule.tick() which records time delta from start or previous tick and up to now.

Example:

    MeterRule meterRule = MeterRule();
    ...
    void benchTest() {
        meterRule.start();
        for (repeat in 0..repeatCounts) {
            doOperation();
            meterRule.tick();
        }
        print(meterRule.timeStatistic);
        print(meterRule.rateStatistic);
    }
ResourceRuleshared ResourceRule

A file packaged within a module and loaded before all tests started.
See ceylon.language::Resource for details.

SignalRuleshared SignalRule

Provides suspend / resume capability. The rule might be applied when some thread has to await another one completes some calculations.

Example:

    class MyTest() {
        SignalRule rule = SignalRule();

        test async void myTest(AsyncTestContext context) {
            object thread1 extends Thread() {
                shared actual void run() {
                    rule.await(); // awaits until thread2 signals
                    ... verify thread 2 work results ...
                    context.complete();
                }
            }
            thread1.start();

            object thread2 extends Thread() {
                shared actual void run() {
                    ... do some work ...
                    rule.signal(); // wakes up thread1
                }
            }
            thread2.start();
        }
    }
StatisticRuleshared StatisticRule

Lock-free and thread-safely accumulates statistics data of some variate values.
Doesn't collect values, just accumulates statistic data when sample added - see StatisticRule.sample() and StatisticRule.samples().
Statistic data is reseted before each test.

SuiteRuleChainshared SuiteRuleChain

Applies suite rules in the given order. Initialization (i.e. initialize methods) is performed with iterating of the rules in direct order, while diposing (i.e. dispose methods) is performed in reverse order. So, the first initialized is the last disposed.

If submited rule is marked with testRule() annotation it will be executed twice.

TemporaryDirectoryRuleshared TemporaryDirectoryRule

Creates new temporary directory before each test and destroyes it after.

TemporaryFileRuleshared TemporaryFileRule

Creates new temporary file before each test and destroyes it after.

TestRuleChainshared TestRuleChain

Applies test rules in the given order. Initialization (i.e. before methods) is performed with iterating of the rules in direct order, while diposing (i.e. after methods) is performed in reverse order. So, the first initialized is the last disposed.

If submited rule is marked with testRule() annotation it will be executed twice.

TestStatementChainshared TestStatementChain

Applies test statements in the given order.

If submited rule is marked with testRule() annotation it will be executed twice.

Verifiershared Verifier<Element>

Verifies if element given by source matches to specified matcher.
Actually source is called when the verifier is evaluated i.e. after each test. This is main difference to VerifyRule which evaluates source before each test.

VerifyRuleshared VerifyRule<Element>

Verifies if the stored value matches a given matcher when the statement is applied The initial value is evaluated before each test. This is main difference with Verifier which evaluates value after each test. .

Exceptions
TestAccessDeniedshared TestAccessDenied

Exception thrown when access to a value is requested from a test which doesn't own the value.