asyncTest

is an extension of SDK ceylon.test with following capabilities:

  • testing asynchronous multithread code
  • common initialization for a set of test functions
  • reporting several failures for a one particular test execution (test function)
  • marking each failure with String title
  • instantiating private classes and invoking private functions of tested module without additional dependecies in

The extension is based on:

  • AsyncTestContext interface which test function has to operate with (basically, reports on fails to).
  • AsyncTestExecutor class which satisfies ceylon.test.engine.spi::TestExecutor and used by ceylon.test module to execute test functions.
  • TestInitContext interface and init() annotation, which can be used for common initialization of test set.

test procedure

  1. Declare test function, which accepts AsyncTestContext as the first argument:
        test void doTesting(AsyncTestContext context) {...}
    
    The other arguments have to be in accordance with ceylon.test::parameters annotation or other satisfied ArgumentProvider. Mark test function with ceylon.test::test annotation.
  2. Code test function according to AsyncTestContext specification:
  3. Apply ceylon.test::testExecutor annotation:
    • at module level to execute every functions / classes marked with test in the given module
      testExecutor(`class AsyncTestExecutor`)
      native("jvm")
      module mymodule "1.0.0"
      
    • at function level to execute the given function only
      test testExecutor(`class AsyncTestExecutor`)
      void doTesting(AsyncTestContext context) {...}
      
  4. Run test in IDE or command line.

Also see some details in documentation on AsyncTestExecutor and AsyncTestContext.

initialization

Common asynchronous initialization for a set of test functions can be performed by marking these functions with init() annotation. Argument of this annotation is initializer function, which is called just once for all tests.
Initializer has to take first argument of TestInitContext type and other arguments as specified by parameters annotation if marked with.
When initialization is completed TestInitContext.proceed() has to be called.
If some error has occured and test has to be aborted, TestInitContext.abort() can be called.
Initializer can store some values on context using TestInitContext.put() method. These values can be retrieved lately by test functions using AsyncTestContext.get() or AsyncTestContext.getAll().
Alternatively init() annotation can be used at class, package or module level to apply initializer to all test functions of corresponding container.

Initializer may not be marked with init() annotation! Test function, package or module should be marked.

Example:

    // initialization parameters
    [String, Integer] serverParameters => ["host", 123]; 

    // initializer - binds to server specified by host:port,
    // if successful proceeds with test or aborted if some error occured
    parameters(`value serverParameters`)
    void setupServer(TestInitContext context, String host, Integer port) {
        Server server = Server();
        server.bind(host, port).onComplete (
            (Server server) {
                // storing server on context and notifying to continue with testing
                context.put("``host``:``port``", server, server.close); 
                context.proceed();
            },
            (Throwable err) {
                // abort initialization since server binding errored
                context.abort(err, "server ``host``:``port`` binding error");
            }
        );
    }

    // test functions, setupServer is called just once - nevertheless the actual number of test functions 
    test init(`function setupServer`) void firstTest AsyncTestContext context) {
        String serverName = serverParameters.host + ":``port``";
        assert ( exists server = context.get<Server>("serverName") );
        ...
    }
    test init(`function setupServer`) void secondTest(AsyncTestContext context) {
        String serverName = serverParameters.host + ":``port``";
        assert ( exists server = context.get<Server>("serverName") );
        ...
    }

instantiating private classes

See loadAndInstantiate()

invoking private functions

See loadTopLevelFunction()

Platform: Java
By: Lis
License: The MIT License (MIT) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Packages
herd.asynctest
Dependencies
ceylon.collection1.2.1
ceylon.runtime1.2.1
ceylon.test1.2.1
java.base8
org.jboss.modules1.4.4.Final
Annotations
initshared InitAnnotation init(FunctionDeclaration initializer)

Marks a test function (marked with test also) with initializer.
The initializer function has to take first argument of TestInitContext type and other arguments as specified by parameters annotation if marked with.
Initializer is called just a ones and all test functions marked with init annotation uses results of this initialization.
Annotation can be used at class, package or module level which forces to use this initializer for all functions of corresponding level. This behavior can be overriden by apply init directly to function.

Parameters:
  • initializer

    Function which performs initialization.

InitAnnotationshared final InitAnnotation

Annotation class for init().

Functions
loadAndInstantiateshared Result loadAndInstantiate<Result, Arguments>(String moduleName, String moduleVersion, String packageName, String className, Type<Anything>[] typeArguments, Arguments defaultConstructorArguments)
given Arguments satisfies Anything[]

Loads and instantiates shared or unshared class with name className from package packageName of module moduleName, with version moduleVersion.
The object is instantiated using default contructor.
The module has to be already loaded or has to be on your repo path.

Parameters:
  • moduleName

    name of the module which contains declaration

  • moduleVersion

    version of the module

  • packageName

    full name (including module name) of the package which contains declaration

  • className

    name of the declaration

  • typeArguments

    generic type arguments to be applied to declaration

  • defaultConstructorArguments

    arguments of the default constructor

Throws
By: Lis
See also loadTopLevelFunction(), defaultConstructor
loadTopLevelFunctionshared Return(*Arguments) loadTopLevelFunction<Return, Arguments>(String moduleName, String moduleVersion, String packageName, String functionName, Type<Anything>[] typeArguments)
given Arguments satisfies Anything[]

Loads shared or unshared function with name functionName from package packageName of module moduleName, with version moduleVersion.
The module has to be already loaded or has to be on your repo path.

Parameters:
  • moduleName

    name of the module which contains declaration

  • moduleVersion

    version of the module

  • packageName

    full name (including module name) of the package which contains declaration

  • functionName

    name of the loaded function

  • typeArguments

    generic type arguments to be applied to declaration

Throws
  • ModuleLoadingError

    Unable to load module.

  • PackageLoadingError

    Unable to find package.

  • TopLevelDeclarationLoadingError

    Unable to find class declaration.

  • IncompatibleTypeException

    If the specified Return or Arguments type arguments are not compatible with the actual declaration.

  • TypeApplicationException

    If the specified closed type argument values are not compatible with the actual result's type parameters.

By: Lis
See also loadAndInstantiate(), apply
Interfaces
AsyncTestContextshared AsyncTestContext

Represents object passed to test function and allows to interact with asynchronous test framework i.e. AsyncTestExecutor.

General test procedure within test function is:

  1. Notify test framework on test procedure starting.
  2. Perform the test itself. Notify test framework on fails. Several notifications are allowed. Each fail notification is represented as test variant.
  3. Notify test framework on test procedure completion, this step is nesseccary to continue testing with next execution.

Example of tested function:

test void doTesting(AsyncTestContext context) {
    // start testing
    context.start();

    // perform test procedure and notify about fails, if no fails notified test is considered successfull
    context.fail(Exception("exception"), "some exception");
    context.fail(AssertionError( "assert"), "some assert");
    context.abort(Exception( "exception"), "test aborted");
    context.assertTrue(true, "to be `false`");
    context.assertNotNull(null, "to be nonull");

    // complete testing
    context.complete("title which is added to test variant name only if test is succeeded");
}

Common initialization for a set of test functions can be performed using init() annotation and TestInitContext.

TestInitContextshared TestInitContext

Represents a context pushed to test initializer.
Initializer can fill in some values, which can be retrieved late from AsyncTestContext.

Classes
AsyncTestExecutorshared AsyncTestExecutor

Test executor.

capabilities

  • testing both classes and top level functions
  • parameterized testing with a set of function arguments, see parameters annotation
  • testing asynchronous multithread code (only one test function is run at the moment, main test thread waits when execution completes)
  • functions marked with afterTest and beforeTest annotations are executed synchronously. So put asynchrnous initialization logic into test function directly or perform initialization in separated thread and block tested thread while initialization completed
  • reporting several failures for a one particular test function (each failure is reported as test variant)
  • mark each failure with String title

In order to utilize this executor capabilities test function has to accept AsyncTestContext as the first argument:

    test void doTesting(AsyncTestContext context) {...}

The other arguments have to be in accordance with ceylon.test::parameters annotation.

running

To run the test using this executor testExecutor annotation to be applied:

  • at module level to apply to every functions / classes marked with test in the given module
    testExecutor(`class AsyncTestExecutor`)
    native("jvm")
    module mymodule "1.0.0"
    
  • at function level to apply to the given function only
    testExecutor(`class AsyncTestExecutor`)
    test void doTesting(AsyncTestContext context) {...}
    

Following procedure is as usual for SDK ceylon.test module - mark tested functions with test annotation and run test in IDE or command line.

test logic

There are four cases of tested function type:

  1. Anything(AsyncTestContext) - executed using this test executor.
  2. parameters() Anything(AsyncTestContext, ...) - executed using this test executor with several variants provided by parameters annotation.
  3. Anything() - executed using ceylon.test SDK DefaultTestExecutor.
  4. parameters() Anything(...) - executed using ceylon.test SDK DefaultTestExecutor with several variants provided by parameters annotation.

When test function taking AsyncTestContext as first argument is executed it is expected the function will do following steps:

  1. Initialization.
  2. Notifying test framework on test procedure starting - AsyncTestContext.start().
  3. Performing the test, reporting on fails via AsyncTestContext. Several error reports are allowed. Each fail report is represented as test variant.
  4. Disposing or cleaning.
  5. Notifying test framework on test procedure completion - AsyncTestContext.complete(). This step is nesseccary to continue testing with next execution.

Test function is responsible to catch all exceptions / assertions and to redirect them to AsyncTestContext.

ceylon.test features

  • After / before test hooks can be used by applying afterTest and beforeTest annotations. Functions marked by these annotations are executed synchronously after and before test correspondently.
  • Parametrized test with a set of variants can be performed using parameters annotation.
  • Test can be skipped marking it by ignore annotation.
  • Hooks the test with TestListener can be used. But all events except TestRunStartedEvent are raised after actual testing is completed.
  • It is not recommended to use ceylon.test::assertXXX functions together with AsyncTestContext, since this functions simply throws an exception which leads to testing completion. Use AsyncTestContext instead.
Exceptions
DeclarationLoadingErrorshared abstract DeclarationLoadingError

Exception thrown when declaration is not found on module or package.

DefaultConstructorNotFoundshared DefaultConstructorNotFound

Exception thrown when loaded / instantiated class doesn't contain default constructor.

ModuleLoadingErrorshared ModuleLoadingError

Exception thrown when module is not found.

PackageLoadingErrorshared PackageLoadingError

Exception thrown when package is not found.

TopLevelDeclarationLoadingErrorshared TopLevelDeclarationLoadingError

Exception thrown when top level declaration (function or class) is not found.