• tests can be grouped into composable modules
• simple syntax for input/output arguments
• nestable assertions with titles that get concatenated
• hooks for custom before/after/inbetween procedures and custom reporters
• using test-modules is optional

• all modules and procedures for testing are compiled before test procedures are run, so that no auto-compile messages or compile errors appear while testing
• test-modules can extend or customise main test runner settings
• no restriction on how modules will run tests. for example, modules can run tests "repeated three times, in parallel"
• tests in multiple files can be run as one, with grouped output
• "only" and "exclude" options for selecting specific tests or modules to be run

• scheme interface - execute tests from within scheme, for continuous integration for example
• standard-output result reporting is optional
• space efficient reporting format "compact"
• extendable report formats, stored in a hashtable and selected via test settings

tests are procedures, test-modules are r6rs-libraries, mostly functional, no hidden state variables, no goops objects, no opaque data-types,

(define-test-module (test module sph string)
  (import (sph string))

  (test-execute-procedures-lambda
    (string-indices
      ("/a////b//c/d" "/") (0 2 3 4 5 7 8 10)
      ("/a////b//c/d" "//") (2 4 7)
      ("abcd" "bc") (1)
      ("abcdbcefbcg" "bc") (1 4 8)
      ("abcd" "cd") (2)
      ("ab" "") (0 1 2)
      ("" "") (0)
      ("" "ab") ()
      ("abcd" "cb") ())
    (string-multiply
      ("a" 0) ""
      ("a" 3) "aaa"
      ("" 3) "")
    (string-replace-string
      ("/a////b//c/d" "//" "/") "/a//b/c/d"
      ("abcd" "bc" "") "ad"
      ("abcdbcefbcg" "bc" "hij") "ahijdhijefhijg"
      ("abcd" "abc" "e") "ed"
      ("abcd" "cd" "efg") "abefg"
      ("ab" "" "") "ab"
      ("ab" "" "/") "/a/b/"
      ("" "" "/") "/"
      ("" "" "") "")))

test-execute-procedures-lambda ist a helper for defining basic input/output tests.

string-indices 1 2 3 4 5 6 7 8 9
string-multiply 1 2 3
string-replace-string 1 2 3 4 5 6 7 8 9

• inp: the arguments to the test-procedure
• exp: the expected result
• out: the actual result

string-indices 1 2 3 4 5 6 7 8 9
string-multiply 1 2 3
string-replace-string 1 2 3 4 5 6 7 8 9
string-quote 1
  failure string-quote 2
  inp "t'est"
  exp "\"t'est\"x"
  out "\"t'est\""

test module sph vector
  vector-append 1
  vector-produce 1
  vector-range-ref 1 2 3
  vector-select 1
test module sph record
  define-record 1
  record 1
  alist->record 1
  record-field-names 1

(define-test-module (symbol ...)
  (import r6rs-library-import-spec ...)
  any ...
  procedure:{settings:list -> boolean/vector:test-result})

the following libraries are implicitly imported: (guile) (rnrs base) (sph) (sph test). the last expression of a test-module must be a procedure. this procedure will be executed to get the test result for the module.

(define-test-module (test module mymodule)
  (import (mymodule))

  (lambda (settings)
    ;create a test-result, a boolean would do it, but the following creates a test-result record with more information
    (assert-true "addition-test" (= 3 (+ 2 1)))))

the rest of the define-test-module body is exactly like the body of a r6rs-library definition. another example of how a module can look like:

(define-test-module (test module my-module)
  (import (my-module))

  (define (my-utility a b) (+ a b))

  (define-test (addition)
    (assert-true "addition-test" (= 3 (+ 2 1))))

  (define-test (addition-2 arguments)
    (apply + arguments))

  (test-execute-procedures-lambda
    addition
    (addition-2 (5 4) 9)
    (+
      (1 2) 3
      (3 4) 7)))

test-results in a test-modules execute procedure can be created in any way scheme permits. (sph test) offers a few additional bindings for that. they all use a specific notation for tests to be run, the test arguments and expected-results, called "test-spec".

symbol/(symbol [any/(any ...) ...])

procedure-name/(procedure-name [arguments-and-expected-result-alternatingly ...])

(string-quote "'test'" "\"'test'\"")
(string-multiply ("" 3) "")

(test-execute-procedures-lambda (my-procedure (unquote (+ 1 2)) 3))

"test-execute-procedures" and similar bindings look for custom test procedures with certain type signatures. the following syntax can be used to define such procedures.

(define-test (name argument-name ...) test-result)

(define-test name procedure)
(define-test (abc) #t)
(define-test (abc arguments-list) #t)
(define-test (abc arguments-list expected-result) #t)
(define-test (abc name index arguments-list expected-result) #t)

test-{name} :: [name index] arguments expected-result -> any/vector/boolean:test-result

(define (test-abc) #t)
(define (test-abc arguments-list) #t)
(define (test-abc arguments-list expected-result) #t)
(define (test-abc name index arguments-list expected-result) #t)

• module test results can be booleans indicating success status
• test-execute-procedures results can be booleans or any other result to be compared with the expected result
• alternatively any result can be a test-result record with additional information

(test-create-result [boolean:success? string:title integer:index any:result list:arguments any:expected])

given the recommended directory structure for tests, one can create standard r6rs libraries in files stored in a load-path under test/helper/ and import them in test-modules

currently it is fully up to the user to create and destroy eventually necessary environments for tests. for example database initialisation with test data. given the hook system of (sph test), it should be possible to create nice abstractions for that.

(define-as test-settings-default alist-q
  reporters test-reporters-default
  reporter-name (q default)
  search-type (q prefix)
  hook
  (alist-q procedure-before ignore
    procedure-after ignore
    procedure-data-before ignore
    procedure-data-after ignore
    module-before ignore module-after ignore modules-before ignore modules-after ignore)
  random-order? #f parallel? #f exception-strings? #t exclude #f only #f until #f)

if true, exeptions occuring in test-execute-procedures tests are catched and converted to strings that will be the result of the test procedure. this way, tests for if an exception occurs can be made.

list of procedure or module names to exclude from test execution

the association list stored under the "hook" key contains procedures that are called at specific times in the test execution process. the type signatures can be seen in the following scheme comments.

(define-as test-report-hooks-null alist-q
  ;settings name ->
  procedure-before ignore
  ;settings result ->
  procedure-after ignore
  ;settings name index data ->
  procedure-data-before ignore
  ;settings result ->
  procedure-data-after ignore
  ;settings module-name ->
  module-before ignore
  ;settings module-name result ->
  module-after ignore
  ;settings module-names ->
  modules-before ignore
  ;settings module-names result ->
  modules-after ignore)

• takes preference over "exclude"
• either "only" or "exclude" are applied, not both
• "only" and "exclude" is used for both module and procedure tests

• tests are executed in separate threads
• all tests will be tried to be executed even if one fails

• a load path (a root path from which module names start) to restrict the search for modules to
• one use case is to execute only tests local to a project directory
• this can be used to execute test-modules that have not been installed into a guile load path, without conflicting with modules that are installed there

the test order is randomised before execution

default reporter names are "compact", "null" and "default" (which is equivalent to "null")

symbol:name -> (procedure:report-result:{test-result [port] ->} . list:hook-configuration)

reporter configuration is a pair that consists of a procedure for reporting a test-result object and a hook-configuration

specifies how to match module names: partially (prefix or prefix-not-exact) and therefore possibly multiple modules, or as exact, full module names

stops when reaching the specified procedure/module

• modules are r6rs-libraries that export only one procedure named "execute" that is supposed to create a test-result
• execute :: settings -> test-result
• modules are resolved in the scheme implementation specific way
• libraries are used to give test-modules a separated scope to include the modules they depend on
• it also gives test-modules a resolvable name. an alternative implementation could have evaluated top-level file content, which might use "import", in a custom environment, but this might lead to less separation of the module content. test-modules can be imported like other libraries as long as the define-test-module syntax is available before import
• to use the define-test-module syntax, the define-test-module syntax definition must be previously loaded in the environment in which the module is to be loaded. to archieve this, module file content is first evaluated in the top-level environment to define the module, then the module object is resolved
• modules export the procedure to accept the settings binding and to delay execution

• list/record/boolean
• procedure output is compared with the expected value to determine test success
• the arguments and expected value are only set on failure
• procedure tests with multiple test-data create only one test-result with the index set to the last tested index

this macro exists for leaving out optional parameters and to avoid the need to manually prepend "test-" to the name.

• mocha. defines test procedures and data inline
• trc-testing
• rails testing

• bunny-test
• guile-lib unit-test uses goops
• schemeunit
• srfi-64
• test-manager
• testeez r5rs