window.markdeepOptions={tocStyle:"medium"} Testing

Contents

Testing
  1.1  Creation
  1.2  Echoing
  1.3  Results
  1.4  Directories
  1.5  Inputs
  1.6  Arguments
  1.7  Assert
  1.8  Options
  1.9  Tracing
  1.10  Errors
  1.11  Builtins

Javascript Shell Interpreter
 DOWNLOAD / DEVELOP / DEPLOY

Types Builtin Reference  Index  Fossil  Demos Log Test Debug Misc

   

Testing

Jsi test scripts invoked with "-u” produce output that is compared with text in an embedded comment:

puts('FOO'); /* =!EXPECTSTART!= FOO =!EXPECTEND!= */

A test fails if program output does not match an embedded EXPECT comment, or an error occurs.

jsish -u test.js
[PASS] test.js

Here is larger example:

#!/usr/local/bin/jsish -u %s function foo(n) { return "FOO"+n; } puts(foo(1)); puts(foo(2)); /* =!EXPECTSTART!= FOO1 FOO2 =!EXPECTEND!= */
   

Creation

After creating a test script you can the auto-create the EXPECT comment with "-update”:

jsish -u -update true foo1.jsi foo2.jsi
jsish -u -update true tests/

If script already contains an EXPECT comment, it will be updated in-place.

   

Echoing

To echo expressions in test-mode use a semi-colon in column one:

var x; ;x=1;

This outputs the expression and value, eg:

jsish -U test2.js
x=1 ==> 1

that is, lines are implicitly rewritten to:

  "printf(\"%%s ==> \",\"%s\"),puts(%s);\n"

Here is a larger example:

#!/usr/local/bin/jsish -u %s function a(n) { var sum = 0; for (var i = 0; i < n; i++) sum = sum + i; return sum; }; ;'===Begin Test==='; ;a(10); ;a(100); ;a(1000); /* =!EXPECTSTART!= '===Begin Test===' a(10) ==> 45 a(100) ==> 4950 a(1000) ==> 499500 =!EXPECTEND!= */

In the transformed code the expansion is:

puts("'===Begin Test==='");
puts("a(10) ==> ", a(10));
puts("a(100) ==> ", a(100));
puts("a(1000) ==> ", a(1000));

Note the special case of single-quoted lines which are output verbatim

To see this output we can use the -U option:

jsish -U tests/func.js
'===Begin Test===' a(10) ==> 45 a(100) ==> 4950 a(1000) ==> 499500

Implicit and explicit puts can freely be intermixed.

#!/usr/local/bin/jsish -u %s function foo() { return true; } function bar() { return true; } ;bar(); puts('foo() ==> ', foo()); puts('DONE'); /* =!EXPECTSTART!= bar() ==> true foo() ==> true DONE =!EXPECTEND!= */

The following rules apply to implicit puts:

   

Results

By default, a test result is either PASS (when output matches) else a FAIL followed by a description of the differences:

jsish -u prob/testfail.js
[FAIL] prob/testfail.js at line 2 of output: output: <bar failed=""> expect: <bar passed=""> ====================DIFFSTART foo passed -bar passed +bar failed baz passed ====================DIFFEND*

The return code is zero if the test did not failed.

   

Directories

Arguments to jsish -u can either be one or more files, or a single directory in which to look for .js or .jsi files:

jsish -u tests/
[PASS] tests/49.js [PASS] tests/99bottles.js [PASS] tests/alias.js [PASS] tests/apply.js [FAIL] tests/arg.js at line 9 of output: output: <4> expect: <5> [FAIL] tests/arg2.js at line 9 of output: output: <1> expect: <2> [PASS] tests/alias.js Test tests/array.js ... [PASS] tests/yhsj.js 2 FAIL, 97 PASS: runtime 21551 ms

The return code is the number of failed tests (up to a maximum of 127).

echo $?
2

There is a summary when multiple test are run, but no DIFF (unless context>3).

   

Inputs

A scripts requiring test input (to read from stdin) it can be specified as follows:

/*
=!INPUTSTART!=
45
4950
=!INPUTEND!=
*/

This can also be provided from the command-line using -inStr or -inFile. The default is no input.

   

Arguments

A may script may specify arguments with an ARGS comment:

/*
=!ARGSSTART!=
-debug 1 able baker charlie
=!ARGSEND!=
*/

This should be on a single line, from which newlines will be stripped.

This can also be provided from the command-line using -args. The default is no arguments.

   

Assert

Asserts, which are normally off by default, are enabled during unit-testing. This can result in test scripts failing when assert throws an uncaught error.

There are various ways to adjust the behaviour of assert when working with test scripts, eg. assert.jsi:

#!/usr/local/bin/jsish -u %s "use asserts"; // note: test mode already enables this. assert(true,'true'); assert(2*3 == 6,'math'); try { assert(false,'false'); } catch(e) { puts('caught error'); } ;Interp.conf({asserts:false}); var x = 1; ;x; ;assert(false,'false2'); ;assert(false===true); ;Interp.conf({asserts:true}); var i=1, j=2; ;assert(function () { return (i < j); },'fail'); try { assert(false==true); } catch(e) { puts('caught error2: '+e); } try { ; assert(false,'false'); } catch(e) { puts('caught error2: '+e); } ;assert(false,'this assert failed',{mode:'puts', noStderr:true}); ;Interp.conf({assertMode:'puts', noStderr:true}); ;assert(true===false); ;assert(false,'assert also failed');
   

Options

Option help can be dump with:

 jsish -u -h:
Run script(s) as unit-tests setting return code to number of failed tests. Options/defaults: -args "" // Argument string to call script with -context 3 // Number of context lines for DIFF (>3 forces dir diff). -debug false // Enable debugging messages. -echo false // Run with puts/assert output showing file:line number. -evalFile "" // File to source in subinterp before test. -exec false // Use exec instead of running test in a sub-interp. -expectFile null // File to use for expected output. -failMax 0 // Quit after this many failed tests. -inFile null // File to use for stdin input. -silent false // Run quietly, showing only failures. -update false // In place update or create of EXPECT section. -verbose false // Echo values for inputs/outputs/args.

A script can detect unit-test mode using:

if (Interp.conf('unitTest') > 0)
    puts("We are testing...");
   

Tracing

When a test fails, -echo can show the source line associated with output:

jsish -u -echo true prob/testfail.js
Test prob/testfail.js /home/user/src/jsi/jsi/tests/prob/testfail.js:7: "foo passed", /home/user/src/jsi/jsi/tests/prob/testfail.js:8: "bar failed", /home/user/src/jsi/jsi/tests/prob/testfail.js:9: "baz passed",

Edit the script as follows to run from Geany with F9, so as to click-to-navigate the output:

#!/usr/local/bin/jsish -u -echo true %s

You can also try -verbose for even more detail.

   

Errors

In C code there are 4 basic classes of errors (from most to least severe):

And for scripts:

The script tools/randtest.jsi calls all functions with various arguments.

   

Builtins

The builtin self-test files are here: https://jsish.org/fossil/jsi/dir?name=tests

var startOfMarkDeep=true; window.alreadyProcessedMarkdeep||(document.body.style.visibility='visible'); document.title=location.pathname.match(/\/([\w]+)1;

formatted by Markdeep 1.03