======= Testing ======= So you want to easily test the code your writing? The following recipes cover how to write automated tests. We will use an established and well-designed regression testing framework called 'RT'. In the last recipe we will discuss other options. Install RT ---------- You can learn more about Dr. Richard C. Waters' `RT on CLiki `__. We will get up and running quickly, by doing a standard asdf-install which will pull down Kevin Rosenburg's packaged version. **Note: sbcl users can simply use sb-rt which is prebundled with their system**. .. code:: lisp ;; Install rt from cliki (asdf-install:install :rt) This command downloads and installs the RT package. This step only needs to be done once. If you just did the install then you can skip the next step. .. code:: lisp (asdf:operate 'asdf:load-op :rt) This line loads the RT package into your Lisp image. Write a Test ------------ Normally we would be writing tests against our own code, but for clarity let's write some tests against operations from the `String recipes `__ sections. .. code:: lisp ; using *my-string* from the substrings recipe "Groucho Marx" (rt:deftest test-subseq-one-index (subseq *my-string* 8) "Marx") TEST-SUBSEQ-ONE-INDEX That substring recipe told us that we could expect "Marx" if we ask for substring to 8. We use the deftest macro, naming our test ``test-subseq-one-index``. In the body of the test we specify the code we want to test, followed by the expected results. The code ``(subseq *my-string* 8)``\ is evaluated and the number of values return must match the number of expected results. In this example their is only one value returned and it matches the one expected value ``"Marx"``. For each actual, expected pair, they must be **``equal``**. Note that the REPL prints the test name and under the covers this test is added to the current test suite. Run a Test ---------- We have defined a test, let's run it. .. code:: lisp CL-USER> (rt:do-test 'test-subseq-one-index) TEST-SUBSEQ-ONE-INDEX We have RT test our assertion for how ``subseq`` should behave. The test passes, so no errors are reported. ``do-test`` simply returns the name of the test. In the next example we will see how RT behaves with failing tests. Tests can be run one at a time with ``do-test`` as you incrementally build up your system, or they can be run all at once with ``do-tests``. For our second test, let's define and run a failing test. .. code:: lisp CL-USER> (rt:deftest test-subseq-two-indicies (subseq *my-string* 0 7) "Xroucho") TEST-SUBSEQ-TWO-INDICIES CL-USER> (rt:do-test 'test-subseq-two-indicies) Test TEST-SUBSEQ-TWO-INDICIES failed Form: (SUBSEQ *MY-STRING* 0 7) Expected value: "Xroucho" Actual value: "Groucho". NIL We purposely provide an expected value which is wrong, so that the test will fail. Notice RT describes a failed test to standard output. Also the ``do-test`` function returns ``NIL`` instead of the test as before. The great value of test code is running it automatically and frequently to find issues early in development. RT provides the ``do-tests`` function to regress all of the tests. .. code:: lisp CL-USER> (rt:do-tests) Doing 2 pending tests of 2 tests total. TEST-SUBSEQ-ONE-INDEX Test TEST-SUBSEQ-TWO-INDICIES failed Form: (SUBSEQ *MY-STRING* 0 7) Expected value: "Xroucho" Actual value: "Groucho". 1 out of 2 total tests failed: TEST-SUBSEQ-TWO-INDICIES. NIL Let's redefine ``test-subseq-two-indicies`` with a passing expected value. Additionally let's define a third test. .. code:: lisp CL-USER> (rt:deftest test-subseq-two-indicies (subseq *my-string* 0 7) "Groucho") WARNING: Redefining test TEST-SUBSEQ-TWO-INDICIES TEST-SUBSEQ-TWO-INDICIES (rt:deftest test-concatenate-three-strings (concatenate 'string "Karl" " " "Marx") "Karl Marx") TEST-CONCATENATE-THREE-STRINGS CL-USER> (rt:do-tests) Doing 3 pending tests of 3 tests total. TEST-SUBSEQ-ONE-INDEX TEST-SUBSEQ-TWO-INDICIES TEST-CONCATENATE-THREE-STRINGS No tests failed. T Note that ``do-tests`` picks up our third test ``test-concatenate-three-strings`` because it runs the entire test suite and prints a report to standard out that no tests have failed. The value ``T`` is returned to signal a successful test run. RT Odds and Ends ---------------- If we pretend for a moment that ``subseq`` and ``concatenate`` are Lisp operations which we have written, you can see how easy it is to build up a collection of tests to exercise and regress code as we write and maintain it. If you like using RT so far, you will want to read up on the documentation which comes with it, or read the `original paper `__ from 1991 which covers RT and COVER (a code coverage framework). At a minimum you will want to add .. code:: lisp (rem-all-tests) to the beginning of your test code for the current project. You can use ``continue-testing`` to run all of the tests which have either failed, or are newly defined. Other Test Frameworks --------------------- Test Driven Development, xUnit testing frameworks, Automated Acceptance Tests, etc are quite the rage in some areas of contemporary programming. If RT doesn't fit your needs, or you would like to see what else is out there, please see the following: - `CLiki Test Frameworks `__ - `ALU Wiki Test Frameworks `__ - `Unit Testing `__ on The Common Lisp Directory and one of many which is a good choice to look at is: - `FiveAM `__ - Contemporary and very Lisp