class: center, middle # test.check ### Atl-Clj ### Daniel Gibbons --- # Daniel Gibbons * Staff Engineer at Damballa * Master of Science, Computer Science * Georgia State University * Previously worked at Georgia State University, Turner Broadcasting, and AirWatch by VMware * daniel@danielgibbons.net * http://www.danielgibbons.net --- # test.check * Initially written by Reid Draper in November 2013 * Renamed from simple-check to test.check when moved to `clojure.contrib` * Property-based testing tool * Universal quantifications instead of individual test cases * Define expected properties of functions being tested * Modeled after Haskell's QuickCheck * Tests whether random inputs meet specified function properties * Useful for finding edge cases * Can be used to test a large number of random inputs * Uses `java.util.Random` * Can be seeded * Random input generation provides a large number of test cases automatically --- # Generators * Generates a lazy sequence of random values ```lisp user=> (gen/sample gen/int) (0 -1 -2 -1 3 4 -6 -5 -5 -3) user=> (gen/sample gen/string-ascii 3) ("" "&" ";#") user=> (-> [gen/int gen/string-ascii] gen/one-of (gen/sample 5)) ("" 0 "v" 2 "*GM") user=> ``` * Compound generators are generators that take other generators as arguments ```lisp (gen/sample (gen/not-empty (gen/map gen/keyword (gen/not-empty (gen/vector gen/int)))) 2) ``` ```lisp ({:G*J [-2 -3 -3]} {:I [2]}) ``` --- # Generators #### Generator Combinators * `fmap` - Produces a generator by applying a function to another generator ```lisp (gen/sample (gen/fmap set (gen/vector gen/int)) 5) ``` ```lisp (#{} #{} #{} #{1 -3} #{-1 3}) ``` * `such-that` - Produces a generator that enforces `pred` against each generated value * If no element is found after 10 tries, a `RuntimeException` will be thrown ```lisp (gen/sample (gen/such-that even? gen/int) 5) ``` ```lisp (0 -4 0 2 -2) ``` * `bind` - Produces a generator on the already realized values of another generator --- # Generators * `choose` - Returns numbers within a range * `one-of` - Random values from vector of generators * `frequency` - Values selected based on provided frequency for each type of generator * `elements` - Randomly chooses elements from the given collection * `not-empty` - Modifies a generator to not generate empty collections * `boolean` - `true` or `false` * `tuple` - Compound generator for generating pairs of values in vectors * `int` - Positive/negative integer * `nat` - Natural numbers * `pos-int` - Positive integers (alias for `nat`) * `neg-int` - Negative integers * `s-pos-int` - Strictly positive integers * `s-neg-int` - Strictly negative integers * `vector` - Compound generator which produces a vector whose elements are produced by the given generator * `list` - Like `vector`, but with lists * `byte` - `java.lang.Byte` * `bytes` - `byte[]` --- # Generators * `map` - Compound generator, which takes a key generator and value generator * `hash-map` - Compound generator, which takes varargs of keys and a value generators * `char` - Character from 0-255 * `char-ascii` - Character from 32 to 126 * `char-alpha-numeric` - Character from 48 to 57, 65 to 90, and 97 to 122 * `char-alpha` - Character from 65 to 90 and 97-122 * `char-symbol-special` - Random variation of `*`, `+`, `!`, `-`, `_`, and `?` * `char-keyword-rest` - Characters that can be the second char onwards of a keyword * `char-keyword-first` Characters that can be the first char of a keyword * `string` - String of `char` values, which can be unprintable * `string-ascii` - String of `char-ascii` values * `string-alpha-numeric` - String of `char-alpha-numeric` values * `keyword` - Generates keywords using `char-keyword-first` and `char-keyword-rest` * `ratio` - `Clojure.lang.Ratio` values --- # Properties * Result of applying the arguments produced by the generator to the function to test * Properties are also generators * Generate the result of applying the test function to the realized generator values * `for-all*` creates a property with a vector of generators and a function to apply the generators to * `for-all` is a macro for `for-all*` * Use parameters in an expression without a lambda --- # Shrinking * Search for an input that fails deeper in the tree than the root * Rose Tree * Modified depth-first search * Non-exhaustive --- # Namespace Structure ```bash ├── check │ ├── clojure_test.clj │ ├── generators.clj │ ├── properties.clj │ └── rose_tree.clj └── check.clj ``` --- # Examples ```lisp (ns test.core-test (:require [clojure.test :refer :all] [test.core :refer :all] [clojure.string :as s] [clojure.test.check :as tc] [clojure.test.check.generators :as gen] [clojure.test.check.properties :as prop] [clojure.test.check.clojure-test :as ct])) ``` ##### `f(f(x))=x` ```lisp (def clojure-string-involution-prop (prop/for-all [str gen/string] (= str (-> str s/reverse s/reverse)))) (tc/quick-check 10000 clojure-string-involution-prop) ``` ##### Evaluation ```lisp {:result true, :num-tests 10000, :seed 1402348973757} ``` --- # Examples ##### clojure.test integration ```lisp (ct/defspec clojure-string-reverse-satisfies-involution 10000 (prop/for-all [str gen/string] (= str (-> str s/reverse s/reverse)))) ``` ##### Evaluation ```bash $ lein test lein test test.core-test {:test-var clojure-string-reverse-satisfies-involution, :result true, :num-tests 10000, :seed 1402349598440} Ran 1 tests containing 1 assertions. 0 failures, 0 errors. $ ``` --- # Examples ##### Failing Example ```lisp (def less-than-5-prop (prop/for-all [i gen/int] (< i 5))) (tc/quick-check 10000 failing-prop :max-size 10) ``` ##### Evaluation ```lisp {:result false, :seed 1402352044477, :failing-size 8, :num-tests 9, :fail [7], :shrunk {:total-nodes-visited 9, :depth 2, :result false, :smallest [5]}} ``` --- # Questions?