Testing Haskell with Hspec

Today I will be looking at Hspec, a Haskell BDD automated testing framework roughly based on Ruby’s RSpec.

With Haskell being a strongly typed language, it is hard to imagine it being anything like RSpec.

Certainly the implementation has to be very different, because RSpec uses much of Ruby’s dynamic nature to make it as concise and intuitive as it is.

One tiny detail I noticed straight off the bat is that RSpec has a capital S in the name while Hspec does not.

Let’s try a small example and see what it is like.

What we will be testing

By the end of this, we will have implemented a custom zip’ recursive function which acts the same as zip found in the Prelude module. This is a very basic function that accepts a list of tuples, and retuns a list of tuples containing elements from both lists that occur in the same positions.

Example:

Input: zip [1,2,3] [9,8,7]

Output: [(1,9),(2,8),(3,7)]

Setup

There is a command line tool called Cabal that enables you to install Haskell packages and automatically deal with their dependencies. By default cabal will look in Hackage, which is Haskell’s central package archive.

cabal update && cabal install hspec

This takes a few moments to execute, and it flagged some warnings regarding functions with a ’ in the name, but did not break the install. (Having a ’ in a function name is perfectly normal in Haskell and indicates that it is related to a preceding function with the same name but lacking the ‘. This kind of function is called a prime, just like a prime in algebra)

First test

Let’s create our first minimal spec file.

mkdir ~/hspec_test && cd ~/hspec_test
touch zipSpec.hs

It is Hspec convention to name spec files with “Spec” in the name, which enables automatic spec discovery when you want to run an entire suite. You will also generally have one spec file per source file, but in this small example, I will be defining the code inline in the spec file.

Ok lets see what happens when we run this blank spec file.

runhaskell zipSpec.hs

As expected, we get an error to help guide us along the way.

zipSpec.hs:1:33:
    Not in scope: `main'
    Perhaps you meant `min' (imported from Prelude)

This error message tells us that there is no “main” action, and we certainly did not want to run the min function.

Like in C the main action is the required entry point into the program. So let’s create it.

There is also some boilerplate to get this going is pretty minimal and we only need to import Hspec.

Add the following to your zipSpec.hs file.

import Test.Hspec

main = hspec $ do
  describe "zip'" $ do
    it "creates a list of tuples containing elements of both lists occuring in the same position" $ do
      zip' [1,2,3] [9,8,7] `shouldBe` [(1,9),(2,8),(3,7)]

We have created a binding for the main action, which is our spec declaration. If you were wondering about some of strange syntax, dont worry. It is the way that Haskell achieves such a high level of expressiveness. This means that very few lines of code can convey an incredible amount of information.

I would like to point out in particular the $ sign, which is simply syntactic sugar making it possible to omit the parentheses.

We run our test again and get our first actual assertion error.

zipSpec.hs:6:5:
    Not in scope: zip'
    Perhaps you meant one of these:
      `zip3' (imported from Prelude), `zip' (imported from Prelude)

Getting to Green

We can now start filling out our program to satisfy the test.
We start by defining our type signature.

  import Test.Hspec

  zip' :: [a] -> [b] -> [(a,b)]

  main = hspec $ do
  ...

Run again with runhaskell zipSpec.hs

zipSpec.hs:3:1:
    The type signature for zip' lacks an accompanying binding

Next, as instructed by the failing test, we create our binding for the signature.

  import Test.Hspec

  zip' :: [a] -> [b] -> [(a,b)]
  zip' (x:xs) (y:ys) = (x,y):zip' xs ys

  main = hspec $ do
  ...

Run again, and we get our next error message:

zip'
  - creates a list of tuples containing elements of both lists occuring in the same position FAILED [1]

1) zip' creates a list of tuples containing elements of both lists occuring in the same position
uncaught exception: PatternMatchFail (zipSpec.hs:4:1-37: Non-exhaustive patterns in function zip')

Our full implementation of the function declaration, including the pattern matching code looks like this below. It can deal with two additional base cases where the inputs can be empty lists.

import Test.Hspec

zip' :: [a] -> [b] -> [(a,b)]
zip' _ [] = []
zip' [] _ = []
zip' (x:xs) (y:ys) = (x,y):zip' xs ys

main = hspec $ do
  describe "zip'" $ do
    it "creates a list of tuples containing elements of both lists occuring in the same position" $ do
      zip' [1,2,3] [9,8,7] `shouldBe` [(1,9),(2,8),(3,7)]

Running again, we get our first passing test.

 zip'
  - creates a list of tuples containing elements of both lists occuring in the same position

Finished in 0.0002 seconds
1 example, 0 failures

If we were to do this test first style, we would drive out the edgecases of empty lists being passed in. I am not doing this for brevity.

Obviously we do not know that we are testing anything until we have seen it fail for the right reason. Lets break it and see how the test responds. Change the assertion from [(1,9),(2,8),(3,7)] to [], resulting in an obivous assertion error and run the test again.

And back to red again

We run it again and see that it complains about the failed assertion, which is great.

zip'
  - creates a list of tuples containing elements of both lists occuring in the same position FAILED [1]

1) zip' creates a list of tuples containing elements of both lists occuring in the same position
expected: []
but got: [(1,3),(2,4)]
 

Ok so this is looking promising. We now have a test that enables us to start poking at haskell to see how it responds.

From here you can try out more functions, and make sure that you get the output that you expect. You will need to include the different library files depending on what functionality you are trying to test.

Hspec RSpec similarities

Describes can be nested to give extra context. In fact context in Hspec is just an alias for describe.

There are also before blocks for setup, before the test runs.
You can also create a spec helper file, just like RSpec that is imported into your specs, helping keep tests DRY.

More testing frameworks for Haskell

QuickCheck

HTF

HUnit

Smallcheck


April 6, 2014