Keep calm and do more testing

As data scientists, we are all familiar with what happens when a process, package or application breaks. We dive into it with interest to try to diagnose where the unanticipated error occurred. Did something unexpected occur in the raw data? Did our code not anticipate a particular permutation of inputs?

The process involved in fixing the issue can range from a simple tweak to days of hair-pulling and head banging due to frustrating debugging and rewriting.

Recently I started to get into testing in R, and was pleasantly surprised by how intuitive it was. When we develop a process, package or application, it does not take a great deal more effort to add testing to the workflow, but the benefits in terms of debugging can be substantial.

Testing infrastructure in R

The key package in R that enables smooth, automated testing is testthat. When inside your R project, you can initialize a testing infrastructure by a simple command

devtools::use_testthat()

This command will, among other things, create a directory inside your project called tests/testthat in which you can place R scripts that perform testing.

If your testing requires environment variables to be set locally, eg database credentials, you can set these through setting up a project specific environment using

usethis::edit_r_environ("project")

which will open an editable file in RStudio where you can enter the variable names and values for that specific project.

To write the tests themselves, you can create R scripts that have the test code in them and place them in the new tests/testthat subdirectory. These scripts are just like any other piece of R code, except that they use convenient wrappings and testthat functions designed for testing. The critical function is testthat::test_that() which takes two arguments:

  1. desc, a description of the test in the form of a character string, eg "Test that I have all the cat data I expect"
  2. code, the code which performs the calculations you need and then tests if those calculations return the expected value, wrapped in {}.

Typically the codewill end in a comparison function of some form. testthat contains multiple comparison functions, such as:

  • testthat::expect_equal() which tests whether two arguments are equal.
  • testthat::expect_gte() which tests whether the first argument is greater than or equal to the second
  • testthat::expect_s3_class() which tests if the first argument has the S3 class given in the second argument
  • testthat::expect_true() which tests that the statement contained in the argument evaluates to TRUE

Once you have written your test scripts, you can then run all your tests automatically and see a summary of the performance using the simple command devtools::test().

Examples of testing code

Here are a couple of really simple examples. Let’s say that we have a process which generates the average mpg by cyl in the mtcars dataset and writes it to a table in a database called mpg_by_cyl.

Now, being a great R citizen, you know your mtcars dataset pretty well, and you know that there are three cyl values, so you would expect your mpg_by_cyl database table to have three rows. So, in your test script you could grab mpg_by_cyl from the database and you could write the following simple test:

testthat::test_that("mpg_by_cyl has expected number of rows", {

mpg_by_cyl %>%
nrow() %>%
testthat::expect_equal(3)
})

When you run your test, if mpg_by_cyl does indeed have three rows, you will get a nice Woot! or some similar word of encouragement from devtools::test(). If it doesn’t come back as expected, you’ll be alerted to the condition that failed and a nice soothing message like Nobody`s perfect! Then you know that something didn’t happen as expected in relation to mpg_by_cyl.

Here’s another example. Let’s say you have a script that generates an object date containing a character string to represent a recent date of the format "%d %B %Y", for example "14 December 2017", and your process writes this to a database somewhere. You want to test whether a meaningful date was indeed generated and written to the database. One way of doing this would be to check if the string contains " 20" as you would expect for any recent date in the expected format. So you could get your test file to grab date from the database it was written to, and then write the following test:

testthat::test_that("date contains a recent date", {

testthat::expect_true(grepl(" 20", date))
})

This will evaluate whether or not what was written to the database contains the expected string, and if it fails you know that something went wrong with this process. (Of course this does not completely verify that the expected date was written, but data scientists will determine how to write a specific test to give them the degree of certainty they need).

Personally, I find the infrastructure for testing in R very intuitive and I would recommend that you get into the habit of writing tests for all your packages, processes and applications. You can thank me later!

For more information on testing in R, check out Hadley Wickham’s open book here.

Leave a Reply

%d bloggers like this: