Meet Crockford’s JSCheck

There are dozens of JavaScript testing frameworks, but most of them function in, more or less, the same way. However, Douglas Crockford’s JSCheck is considerably different from most. In this tutorial, I’ll show you how it’s different and why you should consider using it!

Crockford describes JSCheck as a “specification-driven testing tool.

Crockford describes JSCheck as a “specification-driven testing tool.” When using the frameworks you’re used to, you would write a test for a given piece of functionality, and, if that test passes, declare that the given functionality is working correctly. However, it’s possible that you might miss some of edge cases or exceptions that your tests don’t cover.

While uncovering edge cases isn’t the express purpose of JSCheck, it is a nice side benefit. The main idea behind JSCheck is this: the specification you write will actually describe how the code you are testing should work. Then, JSCheck will take that specification (called a claim in JSCheck-lingo), and generate random tests to prove the claim. Finally, it will report the results to you.

Sounds interesting? Read on! Sounds familiar? You might have used the Haskell testing tool, QuickCheck, on which JSCheck was based.

Some Code to Test

Of course, before actually writing our claim, we’ll want to have some code to test. Recently, I wrote a mini-password scorer, similar to the functionality on HowSecureIsMyPassword.net. It really isn’t fancy: you just pass the function a password and get a score back. Here’s the code:

passwordScorer.js

(function () {
    var PasswordScorer = {};

    PasswordScorer.score = function (password) {
        var len = password.length,
            lengthScore = 0,
            letterScore = 0,
            chars = {}

        if      (len >= 21) { lengthScore = 7; }
        else if (len >= 16) { lengthScore = 6; }
        else if (len >= 13) { lengthScore = 5; }
        else if (len >= 10) { lengthScore = 4; }
        else if (len >=  8) { lengthScore = 3; }
        else if (len >=  5) { lengthScore = 2; }

        var re = [ null, /[a-z]/g, /[A-Z]/g, /\d/g, /[!@#$%\^&amp;\*\(\)=_+-]/g];

        for (var i = 1; i < re.length; i++) {
            letterScore += (password.match(re[i]) || []).length * i;
        }

        return letterScore + lengthScore;
    };

    (typeof window !== 'undefined' ? window : exports).PasswordScorer = PasswordScorer;
}());

It’s pretty simple code, but here’s what’s going on: the score is made up of two sub-scores. There’s a starting score, that’s based on the length of the password, and then an additional score for each character, 1 point for each lowercase letter, 2 points for each uppercase letter, 3 points for each number, and 4 points for each symbol (from a limited set).

So, this is the code we’re going to test: we’ll randomly generate some passwords with JSCheck and make sure they get an appropriate score.

Writing our Claim

Now we’re ready to write our claims. First, head over the JSCheck Github page and download the jscheck.js file. I like to run my tests in the terminal, via NodeJS, so add this single line to the very bottom of the file:

(typeof window !== 'undefined' ? window : exports).JSC = JSC;

This won’t affect the way the file behaves in the browser at all, but it will make it work as a module within Node. Notice that the jscheck.js file exposes JSC as the single global variable for the whole library. If we weren’t making this adjustment, that’s how we’d access it.

Let’s open passwordScorerSpec.js and start things:

JSC = require("./../vendor/jschec";).JSC;
PasswordScorer = require("./../lib/passwordScore";).PasswordScorer;

Since I’m running these tests in NodeJS, we’ll have to require the modules we want. Of course, you’ll want to make sure that paths match your file locations.

Now, we’re ready to write our first claim. Of course, we use the JSC.claim method. This method accepts three parameters, with an optional fourth. The first parameter is just a string, a name for the claim. The second parameter is called the predicate: it’s the actual testing function. Very simply, this function should return true if the claim is true, and false if the claim is false. The random values that JSCheck will generate for the test will be passed as parameters to the predicate.

But how does JSCheck know what type of random values to hand the predicate? That’s where the third parameter, the specifier comes into play. This is an array, with an item for each parameter for predicate. The items in the array specify what types to give the predicate, using JSCheck’s specifier functions. Here are a few of them:

JSC.boolean() returns either true or false.
JSC.character() takes a min and max character and returns a single character from that range. It can also take a single character code and return that character.
JSC.integer() will return a prime number. Or, pass it a single parameter to get an integer (whole number) between 1 and the parameter, or two parameters for an integer in that range.

You get the idea. There are other specifiers, and we’ll use some now as we write our first claim.

JSC.claim("All Lowercase Password";, function (password, maxScore) {
  return PasswordScorer.score(password) <= maxScore;
}, [
  JSC.string(JSC.integer(10, 20), JSC.character('a', 'z')),
  JSC.literal(26)
]);

Our first parameter is a name. The second is the testing function: it receives a password and a max score, and returns true if the score for that password is less than or equal to the max score. Then, we have our specifier array. Our first parameter (the password) should be a string, so we use the JSC.string() method: it can take two parameters, the number of characters in the string, and value for those characters. As you can see, we’re asking for a password between 10 and 20 characters. For the value, we’re using the JSC.characters() method to get random characters between ‘a’ and ‘z’.

The next value is our maxScore parameter. Sometimes, we don’t want the randomness that JSCheck offers, and this is one of those times. That’s why there’s JSC.literal: to pass a literal value the predicate. In this case, we’re using 26, which should be the max score for any all-lowercase password between 10 and 20 characters.

Now we’re ready to run the test.

Running our Claim

Before we actually run the claim and get the report, we have to setup the function that will receive the report. JSCheck passes the report to a callback function of JSC.on_report. Hence:

JSC.on_report(function (str) { 
  console.log(str); 
});

Nothing fancy. Now, all that’s left is to call JSC.check(). Now, we can head to our terminal and run this:

node path/to/passwordScorerSpec.js

Behind the scenes, JSCheck runs the predicate 100 times, generating different random values each time. You should see your report printed out.

All Lowercase Passwords 100 of 100
 pass 100

They all passed, but that’s not much of a report, eh? Well, if any of our tests had failed, they would have been included in the report. However, you can adjust the level of output with the JSC.detail function: pass it a number between 0 and 4 (inclusive) to get anything for no output to all the test cases. The default value is 3.

Adding a Classifier

Remember how I said that JSC.claim could take a fourth parameter? It’s called a classifier, and it receives the same parameters that the predicate receives. Then, it can returns a string to classify, or group, our test cases. I’ll admit I wasn’t really sure where this would be useful until I was creating the above example claim. See, I made a mistake in the predicate and compared the score to the maxScore with the operator instead of the operator, so any passwords that scored 26 points were failing. I was seeing reports that looked something like this:

All Lowercase Passwords 96 of 100
 FAIL [12] ("vqfqkqqbwkdjrvplkrx";,26)
 FAIL [21] ("nhgkznldvoenhqqlfza";,26)
 FAIL [62] ("eclloekuqhvnsyyuekj";,26)
 FAIL [78] ("rvrkfivwtdphrhjrjis";,26)
 pass 96 fail 4

It’s still not entirely obvious why some tests are failing. So I added a classifier function that grouped the test cases by score: like I said, the function takes the same parameters as the predicate, and it returns a string. Every test case that gets the same string back from the classifier will be grouped together in the report.

function (password, maxScore) {
	return PasswordScorer.score(password) + " points";; 
}

This function should be the last parameter of our claim. Now, you’ll get a report that’s something like this:

All Lowercase Passwords 96 of 100
 FAIL [4] 26 points:("illqbtiubsmrhxdwjfo";,26)
 FAIL [22] 26 points:("gruvmmqjzqlcyaozgfh";,26)
 FAIL [34] 26 points:("chhbevwtjvslprqczjg";,26)
 FAIL [65] 26 points:("kskqdjhtonybvfewdjm";,26)
14 points: pass 8
15 points: pass 5
16 points: pass 12
18 points: pass 10
19 points: pass 12
20 points: pass 11
22 points: pass 12
23 points: pass 8
24 points: pass 10
25 points: pass 8
26 points: pass 0 fail 4

You can see how the tests are grouped by how many points the passwords are worth. Now, it’s easy to see that the only passwords that fail the tests are the passwords that score 26 points. And while the problem here was with the test, and not the code, it still shows how it can be useful to add a classifier function to your claims.

Final Thoughts

So, at the end of the day, it JSCheck worth using? Here’s what I think: it’s not something you’re necessarily going to use with every code base, but sometimes you’ll find it useful to be able to create random test cases that will rigorously test a given piece of code. When that’s what you want to do, I haven’t seen a tool better for that than JSCheck.

JSCheck has a few other options and a bunch of specifiers that we haven't reviewed in this tutorial; head over to JSCheck.og to read about those. Otherwise, I’d love to hear your thoughts about JSCheck in the comments!

HIGHLIGHTS OF THE DAY