Jay Fields's Blog, page 13

Clojure: Flatten Keys

I recently needed to take a nested map and flatten the keys. After a bit of trial and error I came up with the following code. (note: the example expects Expectations)(ns example
(:use expectations))

(defn flatten-keys* [a ks m]
(if (map? m)
(reduce into (map (fn [[k v]] (flatten-keys* a (conj ks k) v)) (seq m)))
(assoc a ks m)))

(defn flatten-keys [m] (flatten-keys* {} [] m))

(expect
{[:z] 1, [:a] 9, [:b :c] Double/NaN, [:b :d] 1, [:b :e] 2, [:b :f :g] 10, [:b :f :i] 22}
(flatten-keys {:z 1 :a 9 :b {:c Double/NaN :d 1 :e 2 :f {:g 10 :i 22}}}))
As the test shows, the code converts{:z 1 :a 9 :b {:c Double/NaN :d 1 :e 2 :f {:g 10 :i 22}}}into{[:z] 1, [:a] 9, [:b :c] Double/NaN, [:b :d] 1, [:b :e] 2, [:b :f :g] 10, [:b :f :i] 22}Improvement suggestions welcome.
© Jay Fields - www.jayfields.com

Clojure: Another Testing Framework - Expectations

Once upon a time I wrote Expectations for Ruby. I wanted a simple testing framework that allowed me to specify my test with the least amount of code.

Now that I'm spending the majority of my time in Clojure, I decided to create a version of Expectations for Clojure.

At first it started as a learning project, but I kept adding productivity enhancements. Pretty soon, it became annoying when I wasn't using Expectations. Obviously, if you write your own framework you are going to prefer to use it. However, I think the productivity enhancements might be enough for other people to use it as well.

So why would you want to use it?

Tests run automatically. Clojure hates side effects, yeah, I hear you. But, I hate wasting time and repeating code. As a result, Expectations runs all the tests on JVM shutdown. This allows you to execute a single file to run all the tests in that file, without having to specify anything additional. There's also a hook you can call if you don't want the tests to automatically run. (If you are looking for an example, there's a JUnit runner that disables running tests on shutdown)

What to test is inferred from your "expected" value. An equality test is probably the most common test written. In Expectations, an equality test looks like the following example.(expect 3 (+ 1 2))That's simple enough, but what if you want to match a regex against a string? The following example does exactly that, and it uses the same syntax.(expect #"foo" "afoobar")Other common tests are verifying an exception is thrown or checking the type of an actual value. The following snippets test those two conditions.(expect ArithmeticException (/ 12 0))

(expect String "foo")Testing subsets of the actual value. Sometimes you want an exact match, but there are often times when you only care about a subset of the actual value. For example, you may want to test all the elements of a map except the time and id pairs (presumably because they are dynamic). The following tests show how you can verify that some key/value pairs are in a map, a element is in a set, or an element is in a list.;; k/v pair in map. matches subset
(expect {:foo 1} (in {:foo 1 :cat 4}))

;; key in set
(expect :foo (in (conj #{:foo :bar} :cat)))

;; val in list
(expect :foo (in (conj [:bar] :foo)))Double/NaN is annoying. (not= Double/NaN Double/NaN) ;=> true. I get it, conceptually. In practice, I don't want my tests failing because I can't compare two maps that happen to have Double/NaN as the value for a matching key. In fact, 100% of the time I want (= Double/NaN Double/NaN) ;=> true. And, yes, I can rewrite the test and use Double/isNaN. I can. But, I don't want to. Expectations allows me to pretend (= Double/NaN Double/NaN) ;=> true. It might hurt me in the future. I'll let you know. For now, I prefer to write concise tests that behave as "expected".

Try rewriting this and using Double/isNaN (it's not fun)(expect
{:x 1 :a Double/NaN :b {:c Double/NaN :d 2 :e 4 :f {:g 11 :h 12}}}
{:x 1 :a Double/NaN :b {:c Double/NaN :d 2 :e 4 :f {:g 11 :h 12}}})Concise Java Object testing. Inevitably, I seem to end up with a few Java objects. I could write a bunch of different expect statements, but I opted for a syntax that allows me to check everything at once.(given (java.util.ArrayList.)
(expect
.size 0
.isEmpty true))Trimmed Stacktraces. I'm sure it's helpful to look through Clojure and Java's classes at times. However, I find the vast majority of the time the problem is in my code. Expectations trims many of the common classes that are from Clojure and Java, leaving much more signal than noise. Below is the stacktrace reported when running the failure examples from the Expectations codebase.ailure in (failure_examples.clj:8) : failure.failure-examples
raw: (expect 1 (one))
act-msg: exception in actual: (one)
threw: class java.lang.ArithmeticException-Divide by zero
failure.failure_examples$two__375 (failure_examples.clj:4)
failure.failure_examples$one__378 (failure_examples.clj:5)
failure.failure_examples$G__381__382$fn__387 (failure_examples.clj:8)
failure.failure_examples$G__381__382 (failure_examples.clj:8)Every stacktrace line is from my code, where the problem lives.

Descriptive Error Messages. Expectations does it's best to give you all the important information when a failure does occur. The following failure shows what keys are missing from actual and expected as well is which values do not match.running

(expect
{:z 1 :a 9 :b {:c Double/NaN :d 1 :e 2 :f {:g 10 :i 22}}}
{:x 1 :a Double/NaN :b {:c Double/NaN :d 2 :e 4 :f {:g 11 :h 12}}})

generates

failure in (failure_examples.clj:110) : failure.failure-examples
raw: (expect {:z 1, :a 9, :b {:c Double/NaN, :d 1, :e 2, :f {:g 10, :i 22}}} {:x 1, :a Double/NaN, :b {:c Double/NaN, :d 2, :e 4, :f {:g 11, :h 12}}})
result: {:z 1, :a 9, :b {:c NaN, :d 1, :e 2, :f {:g 10, :i 22}}} are not in {:x 1, :a NaN, :b {:c NaN, :d 2, :e 4, :f {:g 11, :h 12}}}
exp-msg: :x is in actual, but not in expected
:b {:f {:h is in actual, but not in expected
act-msg: :z is in expected, but not in actual
:b {:f {:i is in expected, but not in actual
message: :b {:e expected 2 but was 4
:b {:d expected 1 but was 2
:b {:f {:g expected 10 but was 11
:a expected 9 but was NaNnote: I know it's a bit hard to read, but I wanted to cover all the possible errors with one example. In practice you'll get a few messages that will tell you exactly what is wrong.

For example, the error tells you:b {:f {:g expected 10 but was 11With that data it's pretty easy to see the problem in(expect
{:z 1 :a 9 :b {:c Double/NaN :d 1 :e 2 :f {:g 10 :i 22}}}
{:x 1 :a Double/NaN :b {:c Double/NaN :d 2 :e 4 :f {:g 11 :h 12}}})Expectations also tells you, when comparing two lists:
if the lists are the same, but differ only in order
if the lists are the same, but one list has duplicates
if the lists are not the same, which list is larger

JUnit integration. My project uses both Java and Clojure. I like running my tests in IntelliJ and I like TeamCity running my tests as part of the build. To accomplish this using Expectations all you need to do is create a java class similar to the example below.import expectations.junit.ExpectationsTestRunner;
import org.junit.runner.RunWith;

@RunWith(expectations.junit.ExpectationsTestRunner.class)
public class FailureTest implements ExpectationsTestRunner.TestSource{

public String testPath() {
return "/path/to/the/root/folder/holding/your/tests";
}
}The Expectations Test Runner runs your Clojure tests in the same way that the Java tests run, including the green/red status icons and clickable links when things fail.

Why wouldn't you use Expectations?

Support. I'm using it to test my production code, but if I find errors I have to go fix them. You'll be in the same situation. I'll be happy to fix any bugs you find, but I might not have the time to get to it as soon as you send me email.

If you're willing to live on the bleeding edge, feel free to give it a shot.
© Jay Fields - www.jayfields.com

Clojure: Mocking

An introduction to clojure.test is easy, but it doesn't take long before you feel like you need a mocking framework. As far as I know, you have 3 options.Take a look at Midje. I haven't gone down this path, but it looks like the most mature option if you're looking for a sophisticated solution.

Go simple. Let's take an example where you want to call a function that computes a value and sends a response to a gateway. Your first implementation looks like the code below. (destructuring...

Clojure: Using Sets and Maps as Functions

Clojure sets and maps are functions.

Since they are functions, you don't need functions to get values out of them. You can use the map or set as the example below shows.(#{1 2} 1)
> 1

({:a 2 :b 3} :a)
> 2
That's nice, but it's not exactly game changing. However, when you use sets or maps with high order functions you can get a lot of power with a little code.

For example, the following code removes all of the elements of a vector if the element is also in the set.(def banned #{"Steve" "Michael"})

clojure.test Introduction

I'll admit it, the first thing I like to do when learning a new language is fire up a REPL. However, I'm usually ready for the next step after typing in a few numbers, strings and defining a function or two.

What feels like centuries ago, Mike Clark wrote an article about using unit testing to learn a new language. Mike was ahead of his time. This blog entry should help you if you want to follow Mike's advice.

Luckily, Clojure has built in support for simple testing. (I'm currently using Cloju...

Clojure: Destructuring

In The Joy of Clojure (TJoC) destructuring is described as a mini-language within Clojure. It's not essential to learn this mini-language; however, as the authors of TJoC point out, destructuring facilitates concise, elegant code.

What is destructuring?

Clojure supports abstract structural binding, often called destructuring, in let binding lists, fn parameter lists, and any macro that expands into a let or fn. -- http://clojure.org/special_forms

The simplest example of destructuring is assignin...

Clojure: Composing Functions

Before Clojure, I had never used a Functional Programming language. My experience has primarily been in C#, Ruby, & Java. So, learning how to use Clojure effectively has been a fun and eye-opening experience.

I've noticed a few things:The Clojure language has a lot of built in support for maps and hashes (literal syntax, destructuring, etc)Rich Hickey is quoted as saying "It is better to have 100 functions operate on one data abstraction than 10 functions on 10 data structures." Clojure...

High Level Testing with a High Level Language

In the early days of my project we made the decision to high-level test our Java application with Clojure*. One and a half years later, we're still following that path. It seemed worthwhile to document the progress so far.

My current preferred style of testing is rigorous unit testing and less than a dozen high level tests.

This style of testing doesn't catch everything; however, context is king. In my context, we constantly have to balance the number of bugs against the speed at which we deli...

Sacrificing some IDE capabilities

When discussing adopting Clojure or JRuby there are often heated discussions concerning the pros and cons of working with a language that has less IDE support. This blog entry will focus on the common concerns I hear.

I've never met anyone who has mastered Ruby or Clojure and had the opinion that they are more productive in Java. I think that's the best "proof" that the pros outweigh the cons.*

Other than that, things get complicated. The problem is, I understand where reluctant adopters are co...

Clojure: Converting a Custom Collection to a Sequence

I've been doing a bit of clojure lately and I've often found myself looking to convert a custom collection to a sequence. Unfortunately, several custom collections from libraries that we use don't implement Iterable; therefore, I can't simply use the seq function to create a seq.

In Java it's common to simply use a for loop and a get method to iterate through custom collections that don't implement Iterable.
for (int i=0; i<fooCollection.size(); i++) {
Foo foo = fooCollection.get(i);
}There are...