Rodrigo Flores's Corner Code, Cats, Books, Coffee

Book Review: Clojure Applied

The following blog post is about the book Clojure Applied written by Ben Vandgrift and Alex Miller. I have not received a free copy of the book to review it.

As Clojure is becoming more and more popular these days, we have a lot of programmers learning to write code in Clojure, and almost all books tell you how each part works: functions, macros, maps, lists, records, multi-methods, agents, refs, transducers, reducers. But how can you apply all these concepts together ?

I like to think of Clojure like a infinite Lego box: you can build your system using the pieces, but can you make smart choices to select the best tool for each part of your system ? Some parts appear to be similar: lists and vectors, maps and records, multi-methods and protocols, but how do I select one instead of another ?

Clojure Applied is the first Clojure Book I read which wasn't targeted for people initiating in the Clojure world, so it considers that you have some experience. I like to think of it like a book of answers to questions that if you deal with Clojure frequently you will probably wonder about. I particularly liked the chapter 6 "Creating Components": on how to structure your application in an organized way.

Another cool thing is that it not only covers features of the language but also some well-known parts of the ecosystem like the core async library, Stuart Sierra's component library, the schema library and transit and EDN as data serializers. It also covers how to deploy your code to a production server using a platform, a IAAS or using your own servers.

I strongly recommend you to not read this book cover to cover, but to try to write a small application after each chapter applying what you have learned: I did it this way and I really think it was worth it.

A Clojure application generator with Midje

If you're bootstraping a new Clojure application, you would run this command:

lein new app my-awesome-app

And then Leiningen would generate a directory containing the bare minimum to make your application build (with a Hello World example). However, everytime I did that for experimenting purposes, I found that I've always added two libraries: Midje and Schema, so to help me stop doing this repetitive work, I created a template (aka generator) for that. Doing this, I can bootstrap an application with both libraries by just running this command:

lein new app-with-midje my-awesome-app

Midje

Midje is a testing framework for Clojure. I like the way tests are written in Midje:

(facts "average of items
  (fact "average of 1 item is itself"
    (average 1) => 1)
  (fact "average of 2 items is the minor item plus half the distance"
    (average 1 5) => 3)
  (tabular
    (fact "calculates average"
      (average ?a ?b ?c) => ?average)
    ?a ?b ?c ?average
    1  9  11 7
    3  12 30 15
    30 60 90 60))

They're very similar to what you would describe as input and output examples of a function. Also, I think tabular examples shows clearly what you would expect on each case, without having to duplicate code on each fact. It is also possible to mock and redef other functions on a test scope, so you can create isolated unit tests, and it is also possible to build your own checks, so you can create better tests based on your domain. So, in general, I consider Midje to be a great testing library to work with.

Schema

Clojure is not a typed language, so it is fairly common to see maps being used as a kind of typed data. Suppose you have this map describing a person:

(def person {:name "Joe Doe"
             :age  45
             :team "Blackburn Rovers"})

(def another-person {:name "Jane Doe"
                     :age  48})

For some reason, you want to give Joe and Jane a football jersey of their favourite football team. However, you forgot that this field may be not filled, as it is not mandatory for everyone to have a football team, so you wrote your function like this:

(defn ship-football-shirt [customer]
  (ship-shirt customer (buy-shirt (:team customer)))

As you forgot that it is a required, when you call the function with Joe, it works because he has a football team, but when you for Jane, it gives you a NullPointerException (Clojure normally raises a NPE you treat a nil like a map). How to get over this kind of issue ?

To help deal with this kind of problem, there is a library called Schema. After specifying it on your project.clj, you can instantiate on your namespaces and use this way

(require '[schema.core :as s])

(def teams #{"Blackburn Rovers" "Leicester United" ... } )
(def all-teams-ever (s/enum teams)) ; I'm not writing the name of all teams ever


(def Person {(s/required-key :name) s/Str
             (s/required-key :age)  s/Int
             :team                  all-teams-ever})

(s/validate Person person)

To declare a map schema, you should insert the keys and the accepted values for each key. Optionally, you can say that a key is required. If you call the s/validate function with a schema and a map, it will try to validate the type of each value and also the presence of all required keys, throwing an exception in case something doesn't validate. On our example, it requires a person to have a :name key with any string value, an :age key with any integer value and optionally a team with any value specified in the set teams.

To help fix our problem, we can create a derived schema call FootballFan:

(def FootballFan (assoc (dissoc Person :team) (s/required-key :team) all-teams-ever))

And then you can rewrite ship-football-shirt this way:

(s/defn ship-football-shirt [customer :- FootballFan]
  (ship-shirt customer (buy-shirt (:team customer)))

The :- symbol means that the customer symbol should be validated with that schema. When you call the function above inside the body of the macro (s/with-fn-validation ) it will trigger an exception. Why is this function validation optional ? Performance reasons mostly: makes sense to have this check turned on throughout your code on a testing environment but on production it might spend valuable time checking schemas (but a validate call on strategic places like before inserting something to a database or when you're receiving data from and HTTP request definitely makes sense).

Will it avoid receiving an exception ? Absolutely not, you will still get an exception, but this time you will receive a specific error telling you that Jane is missing that key/value. This kind of validation avoid not only strange errors, but you also don't need to program defensively inside ship-football-shirt (as you specified that you only accept maps that validate with that schema).

The App with Midje template

As I use both libraries on almost every application, I created a template for that: a set of files and folders generated given a project name. So, after adding it to my lein profile, I can call lein new app-with-midje awesome-project and it will generate a project.clj with both libraries, an APACHE V2 license,a README, a .gitignore file, a core.clj file to write the application (and a correspondent test file), a repl.clj file that serves as a REPL wrapper to load libraries and functions (like an the autotesting namespace) for general REPL there.

To use the generator, add on your ~/.lein/profiles.clj the plugin and the version (in case you already have the plugin vector, just append it): {:user {:plugins [[app-with-midje/lein-template "0.1.0"]]}}

And then, to generate the application:

lein new app-with-midje my-awesome-app

You can also check the code for the template on Github.

Clojure Ecosystem

One of the things I dislike the most when learning a Programming Language is how frequent you get stuck with something. When dealing with something different than what we're used to deal with every day we're pushed out of our confort zone, which is good, but unconfortable. This blog post will give you a few tips on tools that usually help me when I'm writing Clojure code.

REPL

REPL is an acronym for Read Eval Print Loop: which means a prompt where you send expressions and it prints the result of that expression. Rubysts are probably familiar with irb or pry and Pythonists are familiar with Python interactive interpreter that comes bundled with the language (just run python without arguments to see it).

To run it, execute lein repl on a Leiningen project directory and you shall see an output like this one:

nREPL server started on port 57755 on host 127.0.0.1 - nrepl://127.0.0.1:57755
REPL-y 0.3.7, nREPL 0.2.10
Clojure 1.7.0
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

test.core=>

Just like a shell, it is waiting for your command: you can type:

(+ 2 3)

And it will simply output:

5

And then it will wait for yout next command. So it read the expression (+ 2 3), evaluated the expression to 5 and then output the result: 5.

An useful function is called doc, it will print the docs for a function:

test.core=> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
  Returns the sum of nums. (+) returns 0. Does not auto-promote
  longs, will throw on overflow. See also: +'
nil

What happened here? doc is a function with a side-effect: it prints the documentation for a given function. When you call it with + it prints the documentation for the + function and outputs the return value for the function which is nil.

Docs

As you have noticed, the first way to find docs about a function is through the doc command. It is a succint description, going straight to the point on what it does along with the different arities (an arity is a possible number of arguments that a function accepts, naturally you can sum 0 up to infinite number of integers, but a function may only allow a limited number of arities, like the doc function that only accepts one arity). However, sometimes you don't remember the function name, or an usage example could be useful, on these cases you can use other alternatives.

On Clojure.org there is an online version of what the doc function prints. It may be more visually attactive as the arities are on different lines, but it is essentially the same thing. Clojure.org also provides a really useful cheatsheet, that may also be useful as it provides a quick reference for the most common used functions. If you have already played with another language, you will be familiar with the meaning of most of the functions here. Each entry links to the documentation of that function.

Clojure docs is similar to the documentation on the Clojure website, but also contains examples: you can check here how the arities work: you can call + with an arbitrary number of arguments and it will work.

There is also Grimoire, that is similar to clojure docs, but also contains a link to the function source.

Learning

Books

I had my first contact with Clojure through Bruce Tate's book: Seven Languages in Seven Weeks. It contained a short chapter (to be read and explored within a week) about Clojure. I really liked the reading, but it couldn't of course go deeply into Clojure. So, my first book to learn Clojure was Stuart Halloway's and Aaron Bedra Programming Clojure: a really nice book that teaches you not only the Clojure language, but also the Clojure Way. And speaking about Clojure Way, my next reading was Michael Fogus Joy of Clojure a book maybe more targeted to someone with already some knowledge on the language as it goes deeply into concepts and more subtle things.

There is also good options that were recommended by some friends: Clojure for the Brave and the True, Clojure Programming, Clojure Programming and Living Clojure: this last one deserves a special highlight, it contains a training guide to learn Clojure (similar to a Running program like Couch to 5k but for Clojure).

Online

There are a few websites or projects to learn Clojure online. Back on my first days I tried Clojure Koans and I really liked: is some kind of filling the gaps exercise to make some expressions to be true. Now I see that there are other cool online resources: Try Clj: to try Clojure with an interactive tutorial of the language and 4Clojure with a similar experience compared to Clojure Koans but you do both without even installing anything on your computer.

Lastly, there is Wonderland Clojure Katas, a series of proposed puzzles to be solved with Clojure: the exercises come with a failing test suite and you should build code to make it pass.

Testing

After some years of Ruby, I discovered that the best thing you can do with your code is to write tests for it. It works like a safety net: no matter what you do with your code, if the tests are passing you can be sure that it behaves the way it should (or at least the way it was before the changes). And as learning a programming language requires experimentation, testing is a fast way to see if that new thing works the same way than that not-so idiomatic thing you did before.

The Clojure language itself has a test framework called Clojure Test. It comes bundled with the language, so the only thing you need to do is require the namespace and write your tests. To try Clojure Test, you can do this on a REPL:

(require '[clojure.test :refer [deftest is run-all-tests]])

(deftest one-is-one (is (= 1 1)))
(deftest two-is-one (is (= 2 1)))

(run-all-tests)

The first line requires the test namespace and the :refer [deftest ...] part is just a way to avoid having to call the namespace (or an alias) everytime you call something of that namespace. Then we define the tests: the deftest registers your test expressions as tests and run-all-tests looks up for this definitions and calls it. To assert the value of something you can use the is macro (for now, think on a macro as a special type of function that gets evaluated before the code is run: we will get on macros soon): it will show a summary of the test run and also output all failed tests (the falsy expressions).

FAIL in (two-is-one) (form-init7127606434032677106.clj:1)
expected: (= 2 1)
  actual: (not (= 2 1))

Ran 2 tests containing 2 assertions.
1 failures, 0 errors.
{:test 2, :pass 1, :fail 1, :error 0, :type :summary}

Another popular testing framework is Midje. It has a quite simple syntax fn-call => expected result. After setting up midje (as it isn't bundled in the language you will need to include it on your project), you can write tests this way:

(facts "sum function"
       (fact "one + one is two"
             (+ 1 1) => 2)
       (fact "one + two is four"
             (+ 1 2) => 4))

And the error display is really nice (and colourful):

> (require '[midje.repl :refer :all])
> (autotest)
======================================================================
Loading (test-app-midje.core test-app-midje.core-test)

FAIL "sum function - one + two is four" at (core_test.clj:9)
    Expected: 4
    Actual: 3
FAILURE: 1 check failed.  (But 1 succeeded.)

There is also plenty of other cool features: mocking, loose checkers, exception thrower checker and even the possibility to write custom checking functions. The project wiki on Github also contains some documentation on how to get started with Midje.

Editors & IDEs

As Clojure is becoming more popular each day, I'm pretty sure that almost all popular editors and IDEs may have support for Clojure syntax (or at least a plugin for it). I've already written Clojure code in Vim, Lighttable, Emacs, Sublime Text, Atom and IntelliJ with Cursive. I also know that Eclipse also have a plugin for Clojure called Counterclockwise.

But as being a dialect of LISP, you may need more things on an editor/IDE than simply syntax highlighting and REPL integration: you will need a smarter way to deal with open and close parenthesis/brackets/quotes. One of the things that make me more productive in Clojure, is the ability to deal with open and close parenthesis in a more structured way that is called Paredit or Structural Editing. This feature automatically adds a closing parenthesis/bracket/quote when you add a opening parenthesis/bracket/quote. Also, it has a few more commands to manipulate those expresions in order to be able to edit your code more efficiently. You can see a few examples of commands on this blog post.

Of course, this is not mandatory: you can write Clojure without using this techniquebut if you want to be really productive with Clojure, learning this technique is important. But feel free to skip it for now: I would only ask one thing: try to see if it exists on your favourite editor: if it does, that's fantastic, if it does not, another editor might be a good choice for your Clojure adventure.

Libraries

The best place to find a library is Clojars: it is an equivalente of RubyGems/CPAN for Clojure: there is a search where you can search for the library name or for a term like http, soap or aws, and it will return all the libraries on clojars that match that term. Also, due to Clojure interoperabilty with Java, you can also include and use libraries from Maven.

After finding the library, you can add it to your project through the project.clj dependencies (I explained how to do so on First Steps with Clojure post.

Community

Just like all programming language communities, the Clojure community has some mailing lists, a Planet, an IRC channel and some wikis. You can check a list of these resources on the community page of the Clojure website. Another source for Clojure news and discussions is the Clojure Subreddit where people usually post links, ask questions and discuss. From time to time, there is a thread called "Ask Anything", where new Clojurists can ask any questions.

Videos

You can also check some Video resources:

Summary

The goal of this blog post is to talk about some useful resources when learning Clojure. So, I hope you found your way through the REPL, was able to write tests, found a good editor, learned where to search for libraries and where to ask questions and discuss about the language.

Next blog post I will go deeper into Leiningen: the idea is to create a boilerplate for Clojure apps.