Rodrigo Flores's Corner Code, Cats, Books, Coffee

A Clojure application generator with Midje

If you're bootstraping a new Clojure application, you would run this command:

lein new app my-awesome-app

And then Leiningen would generate a directory containing the bare minimum to make your application build (with a Hello World example). However, everytime I did that for experimenting purposes, I found that I've always added two libraries: Midje and Schema, so to help me stop doing this repetitive work, I created a template (aka generator) for that. Doing this, I can bootstrap an application with both libraries by just running this command:

lein new app-with-midje my-awesome-app

Midje

Midje is a testing framework for Clojure. I like the way tests are written in Midje:

(facts "average of items
  (fact "average of 1 item is itself"
    (average 1) => 1)
  (fact "average of 2 items is the minor item plus half the distance"
    (average 1 5) => 3)
  (tabular
    (fact "calculates average"
      (average ?a ?b ?c) => ?average)
    ?a ?b ?c ?average
    1  9  11 7
    3  12 30 15
    30 60 90 60))

They're very similar to what you would describe as input and output examples of a function. Also, I think tabular examples shows clearly what you would expect on each case, without having to duplicate code on each fact. It is also possible to mock and redef other functions on a test scope, so you can create isolated unit tests, and it is also possible to build your own checks, so you can create better tests based on your domain. So, in general, I consider Midje to be a great testing library to work with.

Schema

Clojure is not a typed language, so it is fairly common to see maps being used as a kind of typed data. Suppose you have this map describing a person:

(def person {:name "Joe Doe"
             :age  45
             :team "Blackburn Rovers"})

(def another-person {:name "Jane Doe"
                     :age  48})

For some reason, you want to give Joe and Jane a football jersey of their favourite football team. However, you forgot that this field may be not filled, as it is not mandatory for everyone to have a football team, so you wrote your function like this:

(defn ship-football-shirt [customer]
  (ship-shirt customer (buy-shirt (:team customer)))

As you forgot that it is a required, when you call the function with Joe, it works because he has a football team, but when you for Jane, it gives you a NullPointerException (Clojure normally raises a NPE you treat a nil like a map). How to get over this kind of issue ?

To help deal with this kind of problem, there is a library called Schema. After specifying it on your project.clj, you can instantiate on your namespaces and use this way

(require '[schema.core :as s])

(def teams #{"Blackburn Rovers" "Leicester United" ... } )
(def all-teams-ever (s/enum teams)) ; I'm not writing the name of all teams ever


(def Person {(s/required-key :name) s/Str
             (s/required-key :age)  s/Int
             :team                  all-teams-ever})

(s/validate Person person)

To declare a map schema, you should insert the keys and the accepted values for each key. Optionally, you can say that a key is required. If you call the s/validate function with a schema and a map, it will try to validate the type of each value and also the presence of all required keys, throwing an exception in case something doesn't validate. On our example, it requires a person to have a :name key with any string value, an :age key with any integer value and optionally a team with any value specified in the set teams.

To help fix our problem, we can create a derived schema call FootballFan:

(def FootballFan (assoc (dissoc Person :team) (s/required-key :team) all-teams-ever))

And then you can rewrite ship-football-shirt this way:

(s/defn ship-football-shirt [customer :- FootballFan]
  (ship-shirt customer (buy-shirt (:team customer)))

The :- symbol means that the customer symbol should be validated with that schema. When you call the function above inside the body of the macro (s/with-fn-validation ) it will trigger an exception. Why is this function validation optional ? Performance reasons mostly: makes sense to have this check turned on throughout your code on a testing environment but on production it might spend valuable time checking schemas (but a validate call on strategic places like before inserting something to a database or when you're receiving data from and HTTP request definitely makes sense).

Will it avoid receiving an exception ? Absolutely not, you will still get an exception, but this time you will receive a specific error telling you that Jane is missing that key/value. This kind of validation avoid not only strange errors, but you also don't need to program defensively inside ship-football-shirt (as you specified that you only accept maps that validate with that schema).

The App with Midje template

As I use both libraries on almost every application, I created a template for that: a set of files and folders generated given a project name. So, after adding it to my lein profile, I can call lein new app-with-midje awesome-project and it will generate a project.clj with both libraries, an APACHE V2 license,a README, a .gitignore file, a core.clj file to write the application (and a correspondent test file), a repl.clj file that serves as a REPL wrapper to load libraries and functions (like an the autotesting namespace) for general REPL there.

To use the generator, add on your ~/.lein/profiles.clj the plugin and the version (in case you already have the plugin vector, just append it): {:user {:plugins [[app-with-midje/lein-template "0.1.0"]]}}

And then, to generate the application:

lein new app-with-midje my-awesome-app

You can also check the code for the template on Github.

Clojure Ecosystem

One of the things I dislike the most when learning a Programming Language is how frequent you get stuck with something. When dealing with something different than what we're used to deal with every day we're pushed out of our confort zone, which is good, but unconfortable. This blog post will give you a few tips on tools that usually help me when I'm writing Clojure code.

REPL

REPL is an acronym for Read Eval Print Loop: which means a prompt where you send expressions and it prints the result of that expression. Rubysts are probably familiar with irb or pry and Pythonists are familiar with Python interactive interpreter that comes bundled with the language (just run python without arguments to see it).

To run it, execute lein repl on a Leiningen project directory and you shall see an output like this one:

nREPL server started on port 57755 on host 127.0.0.1 - nrepl://127.0.0.1:57755
REPL-y 0.3.7, nREPL 0.2.10
Clojure 1.7.0
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

test.core=>

Just like a shell, it is waiting for your command: you can type:

(+ 2 3)

And it will simply output:

5

And then it will wait for yout next command. So it read the expression (+ 2 3), evaluated the expression to 5 and then output the result: 5.

An useful function is called doc, it will print the docs for a function:

test.core=> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
  Returns the sum of nums. (+) returns 0. Does not auto-promote
  longs, will throw on overflow. See also: +'
nil

What happened here? doc is a function with a side-effect: it prints the documentation for a given function. When you call it with + it prints the documentation for the + function and outputs the return value for the function which is nil.

Docs

As you have noticed, the first way to find docs about a function is through the doc command. It is a succint description, going straight to the point on what it does along with the different arities (an arity is a possible number of arguments that a function accepts, naturally you can sum 0 up to infinite number of integers, but a function may only allow a limited number of arities, like the doc function that only accepts one arity). However, sometimes you don't remember the function name, or an usage example could be useful, on these cases you can use other alternatives.

On Clojure.org there is an online version of what the doc function prints. It may be more visually attactive as the arities are on different lines, but it is essentially the same thing. Clojure.org also provides a really useful cheatsheet, that may also be useful as it provides a quick reference for the most common used functions. If you have already played with another language, you will be familiar with the meaning of most of the functions here. Each entry links to the documentation of that function.

Clojure docs is similar to the documentation on the Clojure website, but also contains examples: you can check here how the arities work: you can call + with an arbitrary number of arguments and it will work.

There is also Grimoire, that is similar to clojure docs, but also contains a link to the function source.

Learning

Books

I had my first contact with Clojure through Bruce Tate's book: Seven Languages in Seven Weeks. It contained a short chapter (to be read and explored within a week) about Clojure. I really liked the reading, but it couldn't of course go deeply into Clojure. So, my first book to learn Clojure was Stuart Halloway's and Aaron Bedra Programming Clojure: a really nice book that teaches you not only the Clojure language, but also the Clojure Way. And speaking about Clojure Way, my next reading was Michael Fogus Joy of Clojure a book maybe more targeted to someone with already some knowledge on the language as it goes deeply into concepts and more subtle things.

There is also good options that were recommended by some friends: Clojure for the Brave and the True, Clojure Programming, Clojure Programming and Living Clojure: this last one deserves a special highlight, it contains a training guide to learn Clojure (similar to a Running program like Couch to 5k but for Clojure).

Online

There are a few websites or projects to learn Clojure online. Back on my first days I tried Clojure Koans and I really liked: is some kind of filling the gaps exercise to make some expressions to be true. Now I see that there are other cool online resources: Try Clj: to try Clojure with an interactive tutorial of the language and 4Clojure with a similar experience compared to Clojure Koans but you do both without even installing anything on your computer.

Lastly, there is Wonderland Clojure Katas, a series of proposed puzzles to be solved with Clojure: the exercises come with a failing test suite and you should build code to make it pass.

Testing

After some years of Ruby, I discovered that the best thing you can do with your code is to write tests for it. It works like a safety net: no matter what you do with your code, if the tests are passing you can be sure that it behaves the way it should (or at least the way it was before the changes). And as learning a programming language requires experimentation, testing is a fast way to see if that new thing works the same way than that not-so idiomatic thing you did before.

The Clojure language itself has a test framework called Clojure Test. It comes bundled with the language, so the only thing you need to do is require the namespace and write your tests. To try Clojure Test, you can do this on a REPL:

(require '[clojure.test :refer [deftest is run-all-tests]])

(deftest one-is-one (is (= 1 1)))
(deftest two-is-one (is (= 2 1)))

(run-all-tests)

The first line requires the test namespace and the :refer [deftest ...] part is just a way to avoid having to call the namespace (or an alias) everytime you call something of that namespace. Then we define the tests: the deftest registers your test expressions as tests and run-all-tests looks up for this definitions and calls it. To assert the value of something you can use the is macro (for now, think on a macro as a special type of function that gets evaluated before the code is run: we will get on macros soon): it will show a summary of the test run and also output all failed tests (the falsy expressions).

FAIL in (two-is-one) (form-init7127606434032677106.clj:1)
expected: (= 2 1)
  actual: (not (= 2 1))

Ran 2 tests containing 2 assertions.
1 failures, 0 errors.
{:test 2, :pass 1, :fail 1, :error 0, :type :summary}

Another popular testing framework is Midje. It has a quite simple syntax fn-call => expected result. After setting up midje (as it isn't bundled in the language you will need to include it on your project), you can write tests this way:

(facts "sum function"
       (fact "one + one is two"
             (+ 1 1) => 2)
       (fact "one + two is four"
             (+ 1 2) => 4))

And the error display is really nice (and colourful):

> (require '[midje.repl :refer :all])
> (autotest)
======================================================================
Loading (test-app-midje.core test-app-midje.core-test)

FAIL "sum function - one + two is four" at (core_test.clj:9)
    Expected: 4
    Actual: 3
FAILURE: 1 check failed.  (But 1 succeeded.)

There is also plenty of other cool features: mocking, loose checkers, exception thrower checker and even the possibility to write custom checking functions. The project wiki on Github also contains some documentation on how to get started with Midje.

Editors & IDEs

As Clojure is becoming more popular each day, I'm pretty sure that almost all popular editors and IDEs may have support for Clojure syntax (or at least a plugin for it). I've already written Clojure code in Vim, Lighttable, Emacs, Sublime Text, Atom and IntelliJ with Cursive. I also know that Eclipse also have a plugin for Clojure called Counterclockwise.

But as being a dialect of LISP, you may need more things on an editor/IDE than simply syntax highlighting and REPL integration: you will need a smarter way to deal with open and close parenthesis/brackets/quotes. One of the things that make me more productive in Clojure, is the ability to deal with open and close parenthesis in a more structured way that is called Paredit or Structural Editing. This feature automatically adds a closing parenthesis/bracket/quote when you add a opening parenthesis/bracket/quote. Also, it has a few more commands to manipulate those expresions in order to be able to edit your code more efficiently. You can see a few examples of commands on this blog post.

Of course, this is not mandatory: you can write Clojure without using this techniquebut if you want to be really productive with Clojure, learning this technique is important. But feel free to skip it for now: I would only ask one thing: try to see if it exists on your favourite editor: if it does, that's fantastic, if it does not, another editor might be a good choice for your Clojure adventure.

Libraries

The best place to find a library is Clojars: it is an equivalente of RubyGems/CPAN for Clojure: there is a search where you can search for the library name or for a term like http, soap or aws, and it will return all the libraries on clojars that match that term. Also, due to Clojure interoperabilty with Java, you can also include and use libraries from Maven.

After finding the library, you can add it to your project through the project.clj dependencies (I explained how to do so on First Steps with Clojure post.

Community

Just like all programming language communities, the Clojure community has some mailing lists, a Planet, an IRC channel and some wikis. You can check a list of these resources on the community page of the Clojure website. Another source for Clojure news and discussions is the Clojure Subreddit where people usually post links, ask questions and discuss. From time to time, there is a thread called "Ask Anything", where new Clojurists can ask any questions.

Videos

You can also check some Video resources:

Summary

The goal of this blog post is to talk about some useful resources when learning Clojure. So, I hope you found your way through the REPL, was able to write tests, found a good editor, learned where to search for libraries and where to ask questions and discuss about the language.

Next blog post I will go deeper into Leiningen: the idea is to create a boilerplate for Clojure apps.

First steps in Clojure

I remember the first time I had to deal with Clojure. I just came back after a job interview and I was asked to deliver a solution on a functional language to an exercise. Naturally, I opened a terminal and typed:

$ brew install clojure

brew is the package manager that install open source stuff on OS X. But instead of installing Clojure, homebrew returned this error:

Error: No available formula with the name "clojure"
Clojure isn't really a program but a library managed as part of a
project and Leiningen is the user interface to that library.

To install Clojure you should install Leiningen:
  brew install leiningen
and then follow the tutorial:
  https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md

Unlike Ruby or Python, Clojure does not have a native compiler for OSX or Linux: it runs on the Java Virtual Machine (also called JVM). So, you don't need to install a compiler, you just need to install Java, download a JAR and you should be ready to run Clojure.

Running Clojure from the JAR

But wait, the error said that I need to install Leiningen? Yes, we will get on Leiningen on a moment. But, for now let's run Clojure directly from a JAR. You can download the most recent version of the Clojure JAR from this website: http://clojure.org/downloads and unzip it. Then, you can run on a terminal:

java -cp clojure-1.7.0.jar clojure.main

This -cp option allows you to specify a list of directories, JAR archives and ZIP archives to the Java application launcher, the other argument clojure.main will be the namespace whose function -main will be called. You can also run a Clojure file like a script by passing the Clojure source file as another argument. To make it happen, I'm adding this content to a file called hello_world.clj:

(println "Hello world")

And I will run:

java -cp clojure-1.7.0.jar clojure.main hello.clj

And now you have run your first Clojure program. It is a simple program, the only dependency is on Clojure language. What if I want to write a program that needs to do an HTTP request? HTTP Kit is probably the most popular HTTP client in Clojure (and also can be used as a Server). Let's write a code to do a HTTP request:

(require '[org.httpkit.client :as http])
(println @(http/get "https://api.github.com/users/octocat/orgs"))

For now, no need to focus on the code, just trust me that it will work: I will explain what is this @ later. To run it, you will need something more: you will need to specify the http-kit JAR.

java -cp clojure-1.7.0.jar:http-kit-2.1.19.jar clojure.main hello.clj

And voilá, it should work.

Leiningen

Definitely this is not a straightforward way to run code. You will need to manually download JARs, and remember to insert all of them on your command to run your program. Happily, Leiningen can help us to make it really easy to bootstrap a Clojure application, to manage the dependency and to create different profiles (think of a profile as a "different" way to run your application). If you're familiar with Ruby on Rails, it does basically what Bundler (dependency management), Rake (make-like tool to run different profiles of your application) and Thor (code bootstrap generator). It also compiles your Clojure code into JVM bytecode and also can create a JAR file for you to export it. If you wabt to a comprehensive explanation on everything that Leiningen does, you can check this excellent article about it.

So, to bootstrap a Clojure application with Leiningen you first need to install Leinigen: you can do so by downloading a script, through a package manager or if you're a M$ Windows user, you can use a installer.

After installing it, you should be able to run

lein new app blog-example

lein is the command to invoke Leinigen, new tells Leinigen to bootstrap a new project, app is the template (this argument is optional and it defaults to a library project) and finally blog-example is the name of the application we're bootstrapping.

It should have generated something similar to this:

CHANGELOG.md    README.md   project.clj src
LICENSE     doc     resources   test

./doc:
intro.md

./resources:

./src:
blog_example

./src/blog_example:
core.clj

./test:
blog_example

./test/blog_example:
core_test.clj

To build our hello world application, let's for now focus on two files project.clj and src/blog_example/core.clj. project.clj is the project declaration: it contains the configuration that will be used to run your project, the name, description, the license information, etc.

(defproject blog-example "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.7.0"]]
  :main ^:skip-aot blog-example.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

Let's focus on two things on this file: :dependencies and :main: the first one is a vector (for now, think of vectors as the same as what is called as an array: I will explain more about sequences later) of all the dependencies along with the versions, and :main contains the namespace that will be run when you run lein run: the function that will actually run is the one called -main. Let's take a look at the file core.clj in the directory src/blog_example:

(ns blog-example.core
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!"))

The -main only contains a println call that outputs "Hello World", and this function will be called when you call lein run.

The coolest thing about Leiningen is that you don't even need to install Clojure to run the project: just install the java, Leiningen and the first time you run the program, it will install all the dependencies (including Clojure). Also, you don't even need Leiningen to run your program: you can easily pack it with lein uberjar and run with java -jar jarfile.jar and it will run your Clojure program. So, to summarize Leiningen helps a lot.

If you want to add the http-kit you can add it to the dependencies vector:

lein deps only install the dependencies: but if you run lein run and there is a missing dependency it will install it anyway.

Last, you can add the same code inside the -main function:

(ns blog-example.core
  (:require [org.httpkit.client :as http])
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!")
  (println @(http/get "https://api.github.com/users/octocat/orgs")))

And then running lein run will actually do the request.

Missing Parts

I know: I've skipped some parts to make your understanding of this basic tutorial easier. I will make a quickly explanation here of those parts:

  • @ signal on http call: this is related to the asynchronous nature of HTTP Kit: a call without to the http/get function will be asynchonous and a callback can be passed as an argument and the code inside the callback will be run once the request returns something. Using @ actually turns it synchronously: it will do the request and wait the result. This construction is called promise and I will cover it later.

  • project.clj: besides your app metadata like description and license, it also includes the dependencies, where your compiled code will be generated target-path, the main function (:main) that will be called on lein run and the profiles: that are different possible switches that you can enable or disable: like testing libraries, debug libraries and so on: by default the only profile tells that when generating an jar through the uberjar command it should do the ahead of time compilation. Remember this is just a subset of all possible options for a a Project.clj: it has lot of possible options and configurations and all of them are explained here.

  • core.clj: ns is the namespace declaration: it contains what namespaces should also be required and may also contain a gen-class that will generated a named Java class: this is useful here because the JVM requires a public main method and this gen-class directive will generate this method. Last, the -main function contain an & args as argument: it will contain the arguments given on the command line this & is actually a way to tell a clojure function that the function may receive an arbitrary number of arguments (besides the ones declared before the &) and will be stored on a vector.

Summary

So I hope you learned a few things here:

  • How to "install" Clojure
  • How to run Clojure code only with Java
  • How to install Leiningen
  • How to run code inside Leiningen
  • How to add a new dependency
  • How to require it on your library

Next time I will speak more about the Clojure ecosystem: what they are and how they work:

  • Where to look for Libraries;
  • Editor recommendations;
  • Project.clj overview;
  • Core.clj overview;