Pretty image
Aaron is the coauthor (with Stuart Halloway) of the forthcoming Programming Clojure, Second Edition. Here he gives a practical, hands-on experience with Clojure.

There are many reasons for turning to Clojure to solve problems. Clojure is a general purpose programing language, based on Lisp, that runs on the Java Virutual Machine. It would take up all of your attention, and a lot of your time to go through them all, so you should browse the videos for yourself. One of Clojure’s selling points is its ability to interoperate with any Java code. Billions of dollars have been invested into building supporting infrastructure and libraries around the Java stack, and it is trivial for you to tap into them using Clojure. This article will dive right in to writing Cloure code with a dash of Java interop.

For this article, we will use the Leiningen build tool. There are fantastic installation instructions right on the github project page. Leinigen works on Linux, OS X, and Windows, so you should be covered. Before starting this article, make sure you have the current (1.5.2 or greater) version of Leiningen installed.

In this example we will be building an application to test the availability of websites. The goal here is to check to see if the website returns an HTTP 200 OK response. If anything other than our expected response is received, it should be noted. Let’s start by creating a new project.

 lein new pinger

Open your project.clj file and modify the contents to match what we are going to be working on. Be sure to update Clojure to the latest version.

 (defproject pinger "0.0.1-SNAPSHOT"
  :description "A website availability tester"
  :dependencies [[org.clojure/clojure 1.3.0-beta1]])

Grab the dependencies by running lein deps.

 lein deps

First we need to write the code that connects to a url and captures the response code. We can accomplish this by using Java’s URL Library.

 (ns pinger.core
  (:import (java.net URL)))
 (defn response-code [address]
  (let [connection (.openConnection (URL. address))]
  (doto connection
  (.setRequestMethod "GET")
  (.connect))
  (.getResponseCode connection)))
 (response-code "http://google.com")
 -> 200

Now let’s create a function that uses response-code and decides if the specified url is available. We will define available in our context as returning an HTTP 200 response code.

 (defn available? [address]
  (= 200 (response-code address)))
 (available? "http://google.com")
 => true
 (available? "http://google.com/badurl")
 => false

Next we need a way to start our program and have it check a list of urls that we care about every so often and report their availability. Let’s create a main function.

 (defn -main []
  (let [addresses '("http://google.com"
  "http://amazon.com"
  "http://google.com/badurl")]
 (while true
  (doseq [address addresses]
  (println (available? address)))
  (Thread/sleep (* 1000 60)))))

In this example we create a list of addresses (two good and one bad), and use a simple while loop that never exits to simulate a never-ending program execution. It will continue to check these urls once a minute until the program is terminated. Since we are exporting a -main function, don’t forget to add :gen-class to your namespace declaration.

 (ns pinger.core
  (:import (java.net URL))
  (:gen-class))

Now that we have the fundamentals in place we need to tell leningen where our main function is located. Open up project.clj and add the :main declaration:

 (defproject pinger "0.0.1-SNAPSHOT"
  :description "A website availability tester"
  :dependencies [[org.clojure/clojure "1.3.0-beta1"]]
  :main pinger.core)

It’s time to compile our program into a jar and run it. To do this, run

 lein uberjar
 java -jar pinger-0.0.1-SNAPSHOT-standalone.jar
 true
 false
 true

You should see your program start and continue to run until you press ctrl-c to stop it.

Adding real continuous loop behavior

A while loop that is always true will continue to run until terminated, but it’s not really the cleanest way to obtain the result as it doesn’t allow for a clean shutdown. We can use a scheduled thread pool that will start and execute the desired command in a similar fashion as the while loop, but with a much greater level of control. Create a file in the src directory called scheduler.clj and enter the following code:

 (ns pinger.scheduler
  (:import (java.util.concurrent ScheduledThreadPoolExecutor TimeUnit)))
 (def ^:private num-threads 1)
 (def ^:private pool (atom nil))
 (defn- thread-pool []
  (swap! pool (fn [p] (or p (ScheduledThreadPoolExecutor. num-threads)))))
 (defn periodically
  "Schedules function f to run every 'delay' milliseconds after a
  delay of 'initial-delay'."
  [f initial-delay delay]
  (.scheduleWithFixedDelay (thread-pool)
  f
  initial-delay delay TimeUnit/MILLISECONDS))
 (defn shutdown
  "Terminates all periodic tasks."
  []
  (swap! pool (fn [p] (when p (.shutdown p)))))

This code sets up a function called periodically that will accept a function, initial-delay, and repeated delay. It will execute the function for the first time after the initial delay then continue to execute the function with the delay specified thereafter. This will continue to run until the thread pool is shut down. Since we have a handle to the thread pool, we can do this gracefully via the shutdown function.

Let’s update our application to take advantage of the scheduling code as well as make the -main function only responsible for calling a function that starts the loop.

 (defn check []
  (let [addresses '("http://google.com"
  "http://google.com/404"
  "http://amazon.com")]
  (doseq [address addresses]
  (println (available? address)))))
 (def immediately 0)
 (def every-minute (* 60 1000))
 (defn start []
  (scheduler/periodically check immediately every-minute))
 (defn stop []
  (scheduler/shutdown))
 (defn -main []
  (start))

Make sure to update your namespace declaration to include the scheduler code:

 (ns pinger.core
  (:import (java.net URL))
  (:require [pinger.scheduler :as scheduler])
  (:gen-class))

Not everything in the previous sample is necessary, but it makes for more readable code. Adding the start and stop functions makes it easy to work interactively from the REPL which will be a huge advantage should you choose to extend this example. Give everything one last check by running lein uberjar and executing the jar. The program should function exactly as it did before.

Logging

So far we have produced a program capable of periodically checking the availability of a list of websites. It is, however, lacking the ability to keep track of what it has done and to notify us when a site is unavailable. We can solve both of these issues with logging. There are a lot of logging options for Java applications, but for this example we will use log4j. It gives us a real logger to use, and it gives us email notification. This is great because we will have the ability to send email alerts when a website isn’t available. In order to do this we will need to pull log4j and mail into our application. To make it easier to take advantage of log4j we will also pull in clojure.tools.logging. Open your project.clj file and add clojure.tools.logging, log4j, and mail:

 (defproject pinger "0.0.1-SNAPSHOT"
  :description "A website availability tester
  :dependencies [[org.clojure/clojure "1.3.0-beta1"]
  [org.clojure/tools.logging "0.1.2"]
  [log4j "1.2.16"]
  [javax.mail/mail "1.4.1"]]
  :main pinger.core)

and pull the dependencies in with leiningen:

 lein deps

The great part about the clojure logging library is that it will use any standard Java logging library that is on the classpath so there is no additional wiring required between log4j and your application. Create a folder in the root of your project called resources. Leiningen automatically adds the contents of this folder to the classpath, and you will need that for your log4j properties file. Create a file under the resources directory named log4j.properties and add the following contents:

 log4j.rootLogger=info, R, email
 log4j.appender.R=org.apache.log4j.RollingFileAppender
 log4j.appender.R.File=logs/pinger.log
 log4j.appender.R.MaxFileSize=1000KB
 log4j.appender.R.MaxBackupIndex=1
 log4j.appender.R.layout=org.apache.log4j.PatternLayout
 log4j.appender.R.layout.ConversionPattern=%d{ISO8601} %-5p [%c] - %m%n
 log4j.appender.email=org.apache.log4j.net.SMTPAppender
 log4j.appender.email.SMTPHost=localhost
 log4j.appender.email.From=system@yourapp.com
 log4j.appender.email.To=recipient@yourapp.com
 log4j.appender.email.Subject=[Pinger Notification] - Website Down
 log4j.appender.email.threshold=error
 log4j.appender.email.layout=org.apache.log4j.PatternLayout
 log4j.appender.email.layout.conversionPattern=%d{ISO8601} %-5p [%c] - %m%n

This sets up standard logging to pinger.log and will send an email notification for anything logged as error, which in our case is when a website doesn’t respond with an HTTP 200 response or when an exception is thrown while checking the site. Make sure to change the email information to something that works in your environment.

Let’s update the code and add logging. The goal here is to replace any println statements with log messages. Open core.clj, add the info and error functions from clojure.tools.logging into your namespace declaration, and create a function to record the results.

 (ns pinger.core
  (:import (java.net URL))
  (:use [clojure.tools.logging :only (info error)])
  (:require [pinger.scheduler :as scheduler])
  (:gen-class))
 
 ...
 (defn record-availability [address]
  (if (available? address)
  (info (str address " is responding normally"))
  (error (str address " is not available"))))

Also update check to reflect the changes:

 (defn check []
  (let [addresses '("http://google.com"
  "http://google.com/404"
  "http://amazon.com")]
  (doseq [address addresses]
  (record-availability address))))

Rebuild and try your program again. You should notice a newly created logs directory that you can check for program execution. You should also notice an email come in with an error message. If you get a “connection refused” error on port 25, you will need to set up a mail transport agent on your machine to enable mail sending. You now have a way to notify people of a website failure!

Configuration

We have hard-coded our list of websites to monitor, and that simply won’t work! We need a way to give a list of sites to monitor from some external source. We could use a properties file, database or webservice to accomplish this. For ease of explanation we will go with a properties file. Create a file named pinger.properties in the root of the application and add the following to it:

 urls=http://google.com,http://amazon.com,http://google.com/404

We need a way to load this file in and create a collection of sites to feed into the check function. Create a file named config.clj in the src directory:

 (ns pinger.config
  (:use [clojure.java.io :only (reader)])
  (:import (java.util Properties)))
 (defn load-properties []
  (with-open [rdr (reader "pinger.properties")]
  (doto (Properties.)
  (.load rdr))))
 (def config (load-properties))

As long as pinger.properties is on the classpath, the previous example will read pinger.properties into a Java properties object. Since we don’t want to do this every time we run the website checking routine, we create a var to hold the value for us. All we have left to do is get the url’s attribute and put it into a list. Add the following function into the config namespace:

 (defn urls []
  (str/split (.get config "urls") #","))

Make sure to require clojure.string in your namespace declaration

 (ns pinger.config
  (:use [clojure.java.io :only (reader)])
  (:require [clojure.string :as str])
  (:import (java.util Properties)))

Finally, update the check function in core.clj to use the new configuration function.

 (ns pinger.core
  (:import (java.net URL))
  (:use [clojure.tools.logging :only (info error)])
  (:require [pinger.scheduler :as scheduler]
  [pinger.config :as config])
  (:gen-class))
 
 ...
 (defn check []
  (doseq [address (config/urls)]
  (record-availability address)))

Rebuild your application with leiningen and give it a try. Remember to put pinger.properties on the classpath:

 java -cp pinger.properties:pinger-0.0.1-standalone.jar pinger.core

Wrapping Up

We now have what we need to succeed. In this example we covered:

  • Using Java’s URL to check a website to see if it was available

  • Using Java’s ScheduledTheadPoolExecutor to create a periodically running task

  • Using log4j with clojure.tools.logging to send error notifications

  • Using Java’s property system for configuration

  • Using leiningen to create standalone executable jars

There are quite a few things you could do to expand on this example. We could redefine what it means for a website to be available by adding requirements for certain HTML elements to be present, or for the response to return in a certain time to cover an SLA. Try adding to this example and see what you can come up with.

Aaron Bedra is a developer at Relevance and a member of Clojure/core, where he uses Clojure to solve hard, important problems. He is the co-author of Programming Clojure, Second Edition.

Send the author your feedback or discuss the article in the magazine forum.