Use of Mutable Data Structures in Clojure

One of Clojure’s biggest strengths is that it is backed by the JVM, and has good interoperability with the Java libraries.  When I needed to implement a simulation with events that should be executed in order of their timestamp, I was immediately tempted to use Java’s PriorityBlockingQueue, since neither sorted-map nor sorted-set supported two events with the same time stamp (though I could have used some magic to make it work, but decided simplicity was more important).

I was generating events with several types, and under certain circumstances, events that mapped :type to :arrival needed to be removed because they were no longer valid given the state of the simulation.  Easy enough:

(filter #(not= (:type %) :arrival) event-queue)

But, PriorityBlockingQueue is mutable (oh, and I’d put it in a ref, to make things more interesting), so I’d need to clear it after I did the filter, before repopulating it with the filtered events.  This is a simpler version of the final code:

(let [elements (filter #(not= (:type %) :arrival) @event-queue)]
(.clear @event-queue)
(.addAll @event-queue elements))

So, when I ran this, I kept getting NullPointerException.  Clojure is not well known for its uber-informative stack traces, so I flailed with this for a few hours, looking at it on and off while working on other parts of the code.  Then it hit me: filter is lazy, so the filtering of the elements is not happening until I’m calling in the filter when I make the addAll call.  By that time, @event-queue is empty, so the list of results is nil, which will throw a NullPointerException when passed to the addAll call.  The easy way to fix this is to wrap the call in a doall function, which forces the computation:

(doall (filter #(not= (:type %) :arrival) event-queue))

The moral?  Even though Clojure has good support of Java collections, be wary when you use mutable data structures with the standard Clojure seq functions; many of them are lazy, and unless you know they won’t change, you can get subtle temporal errors creeping into your code.  I got lucky that addAll threw the exception.  If it hadn’t my simulation would have ended and I would have spent a lot more time trying to figure out why.