Etherplex

Just ran across (via Proggit) an interesting blog post about Core War, a game I played a bit with on Linux a few years back in which programs compete for dominance inside a virtual machine.  Very neat idea - you program an agent that will go and fight for you in a virtual world.  Cool.

What I didn’t know was that A.K. Dewdney wrote an article about Core War in Scientific American in 1984.  This is the same A.K. Dewdney that authored one of my very favorite science fiction/philosophy books ever, The Planiverse.  It turns out it is sort of a small world.

I mentioned a couple of posts ago about a technique for scoping state I used when I programmed in Scheme more often than I do now.  As I’ve been picking up Clojure as my hacking language of preference, I was surprised to find that it doesn’t support the same idioms.  In Clojure, any variables referenced when  a function is defined in a local scope are unbound when the function is returned to an outer scope.  This means the code I wrote in Scheme doesn’t have a direct analogue in Clojure.  This was a deliberate decision made by Rich Hickey, Clojure’s designer/author.  As he says:

If locals were variables, i.e. mutable, then closures could close over mutable state, and, given that closures can escape (without some extra prohibition on same), the result would be thread-unsafe. And people would certainly do so, e.g. closure-based pseudo-objects. The result would be a huge hole in Clojure’s approach.

The end result is that when you first pick up Clojure, if you try to close a function definition over a variable, you’ll find it is no longer bound when you leave the enclosing scope, and if you try to close over let-bound value (local), you’ll find that is immutable, and can therefore not be used to store state.  So how do we accomplish the same result as we did in Scheme?

You use Clojure’s shared transactional memory and create a mutable ref that is modified in a transaction to preserve thread-safety:

(let [z (ref 0)] (defn add [] (dosync (ref-set z (inc @z)))))

We still make use of local binding, but we bind z to a ref, which we can then modify inside of a dosync call.  Naturally, an arbitrary number of functions could be defined inside of the let that all shared a reference to z.  Once outside of the let, z is no longer accessible to the casual programmer, but the functions still have full access to it, creating a sort of data hiding/state-preserving closure that protects the “private” variable z.

I don’t use this idiom often, but when I want it, it’s exactly what I want, and avoids the need for more complex constructs to simply have a pair of functions share a variable that isn’t modifiable in other parts of the program.

My Emacs startup time was stretching beyond 10 seconds, which seemed a bit offputting even for a text editor cum operating system. I do have a fair amount of elisp loading at startup, however, and though I have played with byte-code-cache, I determined there must be an easier way to compile all my elisp files in one go, rather than calling byte-compile-file for each file in my elisp directory.  It turns out there is:

C-0 M-x byte-recompile-directory

will recursively traverse a directory stucture and compile all .el files encountered.  My startup time is back to 4 seconds.

In Java, there are four scopes: public, package friendly, protected and private.  These are useful, but rather coarse.  It would be nice to be able to specify a variable that was only visible to a select set of methods.  For example, suppose you wanted to develop a counter that could not be modified without being accessed through methods that modified it.  In Java, this would be accomplished with an inner class that had two methods.  It’s interface would have to be declared publicly, though.  In R5RS Scheme, this would look like:

(let ((z 0)) (list (lambda () (set! z (+ z 1))) (lambda () (set! z (- z 1)))))

This defines two functions that each close over the variable z such that you must call the first to increment z and you must call the second to decrement z.  Because z is not defined at the top level, there is no way to access it and modify it directly — you must go through the functions.  This allows the functions to be a bottleneck to the modification of z.  So, if, for example, you wanted to log every time z was changed, you could add a logging command to each of the functions.  This approach allows arbitrarily fine scoping to be defined.  If you want a variable to be available to only one or two functions, it is trivial to do so using the “let” construct.

I’ll take a look at how you can accomplish similar results in JVM-based functional languages like Scala and Clojure in my next post.

It took me 30 years, but I woke up this Sunday morning and realized that rational numbers can be expressed as the ratio of two integers.

By far the most brain-bendingly elegant definition of all the Fibonacci numbers I have ever encountered:

(def fibs (lazy-cat '(1 2) (map + fibs (drop 1 fibs))))

Found over an the clojure-euler wiki.

Just watched the December 2007 Google Authors session with Randall Munroe, creator of the XKCD webcomic.  Really good talk; he seems like a real geek, which is awesome.  He inspired me to sign up for Project Euler, which I have.  I did the first problem in a couple of minutes, and visited the forum to find out what kind of code others were writing.  I continue to love lisp, so I think I’ll be tackling the challenges in Clojure.  The first problem yields to some basic list operations:

(reduce + (filter #(or (= (rem %1 3) 0) (= (rem %1 5) 0)) (range 1000)))

We’ll see how well Lisp holds up for the later problems.

Clojure is a superb new-ish lisp for the JVM that is quickly becoming my preferred hacking playground.  As with any new language, as soon as you write anything worthwhile, you want to figure out how to distribute it.  In Clojure’s case, even though Clojure is functional and is dynamically compiled, the process of creatng an executable jar isn’t too bad.  I outline it here for reference:

1. Create the application directory structure.  For this example, I’m using org.etherplex.example.JarExample.  I’m going to make a project directory called simply “project”.  I’m assuming Linux/OS X here, Windows users need to adjust this or install Cygwin.

   mkdir -p project/org/etherplex/example

2. Create your application in Clojure, creating a main method that is named properly.  For this example, I used a file called JarExample.clj.  It resides in the project/org/etherplex/example folder.

   (in-ns 'org.etherplex.example)

   (clojure/refer 'clojure)

   (def JarExample-main (fn [& args] (println "This is an example application!")))

   If we want to have our “main” class be called JarExample, then in clojure, we need to define a function called JarExample-main.  In general, Clojure will look for methods of a class in the same package as that class, and will use the form TheClass-methodName to locate the proper class.

3. Now we need to create a stubbed .class file that will dispatch method calls to our Clojure file.  The easiest way to do this is to use the REPL.  I start the REPL in the directory that is the parent of my project directory.  In this case, that’s my home directory.
   

   cd ~

   java -jar path/to/clojure.jar

   Now that we are in a Clojure REPL, we can create our class stub:

   (gen-and-save-class "project" "org.etherplex.example.JarExample" :main true)

   First, we specify the path that contains our files (”project”) and then the fully qualified class we want to create.  We also specify a flag that indicates that we want Clojure to create a main method that dispatches out to the Clojure file we created in Step 2.

4. Unpack clojure.jar so you can include its contents in your jar file.  We need to make sure our jar has all the dependencies in it so it is easy to distribute.

   cd ~

   unzip path/to/clojure.jar

   mv clojure project

5. Now we need to create a manifest file for the jar so Java know to execute JarExample.class.  This is the contents of a file called manifest.txt that I put in my project directory.

   Main-Class: org.etherplex.example.JarExample

   Class-Path: .

   It is vital that this file have a trailing newline!

6. Finally, we can create our jar file:
   

   cd ~/project

   jar cmf manifest.txt myjar.jar clojure org

   And execute it with:
   
   

   java -jar myjar.jar

   This will create a jar file that contains clojure and your code, and will interpret/compile your .clj file on the fly and run it.  Good luck!

My Macbook (13″, purchased May ‘07) just ceased to boot (black screen on startup).  Tried all the standard tricks, no luck.  Took it in to the Apple store and they tested and declared that one of the memory (RAM) slots had failed, but it would boot if I removed the RAM from that slot.  They said the flat rate for repair (regardless of problem) is $280.  I’m opting to replace the RAM in the remaining slot with a new stick that has twice the capacity (1×1GB vs. 2×512MB) instead, sacrificing half my RAM bandwidth for $250 (the RAM cost $30).  New Macbook soon?

Trying to figure out if there is anything in the canonical definition of streams (lazy lists) that says that you need to keep the initial elements of the stream after use (if, for example, you’re recursing down the list and the head elements are getting garbage collected). Can streams be used as iterators when traversed recursively?