Clojure RPC, prototyping and early thoughts

Last week, I prototyped an RPC framework, slacker, by clojure and for clojure.

What I did ?

Suppose you have a sets of clojure functions to expose. Define them under a namespace:

(ns slapi)
(defn timestamp []
(System/currentTimeMillis))

;; ...more functions

Expose the namespace with slacker server, on port 2104:

(use 'slacker.server)
(start-slacker-server (the-ns 'slapi) 2104)

On the client side, we use the `defremote` macro to create a facade for `timestamp` function. This API will keep the client code consistent with local mode.

(use 'slacker.client)
(def sc (slackerc "localhost" 2104))
(defremote sc timestamp)
(timestamp)

Internally, slacker uses aleph for transport and carbonite for serialization. I forked carbonite and made it compatible with clojure 1.2 because the aleph mainline is still running on 1.2.

Going further

High-Order Functions

In clojure, functions are treated as first class value. Within memory, you can pass function as parameter to another function. However, this is not supported by serialization framework. So is it possible to add support for that in future?

Lazy sequence as parameter

This is another interesting feature in clojure function call. You can pass a lazy-sequence to clojure function. In RPC, it requires parameters to be evaluated on the server side.

(defn get-first [& args] (first args))
(apply get-first (range))

Example copied from StackOverflow

Coordinated states between several remote servers

With RPC, we can update states on several servers. So do we need something like distributed dosync:

(defremote a1 update-a1-state)
(defremote a2 update-a2-state)
(dosync-distributed
(update-a1-state some-value)
(update-a2-state some-value))

I’m not sure if this is a valid scenario in real world but I think it’s an interesting topic.(distributed STM?)

Conclusion

RPC is the first step to distributed clojure world. I will keep you updated with my prototype.

4 thoughts on “Clojure RPC, prototyping and early thoughts”

Looks like an interesting approach. You may also wish to consider the relative pros/cons of your RPC vs the “dead simple” remote eval as developed by Nakkaya here:

http://nakkaya.com/2010/02/16/net-eval-dead-simple-distributed-computing-for-clojure/

I.e. what problem domains is your system more appropriate for?

Good luck with it. I’ll be interested to see how it develops.

Reply ↓

sunng on November 28, 2011 at 8:09 pm said:

Well, the direct function call is much faster (100x) than the eval approach.

user=> (time (dotimes [i 1000] (System/currentTimeMillis)))
"Elapsed time: 1.58861 msecs"
nil
user=> (time (dotimes [i 1000] (eval '(System/currentTimeMillis))))
"Elapsed time: 627.18387 msecs"

And you know, it’s much securer than eval because you can choose functions to expose. And if you expose your system with eval (like tryclj.com), you need some sandbox mechanism (like clojail) to protect your server.

Reply ↓