Slacker 0.15 released

After another year of development, I'm proud to announce a new major release of my Clojure RPC library, slacker. This release, 0.15.0, includes an update on wire protocol, some performance improvement and bugfix.

The request extension

This release of slacker client use v6 protocol by default. The server can still accept v5 protocol for backward compatibility.

The v6 protocol added an extension field to request and response packet. By using this field, you can exchange some metadata in RPC call. A typical use case is for distributed tracing. Trace ID is passed as extension.

In the API level, we kept the non-invasive principle. Extensions will be injected as var bindings.

(require '[slacker.common :refer [with-extension]])
(require '[slacker.client :as sc])

(def conn (sc/slackerc "127.0.0.1:2104"))
(sc/defn-remote conn slacker.example.api/timestamp)

(def extension-id 16)
(with-extension {extension-id "extension-value"}
  (timestamp))

A more typical usage of extension is in interceptors. slacker-htrace use extension to pass htrace trace id along with rpc call.

(defn client-interceptor [^Tracer tracer span-id-extension-id]
  {:pre (fn [req]
          (let [scope-name (str (:fname req) ":outer")
                trace-scope (.newScope tracer scope-name)]
            (-> req
                (assoc ::trace-scope trace-scope)
                (update :extensions assoc
                        span-id-extension-id
                        (from-span-id(.getSpanId ^TraceScope trace-scope))))))
   :post (fn [resp]
           (when-let [scope (::trace-scope resp)]
             (.close ^TraceScope scope))
           (dissoc resp ::trace-scope))})

(defn server-interceptor [^Tracer tracer span-id-extension-id]
  {:before (fn [req]
             (let [scope-name (str (:fname req) ":inner")
                   span-id (-> req :extensions (get span-id-extension-id) to-span-id)
                   trace-scope (.newScope tracer scope-name span-id)]
               (-> req
                   (assoc ::trace-scope trace-scope))))
   :after (fn [resp]
            (when-let [scope (::trace-scope resp)]
              (.close ^TraceScope scope))
            (dissoc resp ::trace-scope))})

Flexible server thread pool

Previously, you can only configure one thread pool for all exposed namespace. This adds risk that one backend issue may exhaust all your server threads. And your server stops respond requests for any namespace.

To isolate the execution for namespaces, slacker 0.15 added support for finer thread pool configuration.

(start-slacker-server [(the-ns 'slacker.example.api) (the-ns 'slacker.example.api2)]
                      2104
                      :executors {"slacker.example.api"
                                   (ThreadPoolExecutor. 10 10
                                                        0 TimeUnit/MINUTES
                                                        (LinkedBlockingQueue. 100)
                                                        (ThreadPoolExecutor$AbortPolicy.))})

Once all your execution in slacker.example.api blocked, it won't affect requests to slacker.example.api2.

Bugfix and performance improvement

There are some more fixes in release:

  • Fixed issue in stop-slacker-server
  • Use platform specific buffer allocator for (de)serialization
  • Improved client side error report. Full context can be accessed via ex-data.

Middleware

Also I created two example middleware for slacker: