After a year of feature development and minor fixes, Slacker and Slacker Cluster version 0.13 is now available. In this article, I will summarize changes in this release and give a you short introduction of new features and improvements.
Slacker is an RPC framework features non-invasive design. It exposes clojure namespace as remote service, and keeps your remote invocation as simple as local version. Slacker cluster uses Zookeeper for service discovery, helps you to build micro-service based architecture. The grouping function gives you full control over request routing.
Application managed thread pool
During 0.12 series, Slacker server uses Netty managed thread pool for task execution. Netty assign a single thread from its pool to a connection. The thread will be used for all requests from the connection. And these requests will be processed in a serial manner. This works perfect for non-blocking tasks. However, if your tasks are data-intensive, this causes head-of-line blocking issue.
The Netty design is to keep request/esponse ordered for a connection. Slacker uses multiplex on its connection, so ordering is not an issue. In 0.13, we now use an application managed thread pool for task execution. You can still configure the pool size by :threads option. If your tasks are non-blocking ones, just set the threads equals your cores. Otherwise, you can customize the size based on blocking time of your tasks.
Interrupt
0.13 introduces a new low-level API called interrupt and a new option interrupt-on-timeout. This is backend by a new protocol level command, interrupt. The new command allows the client to interrupt server execution for a particular task. The server thread will be released once interrupt received.
Typically you don't have to call interrupt on slacker client. The interrupt-on-timeout option allows you to cancel a tasks on both client and server when it's timeout. Following the design principle of transparency, the cancellation is synchronized to server-side, just like a local invocation.
Plug-able Serializers
To keep our dependency-tree clean, we detects cheshire/nippy/carbonite at runtime, and makes these dependencies totally optional to slacker.
The new default serializer is Clojure EDN because it requires no additional packages. Slacker provides built-in support for cheshire(:json) and nippy(:nippy). Nippy is high recommended for Slacker. It's a clojure-native binary format, compact and fast. We have been using nippy with Slacker in our production for a long time without any issue.
You can also extend our serializer system by create new implementations for serializer multi-method.
Server data for Slacker Cluster
The new Slacker Cluster start-slacker-server offers a new option :server-data. It allows you to assign some data for this server, for example, the environment (production or stage?). The data will be stored to Zookeeper and synchronized to client side.
In the client grouping function, you can use the data to filter servers:
(require '[slacker.client.cluster :refer :all])
(def sc (clustered-slackerc ...))
(defn-remote sc some-function
:grouping (fn [ns func params servers]
;; test :prod? property of server data
(rand-nth (filter #(:prod? (server-data @sc %)) servers))))
The grouping function in this snippet filters production servers, and choose one from them to call.
The server side looks pretty simple:
(require '[slacker.server.cluster :refer [start-slacker-server]])
(start-slacker-server some-port [some-ns]
:server-data {:prod? true}
...)
Besides of these features, we also fixed issues Zookeeper timeout issue on startup, ephemeral node lost and etc.
After almost 4 years of development, we are stepping near to a 1.0 release. Hopefully we will reach the 1.0 milestone in 2016.