slacker 0.2.0 is out

Sat 10 December 2011
  • ANN tags:
  • clojure
  • java
  • project
  • slacker published: true comments: true

Slacker 0.2.0 has been pushed to clojars today. Connection pooling and json serialization are available in this release.

Connection Pool

Generally, pooling connection is a good idea in high concurrence application. To make slacker for real world, connection pool support is a high-prioritized feature in its development. The new connection pool is backended by commons-pool which you might familiar with. To use connection pool, just create slacker client with a new function `slackerc-pool`

[cc lang="clojure"]
(def scp (slackerc-pool "localhost" 2104))
[/cc]

Then you can use this pool just like a single client.

Some options are available to configure the pool by your wish:

  • :max-active, max connections opened by the pool
  • :exhausted-action
    • :fail throw an exception when pool exhausted.
    • :block block current thread and wait until max-wait exceed (throw an exception)
    • :grow automatically create new connection and add it to pool

  • :max-wait max wait time before throwing an exception
  • :min-idle minimal number of pool hold idle connections

    • The options are inherited from GenericObjectPool, you can find detailed information from their javadoc.

    JSON Serialization

    slacker just added json serialization provided by clj-json. According to my test, clj-json is 1x faster than carbonite in serialization.
    [cc lang="clojure"]
    (def sc (slackerc "localhost" 2104 :content-type :json))
    [/cc]

    However, with json serialization, you may lost some clojure types like keyword and set in type conversion. You should be caution when using json as serialization method.

    In next release, I am planning to use fastjson as json lib which provides option to write type name into json so it could be a full featured serialization for clojure. And fastjson is claimed even faster than jackson.

    Performance

    slacker gains high performance with its non-blocking server, serialization and direct function call. As tested on a dual 6 core server, it reaches 10000+ TPS for a single client (50 connections, 50 threads). The server just use 35% CPU so I consider it could have even more TPS if there is two or more client machines.

    So if you are interested in some benchmarks, you can test it with client like this. All the requests are using synchronous call because I believe it's the most common case you use slacker.

    Next steps

    Inspired by discussion in cn-clojure mailing list, I'm going to add HTTP transport for slacker. With HTTP transport, it's easier to debug and evaluate your clojure functions, it also makes slacker available to ClojureScript.

    At lst, thanks Zach Tellman for reviewing my client code.