slacker 0.2.0 is out

Posted on by
Reply

Slacker 0.2.0 has been pushed to clojars today. Connection pooling and json serialization are available in this release.

Connection Pool

Generally, pooling connection is a good idea in high concurrence application. To make slacker for real world, connection pool support is a high-prioritized feature in its development. The new connection pool is backended by commons-pool which you might familiar with. To use connection pool, just create slacker client with a new function `slackerc-pool`

(def scp (slackerc-pool "localhost" 2104))

Then you can use this pool just like a single client.

Some options are available to configure the pool by your wish:

  • :max-active, max connections opened by the pool
  • :exhausted-action
    • :fail throw an exception when pool exhausted.
    • :block block current thread and wait until max-wait exceed (throw an exception)
    • :grow automatically create new connection and add it to pool
  • :max-wait max wait time before throwing an exception
  • :min-idle minimal number of pool hold idle connections

The options are inherited from GenericObjectPool, you can find detailed information from their javadoc.

JSON Serialization

slacker just added json serialization provided by clj-json. According to my test, clj-json is 1x faster than carbonite in serialization.

(def sc (slackerc "localhost" 2104 :content-type :json))

However, with json serialization, you may lost some clojure types like keyword and set in type conversion. You should be caution when using json as serialization method.

In next release, I am planning to use fastjson as json lib which provides option to write type name into json so it could be a full featured serialization for clojure. And fastjson is claimed even faster than jackson.

Performance

slacker gains high performance with its non-blocking server, serialization and direct function call. As tested on a dual 6 core server, it reaches 10000+ TPS for a single client (50 connections, 50 threads). The server just use 35% CPU so I consider it could have even more TPS if there is two or more client machines.

So if you are interested in some benchmarks, you can test it with client like this. All the requests are using synchronous call because I believe it’s the most common case you use slacker.

Next steps

Inspired by discussion in cn-clojure mailing list, I’m going to add HTTP transport for slacker. With HTTP transport, it’s easier to debug and evaluate your clojure functions, it also makes slacker available to ClojureScript.

At lst, thanks Zach Tellman for reviewing my client code.

从GNOME网站安装exaile-doubanfm-gnome-shell-extension

Posted on by
3

最近GNOME发布了期待已久的extension.gnome.org，这个网站允许你直接通过浏览器安装和管理gnome-shell扩展，有点类似app store的感觉，混乱的~/.local/share/gnome-shell/extensions/终于有了一个官方的界面。

网站开通的第一时间，我提交了exaile-doubanfm-gnome-shell-extension，经过review和修改，这个扩展也得到了进一步的完善，适配了gnome-shell 3.2的风格。

你可以从这个地址直接安装启用
https://extensions.gnome.org/extension/24/exaile-doubanfm-control/

它会在exaile douban.fm启动后显示一个菜单在gnome-shell上，你可以通过这个菜单进行基本的操作。

如果喜欢，别忘了在extension.gnome.org上vote一下 ：）

Slacker 0.1.0 is out

Posted on by
2

Glad to roll out the first release of the slacker framework. Slacker is a clojure RPC framework on top of a TCP binary protocol. It provides a set of non-invasive APIs for both server and client. The remote invocation is completely transparent to user.

In addition to APIs introduced in last post, asynchronous approach is supported in client API :

(defremote remote-func :async true)
@(remote-func)

If you add option `:async` to defremote, then the function facade will return a promise. You have to deref it by yourself. Also you can use the `:callback` option in defremote to specify a callback function.

(defremote remote-func :callback #(println %))
(remote-func)

This gives you much more flexibility of using remote function. But be careful it will break consistency between local and remote mode.

To use slacker, add it to your project.clj

:dependencies [[info.sunng/slacker "0.1.0"]]

You can find examples on the github page.

Exaile豆瓣电台插件0.0.11发布

Posted on by
7

很高兴时隔半年后我继续发布了Exaile豆瓣电台插件的更新，从第一个版本发布到现在已经有一年半的时间，这期间豆瓣电台插件已经陆续出现在Rhythmbox、Banshee等播放器上。作为第一个视图把豆瓣电台移植到本地的尝试，我感到甚是欣慰：）

这次的更新修正了长久依赖困绕用户登录问题，现在我们有一个专门的界面来输入验证码。这个功能要感谢DigitalPig用户在github的报告（鞭策作用），此外，我参考了豆瓣电台banshee插件的实现，节省了研究含验证码登录的时间，感谢。总而言之，没有用户的推动，这个项目也不会坚持这么久。

除此之外，插件还有一些支持了新的豆瓣说的推荐，优化了播放列表载入的策略。

另外值得highlight的是，对应的gnome-shell扩展发布了0.0.2版本，唯一的更新是专辑封面现在会显示在gnome-shell的菜单中。

你可以从github获得最新的插件和gnome-shell扩展：

有任何问题都可以在github或这里留言。

Clojure RPC, prototyping and early thoughts

Posted on by
4

Last week, I prototyped an RPC framework, slacker, by clojure and for clojure.

What I did ?

Suppose you have a sets of clojure functions to expose. Define them under a namespace:

(ns slapi)
(defn timestamp []
  (System/currentTimeMillis))

;; ...more functions

Expose the namespace with slacker server, on port 2104:

(use 'slacker.server)
(start-slacker-server (the-ns 'slapi) 2104)

On the client side, we use the `defremote` macro to create a facade for `timestamp` function. This API will keep the client code consistent with local mode.

(use 'slacker.client)
(def sc (slackerc "localhost" 2104))
(defremote sc timestamp)
(timestamp)

Internally, slacker uses aleph for transport and carbonite for serialization. I forked carbonite and made it compatible with clojure 1.2 because the aleph mainline is still running on 1.2.

Going further

High-Order Functions

In clojure, functions are treated as first class value. Within memory, you can pass function as parameter to another function. However, this is not supported by serialization framework. So is it possible to add support for that in future?

Lazy sequence as parameter

This is another interesting feature in clojure function call. You can pass a lazy-sequence to clojure function. In RPC, it requires parameters to be evaluated on the server side.

(defn get-first [& args] (first args))
(apply get-first (range))

Example copied from StackOverflow

Coordinated states between several remote servers

With RPC, we can update states on several servers. So do we need something like distributed dosync:

(defremote a1 update-a1-state)
(defremote a2 update-a2-state)
(dosync-distributed
  (update-a1-state some-value)
  (update-a2-state some-value))

I’m not sure if this is a valid scenario in real world but I think it’s an interesting topic.(distributed STM?)

Conclusion

RPC is the first step to distributed clojure world. I will keep you updated with my prototype.