Slacker 0.8.0

A new release 0.8.0 of slacker has been pushed to clojars. Let’s go through the changes in this version.

Clojure 1.3 compatible

Slacker finally landed on clojure 1.3. It takes advantages of performance in 1.3. Also, you can use 1.3 API in slacker. For example, a timeout argument is supported in deref, which is useful when dealing with promise returned by slacker’s asynchronous call.

Performance Boost

The performance enhancement is on the highest priority in this release. I have migrated the NIO infrastructure to a new library called link. Now slacker 0.8.0 is at least 8x faster than previous release. There is significant improvement both on per-request latency and overall throughput. And the server thread model is optimized for data-intensive tasks. Heavy IO tasks in hosted functions won’t block the whole server.

slacker as a ring app

Instead of running default transportation, slacker now can be configured as a ring app and deployed on any ring adapter.

(use 'slacker.server)
(use 'ring.adapter.jetty)

(run-jetty (slacker-ring-app (the-ns 'slacker.example.api)) {:port 8080})

This will expose the name space slacker.example.api with HTTP. Functions could be called with following URL pattern:

http://localhost:8080/<namespace>/<function>.<content-type>

For instance: http://localhost:8080/slacker.example.api/timestamp.json

defn-remote

There is a minor update for the defn-remote macro.

In 0.7.0, you have to specify remote namespace with an option:

(defn-remote sc timestamp :remote-ns "slacker.example.api")

In 0.8.0, it’s more convenience:

(defn-remote sc slacker.example.api/timestamp)

To keep the core library compact, in 0.8.0, the cluster support has been moved to a standalone project slacker-cluster.

All above summarized my recent work in the slacker project. If you have any question with this library, feel free to drop me an email sunng@about.me .

Slacker performance enhanced

In the slacker framework, performance issue becomes more and more critical as the basic features are almost completed. As mentioned in cnclojure meetup, I will focus on the performance enhancement in next release.

Now I have worked out a testable version. The new slacker core has been moved to a new NIO library, link. Compared with aleph, link is a thin wrapper of Netty. It takes some nice features from aleph (gloss codec dsl, elegant API), and drops the heavy abstraction, lamina. The new slacker client runs on a real nonblocking connection. Connection pooling is no longer needed.

I have some performance benchmark to visualize the improvement. The test was made on my laptop (Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz). It ran 400,000 calls with 40 threads on a local slacker server.

slacker 0.7.0 (clojure 1.2, aleph 0.2.0): 614005.059259msecs
slacker 0.7.1-SNAPSHOT (clojure 1.3, aleph 0.2.1-beta2): 409110.909142msecs
slacker 0.8.0-SNAPSHOT (clojure 1.3, link 0.2.0-SNAPSHOT): 42468.401522msecs

tps chart

Check out the new slacker on the 0.8-dev branch.

ClojureDocs Android App

利用春节的假期写了一个Android应用,可以在ClojureDocs.org上搜索clojure API,浏览文档、源代码和社区贡献的代码实例。ClojureDocs在我学习Clojure的过程中起了很大的作用,所以我想这个网站应该对很多人有用。

无暇去学习Android平台上繁琐的知识,不过好在有Phonegap这样的框架,可以把网页应用转化为本地应用,并且提供访问本地设备的API。通过Phonegap开发的程序还可以直接移植到iphone平台上。ClojureDocs Android就是运行在Phonegap中。

首页:

搜索界面

API函数界面

你可以从github获得代码和签名过的apk:https://github.com/sunng87/clojuredocs-android

Known Issue,phonegap程序在屏幕旋转时会崩溃,已经在2.3和3.2上重现,目前还不清楚具体的原因。(Edit 20120127: Fixed in 1.0.4)

欢迎任何的pull request。

Roar for mootools 1.4

早在天下大势还处在分久必合的时候,那时候mootools还有不少简单实用的小库,比如我今天搜索”mootools notification”就找到这个08年的库叫做Roar。不过遗憾的是从那以后,这个库就再也没有更新过了。

Mootools本身也沉寂了很久,这个项目恐怕也要思考自己未来的发展方向了。今年9月Mootools迈进了1.4,API上有一些变化。现在的下载页也能看到with/without backward compatibility的版本分开下载。为了用上Roar,我尝试了这两个版本发现都不能使用。最后downgrade到1.2可以确定Roar本身在当时是没有什么问题。

这么多年对mootools痴心不改,所以顺手维护了一下Roar,现在可以在1.4 without compatibility的发布下运行了。主要是几个小修改,大多是一些多年deprecated函数被正式删除:

  • Type常量,原先的String.type,Object.type现在统一到一个Type对象下,变成Type.isString和Type.isObject
  • $empty 常量被删除了,现在直接用function()或Function.from()代替
  • $pick 方法被Array.pick取代,参数现在也必须接受数组类型了
  • $merge 方法被Object.merge取代
  • $type 被typeOf取代
  • 函数对象的create方法被删除了,现在可以用函数对象的bind方法替代
  • Browser.Engine 被删除了,需要用其他Browser的API替代

修改后的Roar,放在这个gist里,测试过可以在firefox和chromium上健康使用。IE没有做测试。这个08年的库,眼看四年过去了,用起来依然不错。

作为mootools的铁杆,我还是会一直专一地坚守下去的。(于是,我也已经变成了多年前那些我眼中为旧事物顽抗到底的老家伙了)

Extend slacker server with interceptors

An interceptor framework was introduced in slacker 0.3.0. It’s designed to allow user to add custom functionality without hacking into the internal of slacker.

Like many server frameworks, slacker abstracts the request processing as a pipeline. The request object is modified by adding or updating attributes through each node of the pipeline. So it’s easy to add your interceptor into the pipeline, with which you can get the data before and after function executed.

To create such an interceptor, you should use the slacker.interceptor/definterceptor macro and slacker.interceptor/definterceptor+ macro:

(definterceptor name
:before interceptor-function
:after interceptor-function)

(definterceptor+ name [arguments]
:before interceptor-function
:after interceptor-function)

definterceptor+ can accept arguments so you can configure the interceptor when you use it.

See a simple example:

(definterceptor log-function-call
  :before (fn [req] (println (str "calling " (:fname req))) req))

(definterceptor+ log-function-call-prefixed [prefix]
  :before (fn [req] (println (str
                               (if (fn? prefix) (prefix) prefix)
                               " calling "
                               (:fname req)))
                    req))

Then, add it to your slacker server by

(use '[slacker.interceptor])
(import '[java.util Date])
(start-slacker (the-ns 'slapi) 2104
  :interceptors (interceptors log-function-call
                              (log-function-call-prefixed
                                (fn [] (.toString (Date.)))))

Now you can log every function call of your slacker server.

For more detail about the interceptor framework, especially the request data, please check the wiki page.

In slacker 0.3.0, there is a built-in interceptor to stats function calls. You can find it at slacker.interceptors.stats. The stats data is expose via JMX. You can also write monitoring application to retrieve the data.

And there will be more built-in interceptors in 0.4.0, includes function call time stats and logging.