Clojure 1.3 compatible

Slacker finally landed on clojure 1.3. It takes advantages of performance in 1.3. Also, you can use 1.3 API in slacker. For example, a timeout argument is supported in deref, which is useful when dealing with promise returned by slacker’s asynchronous call.

Performance Boost

The performance enhancement is on the highest priority in this release. I have migrated the NIO infrastructure to a new library called link. Now slacker 0.8.0 is at least 8x faster than previous release. There is significant improvement both on per-request latency and overall throughput. And the server thread model is optimized for data-intensive tasks. Heavy IO tasks in hosted functions won’t block the whole server.

slacker as a ring app

Instead of running default transportation, slacker now can be configured as a ring app and deployed on any ring adapter.

This will expose the name space slacker.example.api with HTTP. Functions could be called with following URL pattern:

http://localhost:8080/<namespace>/<function>.<content-type>

For instance: http://localhost:8080/slacker.example.api/timestamp.json

defn-remote

There is a minor update for the defn-remote macro.

In 0.7.0, you have to specify remote namespace with an option:

In 0.8.0, it’s more convenience:

To keep the core library compact, in 0.8.0, the cluster support has been moved to a standalone project slacker-cluster.

All above summarized my recent work in the slacker project. If you have any question with this library, feel free to drop me an email sunng@about.me .

Now I have worked out a testable version. The new slacker core has been moved to a new NIO library, link. Compared with aleph, link is a thin wrapper of Netty. It takes some nice features from aleph (gloss codec dsl, elegant API), and drops the heavy abstraction, lamina. The new slacker client runs on a real nonblocking connection. Connection pooling is no longer needed.

I have some performance benchmark to visualize the improvement. The test was made on my laptop (Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz). It ran 400,000 calls with 40 threads on a local slacker server.

slacker 0.7.0 (clojure 1.2, aleph 0.2.0): 614005.059259msecs
slacker 0.7.1-SNAPSHOT (clojure 1.3, aleph 0.2.1-beta2): 409110.909142msecs
slacker 0.8.0-SNAPSHOT (clojure 1.3, link 0.2.0-SNAPSHOT): 42468.401522msecs

Check out the new slacker on the 0.8-dev branch.

The API is very simple. It’s all like python’s data modules (json, pickle)

Clojure types are mapping to python data structures :

But how we win clojure’s heart from ruby?

We are faster.

Considering clojure literal below:

Comparing ruby-clj(0.0.4.5, ruby 1.9.3p0) and pyclj(0.1.3 python 2.7.2):

The result:
ruby: 13.451157809
python: 0.712423086166
Edit 20120216 13:30
ruby-clj 0.0.5.3 has resolved the performance issue
The new result ruby-clj/0.0.5.4 Vs pyclj0.1.4 (on my laptop):
ruby-clj: 2.044872364
pyclj: 1.19659209251

The project is hosted on github. Feel free to join the development and enhance it.

时隔半年日日沉浸在clojure世界里的时候，多亏了github上Daniel Azuma的提示，现在HeatCanvas通过Image Data数组来绘制图像。过去由于不太熟悉Canvas API，我用的是fillRect来填充1像素大小的区域，模拟像素的渲染。但是这种方式导致浏览器渲染的效率非常低。

ImageDataArray允许用户开辟一个固定大小的buffer，并设置每一像素的像素值，然后一次性地渲染到canvas上。详情可以参考这里：Pixel manipulation with canvas

这次性能的提升基本没有影响API，唯一的区别是如果原先自定义了value-color的映射函数的话，现在不再接受hsl的css字符串了，新的API需要你返回一个四个元素的数组，分别代表h, s, l, a，值域[0-1]。

感谢关注HeatCanvas的朋友。

Slacker cluster has a centralized registry, a zookeeper node, stores information of all the namespaces and servers instances in the cluster. Once a client declared remote functions, by calling `defn-remote` or `use-remote`, it reads all available servers offering that namespace from the registry and create connection to each of them. We the user issues a request, the client randomly pick up a connection from them. So the load is eventually distributed to every instance of slacker servers. And thanks to zookeeper’s notification feature, the client watches certain znode. It will be notified when 1. a connected server goes offline 2. a new server serving required namespace added into the cluster. Thus you don’t have to change client code or restart client when server changes.

To start a slacker server and add it to a cluster, you have to provide cluster information using the new :cluster option:

• :zk is the address of zookeeper node
• :name is a znode qualified string, to identify the cluster

On the client side, it’s important to use APIs from `slacker.client.cluster` instead of `slacker.client`:

If all servers provide ‘slacker.example.api go offline, slacker client will raise a “not-found” exception.

Slacker cluster is also designed with simple and clean in mind. You don’t have to change you business code to make it remote or cluster. Everything is transparent and non-invasive. Enjoy it.

One thing in slacker 0.6.x is you can expose multiple namespaces from a single server.

Suppose you have two namespaces `redday.stats` and `redday.api`, both contains functions you want to expose.

This will expose `redday.stats` and `redday.api` on port 6565.

On the client side, we have a new `use-remote` behaviors like clojure’s use. Instead of local one, it imports functions from a remote namespace to your current namespace.

If you need to configure callback to a particular function, you can still use `defn-remote` to specify the callback function. In slacker 0.6.0, a `:remote-ns` is required when you define such a remote function.

The complete code example (both server and client) can be found here.

In next post, I will explain another big new feature of 0.6.x, cluster support.

无暇去学习Android平台上繁琐的知识，不过好在有Phonegap这样的框架，可以把网页应用转化为本地应用，并且提供访问本地设备的API。通过Phonegap开发的程序还可以直接移植到iphone平台上。ClojureDocs Android就是运行在Phonegap中。

首页：

搜索界面

API函数界面

你可以从github获得代码和签名过的apk：https://github.com/sunng87/clojuredocs-android

Known Issue，phonegap程序在屏幕旋转时会崩溃，已经在2.3和3.2上重现，目前还不清楚具体的原因。(Edit 20120127: Fixed in 1.0.4)

欢迎任何的pull request。

LazyPress后台存储采用mongodb. 正是mongodb的schema free特性降低了项目功能重构的成本, 促进了人们更快更频繁地优化产品的模型. 这应该是文档型数据库之于传统关系型数据库最大的优势. (犹如git之于svn, 开分支的成本要低得多, 看似是一个普通的功能改进, 实则鼓励促进了开发人员通过开分支实现自己的修改)
LazyPress运行在最近发布的compojure 1.0.0和ring 1.0.0上, 打包为标准的Java web应用直接部署在cloudfoundry上. 前端继续使用的是我偏爱的Mootools库, 因为使用了很多新的浏览器技术, 所以目前只能保证在最新的Firefox和Chromium/Chrome上正常使用.

Fork me on github: http://github.com/sunng87/lazypress

Breaking Changes

• New maven coordinator: [slacker "0.4.0"] (groupId renamed to slakcer)
• defremote renamed to defn-remote
• SlackerException removed. slacker now uses slingshot for exception handling
• Rename :async option of defn-remote to :async?

What’s new in 0.4.0

• Add new serialization type :clj
• New interceptors: execution time stats, args logger, slow watch dog
• New HTTP interface
• Server inspect commands
• utility functions/macros defn-remote-all, defn-remote-batch and meta-remote

Get more information on github.

Like many server frameworks, slacker abstracts the request processing as a pipeline. The request object is modified by adding or updating attributes through each node of the pipeline. So it’s easy to add your interceptor into the pipeline, with which you can get the data before and after function executed.

To create such an interceptor, you should use the slacker.interceptor/definterceptor macro and slacker.interceptor/definterceptor+ macro:

(definterceptor name
:before interceptor-function
:after interceptor-function)

(definterceptor+ name [arguments]
:before interceptor-function
:after interceptor-function)

definterceptor+ can accept arguments so you can configure the interceptor when you use it.

See a simple example:

Then, add it to your slacker server by

Now you can log every function call of your slacker server.

For more detail about the interceptor framework, especially the request data, please check the wiki page.

In slacker 0.3.0, there is a built-in interceptor to stats function calls. You can find it at slacker.interceptors.stats. The stats data is expose via JMX. You can also write monitoring application to retrieve the data.

And there will be more built-in interceptors in 0.4.0, includes function call time stats and logging.