Python’s Valentines Day Gift to Clojure

Inspired by meh’s Ruby-Clj module, I created the python equivalent “pyclj” last weekend. Pyclj is a clojure literal reader/writer for python. It enables data exchange between python and clojure, in a clojure-native way. It’s Valentines Day today, I’d like to release it as the gift of python to clojure :)

The API is very simple. It’s all like python’s data modules (json, pickle)

import clj

clj.loads("[1 2 3]")
clj.dumps({"a":1, "b":2})

Clojure types are mapping to python data structures :

Clojure Python
list list
vector list
set set
map dict
nil None
string string
int int
float float
boolean boolean
char string
keyword string

But how we win clojure’s heart from ruby?

We are faster.

Considering clojure literal below:

[1 2 3 true false nil {:a 21.3 :b 43.2} "Hello"]

Comparing ruby-clj(0.0.4.5, ruby 1.9.3p0) and pyclj(0.1.3 python 2.7.2):

require 'clj'

s = "[1 2 3 true false nil {:a 21.3 :b 43.2} \"Hello\"]"

t1 = Time.now()
for i in 0...10000
  Clojure.parse(s)
end
puts Time.now()-t1
import clj
import time

s = "[1 2 3 true false nil {:a 21.3 :b 43.2} \"Hello\"]"

t1 = time.time()
for i in range(10000):
  clj.loads(s)
print time.time()-t1

The result:
ruby: 13.451157809
python: 0.712423086166
Edit 20120216 13:30
ruby-clj 0.0.5.3 has resolved the performance issue :)
The new result ruby-clj/0.0.5.4 Vs pyclj0.1.4 (on my laptop):
ruby-clj: 2.044872364
pyclj: 1.19659209251

The project is hosted on github. Feel free to join the development and enhance it.

HeatCanvas performance enhanced

heatcanvas

时隔半年日日沉浸在clojure世界里的时候,多亏了github上Daniel Azuma的提示,现在HeatCanvas通过Image Data数组来绘制图像。过去由于不太熟悉Canvas API,我用的是fillRect来填充1像素大小的区域,模拟像素的渲染。但是这种方式导致浏览器渲染的效率非常低。

ImageDataArray允许用户开辟一个固定大小的buffer,并设置每一像素的像素值,然后一次性地渲染到canvas上。详情可以参考这里:Pixel manipulation with canvas

这次性能的提升基本没有影响API,唯一的区别是如果原先自定义了value-color的映射函数的话,现在不再接受hsl的css字符串了,新的API需要你返回一个四个元素的数组,分别代表h, s, l, a,值域[0-1]。

感谢关注HeatCanvas的朋友。

Slacker Cluster

Cluster support is one of the big thing in slacker 0.6.x. Cluster enables high-availability and load balancing on slacker client and server.

Slacker cluster has a centralized registry, a zookeeper node, stores information of all the namespaces and servers instances in the cluster. Once a client declared remote functions, by calling `defn-remote` or `use-remote`, it reads all available servers offering that namespace from the registry and create connection to each of them. We the user issues a request, the client randomly pick up a connection from them. So the load is eventually distributed to every instance of slacker servers. And thanks to zookeeper’s notification feature, the client watches certain znode. It will be notified when 1. a connected server goes offline 2. a new server serving required namespace added into the cluster. Thus you don’t have to change client code or restart client when server changes.

To start a slacker server and add it to a cluster, you have to provide cluster information using the new :cluster option:

(start-slacker-server (the-ns 'slacker.example.api)
                      2104
                      :cluster {:zk "127.0.0.1:2181"
                                :name "example-cluster"})
  • :zk is the address of zookeeper node
  • :name is a znode qualified string, to identify the cluster

On the client side, it’s important to use APIs from `slacker.client.cluster` instead of `slacker.client`:

(use 'slacker.client.cluster)
;; arguments: cluster-name, zookeeper address
(def sc (clustered-slackerc "example-cluster" "127.0.0.1:2181"))
(use-remote 'sc 'slacker.example.api)

;; call the function from a random server
(timestamp)

If all servers provide ‘slacker.example.api go offline, slacker client will raise a “not-found” exception.

Slacker cluster is also designed with simple and clean in mind. You don’t have to change you business code to make it remote or cluster. Everything is transparent and non-invasive. Enjoy it.

Slacker 0.6: Exposing multiple namespaces

After 98 commits in about one month, I’m glad to announce [slacker "0.6.1"].

One thing in slacker 0.6.x is you can expose multiple namespaces from a single server.

Suppose you have two namespaces `redday.stats` and `redday.api`, both contains functions you want to expose.

  (start-slacker-server [(the-ns 'redday.stats)
                         (the-ns 'redday.api)]
                        6565)

This will expose `redday.stats` and `redday.api` on port 6565.

On the client side, we have a new `use-remote` behaviors like clojure’s use. Instead of local one, it imports functions from a remote namespace to your current namespace.

(use 'slacker.client)
;; create a slacker client
(def scp (slackerc "127.0.0.1:6565"))

(use-remote 'scp 'redday.api) ;; caution, use the symbol of 'scp here
(use-remote 'scp 'redday.stats)

;;top-titles is a function in redday.api
;;now you can use the remote function transparently
(top-titles "programming")

;;check function metadata you can find more slacker properties
(meta top-titles)

If you need to configure callback to a particular function, you can still use `defn-remote` to specify the callback function. In slacker 0.6.0, a `:remote-ns` is required when you define such a remote function.

(defn-remote top-titles :remote-ns "redday.api" :callback #(println %))

The complete code example (both server and client) can be found here.

In next post, I will explain another big new feature of 0.6.x, cluster support.

ClojureDocs Android App

利用春节的假期写了一个Android应用,可以在ClojureDocs.org上搜索clojure API,浏览文档、源代码和社区贡献的代码实例。ClojureDocs在我学习Clojure的过程中起了很大的作用,所以我想这个网站应该对很多人有用。

无暇去学习Android平台上繁琐的知识,不过好在有Phonegap这样的框架,可以把网页应用转化为本地应用,并且提供访问本地设备的API。通过Phonegap开发的程序还可以直接移植到iphone平台上。ClojureDocs Android就是运行在Phonegap中。

首页:

搜索界面

API函数界面

你可以从github获得代码和签名过的apk:https://github.com/sunng87/clojuredocs-android

Known Issue,phonegap程序在屏幕旋转时会崩溃,已经在2.3和3.2上重现,目前还不清楚具体的原因。(Edit 20120127: Fixed in 1.0.4)

欢迎任何的pull request。