Category Archives: ANN

Slacker 0.8.0

Posted on by
2

A new release 0.8.0 of slacker has been pushed to clojars. Let’s go through the changes in this version.

Clojure 1.3 compatible

Slacker finally landed on clojure 1.3. It takes advantages of performance in 1.3. Also, you can use 1.3 API in slacker. For example, a timeout argument is supported in deref, which is useful when dealing with promise returned by slacker’s asynchronous call.

Performance Boost

The performance enhancement is on the highest priority in this release. I have migrated the NIO infrastructure to a new library called link. Now slacker 0.8.0 is at least 8x faster than previous release. There is significant improvement both on per-request latency and overall throughput. And the server thread model is optimized for data-intensive tasks. Heavy IO tasks in hosted functions won’t block the whole server.

slacker as a ring app

Instead of running default transportation, slacker now can be configured as a ring app and deployed on any ring adapter.

(use 'slacker.server)
(use 'ring.adapter.jetty)

(run-jetty (slacker-ring-app (the-ns 'slacker.example.api)) {:port 8080})

This will expose the name space slacker.example.api with HTTP. Functions could be called with following URL pattern:

http://localhost:8080/<namespace>/<function>.<content-type>

For instance: http://localhost:8080/slacker.example.api/timestamp.json

defn-remote

There is a minor update for the defn-remote macro.

In 0.7.0, you have to specify remote namespace with an option:

(defn-remote sc timestamp :remote-ns "slacker.example.api")

In 0.8.0, it’s more convenience:

(defn-remote sc slacker.example.api/timestamp)

To keep the core library compact, in 0.8.0, the cluster support has been moved to a standalone project slacker-cluster.

All above summarized my recent work in the slacker project. If you have any question with this library, feel free to drop me an email sunng@about.me .

What’s new in slacker 0.7.0 ?

Posted on by
Reply

I just released [slacker "0.7.0"] to clojars. This is the first release after my presentation on the Clojure China Meetup. lbt05 contributed an ACL module to slacker, which is the most significant feature in this release.

The ACL module provides a simple DSL to define access rules.

(use 'slacker.acl)
(use 'slacker.server)

(defrules myrule
  (allow ["10.60.15.*"]))

(start-slacker-server ...
                      :acl myrule)

“myrule” defines a limited access control list. Only clients from IP segment 10.60.15.* could access the slacker service.

And there are also minor enhancements in this release:

  • Content compression, new content type :deflate-carb :deflate-json and :deflate-clj
  • In debug mode, server side stacktraces are printed on client
  • Zookeeper node path refined
  • New options in use-remote, :o nly and :exclude
  • Cheshire used as json library

slacker 0.7.0 will be the last version on clojure 1.2 . As aleph 0.2.1 is coming near, we will migrate to clojure 1.3 as soon as possible. If you like to taste slacker on your 1.3 application now, there is a 0.7.1-SNAPSHOT available.

Python’s Valentines Day Gift to Clojure

Posted on by
8

Inspired by meh’s Ruby-Clj module, I created the python equivalent “pyclj” last weekend. Pyclj is a clojure literal reader/writer for python. It enables data exchange between python and clojure, in a clojure-native way. It’s Valentines Day today, I’d like to release it as the gift of python to clojure :)

The API is very simple. It’s all like python’s data modules (json, pickle)

import clj

clj.loads("[1 2 3]")
clj.dumps({"a":1, "b":2})

Clojure types are mapping to python data structures :

Clojure Python
list list
vector list
set set
map dict
nil None
string string
int int
float float
boolean boolean
char string
keyword string

But how we win clojure’s heart from ruby?

We are faster.

Considering clojure literal below:

[1 2 3 true false nil {:a 21.3 :b 43.2} "Hello"]

Comparing ruby-clj(0.0.4.5, ruby 1.9.3p0) and pyclj(0.1.3 python 2.7.2):

require 'clj'

s = "[1 2 3 true false nil {:a 21.3 :b 43.2} \"Hello\"]"

t1 = Time.now()
for i in 0...10000
  Clojure.parse(s)
end
puts Time.now()-t1
import clj
import time

s = "[1 2 3 true false nil {:a 21.3 :b 43.2} \"Hello\"]"

t1 = time.time()
for i in range(10000):
  clj.loads(s)
print time.time()-t1

The result:
ruby: 13.451157809
python: 0.712423086166
Edit 20120216 13:30
ruby-clj 0.0.5.3 has resolved the performance issue :)
The new result ruby-clj/0.0.5.4 Vs pyclj0.1.4 (on my laptop):
ruby-clj: 2.044872364
pyclj: 1.19659209251

The project is hosted on github. Feel free to join the development and enhance it.

HeatCanvas performance enhanced

Posted on by
Reply

heatcanvas

时隔半年日日沉浸在clojure世界里的时候，多亏了github上Daniel Azuma的提示，现在HeatCanvas通过Image Data数组来绘制图像。过去由于不太熟悉Canvas API，我用的是fillRect来填充1像素大小的区域，模拟像素的渲染。但是这种方式导致浏览器渲染的效率非常低。

ImageDataArray允许用户开辟一个固定大小的buffer，并设置每一像素的像素值，然后一次性地渲染到canvas上。详情可以参考这里：Pixel manipulation with canvas

这次性能的提升基本没有影响API，唯一的区别是如果原先自定义了value-color的映射函数的话，现在不再接受hsl的css字符串了，新的API需要你返回一个四个元素的数组，分别代表h, s, l, a，值域[0-1]。

感谢关注HeatCanvas的朋友。

Slacker Cluster

Posted on by
Reply

Cluster support is one of the big thing in slacker 0.6.x. Cluster enables high-availability and load balancing on slacker client and server.

Slacker cluster has a centralized registry, a zookeeper node, stores information of all the namespaces and servers instances in the cluster. Once a client declared remote functions, by calling `defn-remote` or `use-remote`, it reads all available servers offering that namespace from the registry and create connection to each of them. We the user issues a request, the client randomly pick up a connection from them. So the load is eventually distributed to every instance of slacker servers. And thanks to zookeeper’s notification feature, the client watches certain znode. It will be notified when 1. a connected server goes offline 2. a new server serving required namespace added into the cluster. Thus you don’t have to change client code or restart client when server changes.

To start a slacker server and add it to a cluster, you have to provide cluster information using the new :cluster option:

(start-slacker-server (the-ns 'slacker.example.api)
                      2104
                      :cluster {:zk "127.0.0.1:2181"
                                :name "example-cluster"})
  • :zk is the address of zookeeper node
  • :name is a znode qualified string, to identify the cluster

On the client side, it’s important to use APIs from `slacker.client.cluster` instead of `slacker.client`:

(use 'slacker.client.cluster)
;; arguments: cluster-name, zookeeper address
(def sc (clustered-slackerc "example-cluster" "127.0.0.1:2181"))
(use-remote 'sc 'slacker.example.api)

;; call the function from a random server
(timestamp)

If all servers provide ‘slacker.example.api go offline, slacker client will raise a “not-found” exception.

Slacker cluster is also designed with simple and clean in mind. You don’t have to change you business code to make it remote or cluster. Everything is transparent and non-invasive. Enjoy it.