Slacker Cluster

Cluster support is one of the big thing in slacker 0.6.x. Cluster enables high-availability and load balancing on slacker client and server.

Slacker cluster has a centralized registry, a zookeeper node, stores information of all the namespaces and servers instances in the cluster. Once a client declared remote functions, by calling `defn-remote` or `use-remote`, it reads all available servers offering that namespace from the registry and create connection to each of them. We the user issues a request, the client randomly pick up a connection from them. So the load is eventually distributed to every instance of slacker servers. And thanks to zookeeper’s notification feature, the client watches certain znode. It will be notified when 1. a connected server goes offline 2. a new server serving required namespace added into the cluster. Thus you don’t have to change client code or restart client when server changes.

To start a slacker server and add it to a cluster, you have to provide cluster information using the new :cluster option:

(start-slacker-server (the-ns 'slacker.example.api)
                      2104
                      :cluster {:zk "127.0.0.1:2181"
                                :name "example-cluster"})
  • :zk is the address of zookeeper node
  • :name is a znode qualified string, to identify the cluster

On the client side, it’s important to use APIs from `slacker.client.cluster` instead of `slacker.client`:

(use 'slacker.client.cluster)
;; arguments: cluster-name, zookeeper address
(def sc (clustered-slackerc "example-cluster" "127.0.0.1:2181"))
(use-remote 'sc 'slacker.example.api)

;; call the function from a random server
(timestamp)

If all servers provide ‘slacker.example.api go offline, slacker client will raise a “not-found” exception.

Slacker cluster is also designed with simple and clean in mind. You don’t have to change you business code to make it remote or cluster. Everything is transparent and non-invasive. Enjoy it.

Slacker 0.6: Exposing multiple namespaces

After 98 commits in about one month, I’m glad to announce [slacker "0.6.1"].

One thing in slacker 0.6.x is you can expose multiple namespaces from a single server.

Suppose you have two namespaces `redday.stats` and `redday.api`, both contains functions you want to expose.

  (start-slacker-server [(the-ns 'redday.stats)
                         (the-ns 'redday.api)]
                        6565)

This will expose `redday.stats` and `redday.api` on port 6565.

On the client side, we have a new `use-remote` behaviors like clojure’s use. Instead of local one, it imports functions from a remote namespace to your current namespace.

(use 'slacker.client)
;; create a slacker client
(def scp (slackerc "127.0.0.1:6565"))

(use-remote 'scp 'redday.api) ;; caution, use the symbol of 'scp here
(use-remote 'scp 'redday.stats)

;;top-titles is a function in redday.api
;;now you can use the remote function transparently
(top-titles "programming")

;;check function metadata you can find more slacker properties
(meta top-titles)

If you need to configure callback to a particular function, you can still use `defn-remote` to specify the callback function. In slacker 0.6.0, a `:remote-ns` is required when you define such a remote function.

(defn-remote top-titles :remote-ns "redday.api" :callback #(println %))

The complete code example (both server and client) can be found here.

In next post, I will explain another big new feature of 0.6.x, cluster support.

ClojureDocs Android App

利用春节的假期写了一个Android应用,可以在ClojureDocs.org上搜索clojure API,浏览文档、源代码和社区贡献的代码实例。ClojureDocs在我学习Clojure的过程中起了很大的作用,所以我想这个网站应该对很多人有用。

无暇去学习Android平台上繁琐的知识,不过好在有Phonegap这样的框架,可以把网页应用转化为本地应用,并且提供访问本地设备的API。通过Phonegap开发的程序还可以直接移植到iphone平台上。ClojureDocs Android就是运行在Phonegap中。

首页:

搜索界面

API函数界面

你可以从github获得代码和签名过的apk:https://github.com/sunng87/clojuredocs-android

Known Issue,phonegap程序在屏幕旋转时会崩溃,已经在2.3和3.2上重现,目前还不清楚具体的原因。(Edit 20120127: Fixed in 1.0.4)

欢迎任何的pull request。

Weekend Project: LazyPress

发布一个典型的weekend project, 名字叫做LazyPress.顾名思义,这是一个在线的写作系统. 取名Lazy, 除了因为它用Clojure写成,更因为他的简单: 没有繁琐的注册,没有繁琐的分类tag,没有繁琐的格式化,无论是使用还是开发都力求做到最简单.
LazyPress采用Mozilla刚刚发布的BrowserID. 技术作为账号系统, 用户只需要在首次登录后提供一个ID即可(原本这一步也可以省略, 但是为了保护您的邮箱隐私, 现在需要一个LazyPress专用的ID). 这样LazyPress本身不存储用户的密码,也简化了用户账号管理的代价. BrowserID的登录流程非常简单, 速度也比传统的OpenID和OAuth要快, 用户体验要比多次跳转好很多.
LazyPress使用Markdown进行文本格式化, 简单的编辑器可以支持绝大多数格式的要求. 另外,在浏览器后台LazyPress使用HTML5 LocalStorage技术自动保存用户的文本草稿, 如果用户没有成功发布, 可以在下次浏览器打开时进行恢复.

LazyPress后台存储采用mongodb. 正是mongodb的schema free特性降低了项目功能重构的成本, 促进了人们更快更频繁地优化产品的模型. 这应该是文档型数据库之于传统关系型数据库最大的优势. (犹如git之于svn, 开分支的成本要低得多, 看似是一个普通的功能改进, 实则鼓励促进了开发人员通过开分支实现自己的修改)
LazyPress运行在最近发布的compojure 1.0.0和ring 1.0.0上, 打包为标准的Java web应用直接部署在cloudfoundry上. 前端继续使用的是我偏爱的Mootools库, 因为使用了很多新的浏览器技术, 所以目前只能保证在最新的Firefox和Chromium/Chrome上正常使用.

Fork me on github: http://github.com/sunng87/lazypress

slacker 0.4.0 released

Slacker 0.4.0 has been released to clojars.org . There are new features and breaking changes in this release.

Breaking Changes

  • New maven coordinator: [slacker "0.4.0"] (groupId renamed to slakcer)
  • defremote renamed to defn-remote
  • SlackerException removed. slacker now uses slingshot for exception handling
  • Rename :async option of defn-remote to :async?

What’s new in 0.4.0

  • Add new serialization type :clj
  • New interceptors: execution time stats, args logger, slow watch dog
  • New HTTP interface
  • Server inspect commands
  • utility functions/macros defn-remote-all, defn-remote-batch and meta-remote

Get more information on github.