Tag Archives: clojure

Slacker 0.8.0

Posted on by
2

A new release 0.8.0 of slacker has been pushed to clojars. Let’s go through the changes in this version.

Clojure 1.3 compatible

Slacker finally landed on clojure 1.3. It takes advantages of performance in 1.3. Also, you can use 1.3 API in slacker. For example, a timeout argument is supported in deref, which is useful when dealing with promise returned by slacker’s asynchronous call.

Performance Boost

The performance enhancement is on the highest priority in this release. I have migrated the NIO infrastructure to a new library called link. Now slacker 0.8.0 is at least 8x faster than previous release. There is significant improvement both on per-request latency and overall throughput. And the server thread model is optimized for data-intensive tasks. Heavy IO tasks in hosted functions won’t block the whole server.

slacker as a ring app

Instead of running default transportation, slacker now can be configured as a ring app and deployed on any ring adapter.

(use 'slacker.server)
(use 'ring.adapter.jetty)

(run-jetty (slacker-ring-app (the-ns 'slacker.example.api)) {:port 8080})

This will expose the name space slacker.example.api with HTTP. Functions could be called with following URL pattern:

http://localhost:8080/<namespace>/<function>.<content-type>

For instance: http://localhost:8080/slacker.example.api/timestamp.json

defn-remote

There is a minor update for the defn-remote macro.

In 0.7.0, you have to specify remote namespace with an option:

(defn-remote sc timestamp :remote-ns "slacker.example.api")

In 0.8.0, it’s more convenience:

(defn-remote sc slacker.example.api/timestamp)

To keep the core library compact, in 0.8.0, the cluster support has been moved to a standalone project slacker-cluster.

All above summarized my recent work in the slacker project. If you have any question with this library, feel free to drop me an email sunng@about.me .

Slacker performance enhanced

Posted on by
1

In the slacker framework, performance issue becomes more and more critical as the basic features are almost completed. As mentioned in cnclojure meetup, I will focus on the performance enhancement in next release.

Now I have worked out a testable version. The new slacker core has been moved to a new NIO library, link. Compared with aleph, link is a thin wrapper of Netty. It takes some nice features from aleph (gloss codec dsl, elegant API), and drops the heavy abstraction, lamina. The new slacker client runs on a real nonblocking connection. Connection pooling is no longer needed.

I have some performance benchmark to visualize the improvement. The test was made on my laptop (Intel(R) Core(TM)2 Duo CPU T5870 @ 2.00GHz). It ran 400,000 calls with 40 threads on a local slacker server.

slacker 0.7.0 (clojure 1.2, aleph 0.2.0): 614005.059259msecs
slacker 0.7.1-SNAPSHOT (clojure 1.3, aleph 0.2.1-beta2): 409110.909142msecs
slacker 0.8.0-SNAPSHOT (clojure 1.3, link 0.2.0-SNAPSHOT): 42468.401522msecs

tps chart

Check out the new slacker on the 0.8-dev branch.

My favorite feature in leiningen 2

Posted on by
2

Recently, the leiningen team has released a preview version for leiningen 2. It brings new features to the clojure build tool. But my favorite one is not listed in any document. So I would like to share with you here.

As you know, leiningen is a project oriented tool. It manages dependencies for a particular project. In Java world, Maven and Gradle are also working in this manner.

In contrast, there are tools which manages dependencies in a system scope. For instance, pip(python), npm(nodejs) and gems(ruby). One advantage of these tools is easy for evaluating a library. Concretely, when you want to test pyclj, just run “pip install pyclj” to install it. Then open a REPL and type “import pyclj”. That’s pretty easy.

But in clojure/leiningen, to take a tutorial of “core.logic”, we have following steps:

  1. Find a right directory and type “lein new logic-abc” to create a project.
  2. cd into it, edit project.clj, add core.logic as a dependency
  3. Run `lein deps`
  4. Start a REPL and follow the tutorial

So I guest you must have a lot of empty projects created for such purpose.

Now we could say goodbye to this situation. Leiningen 2 has move its dependency management core to a new library, called pomegranate. Pomegranate wraps aether, which is a maven library created by sonatype. With pomegranate, we can add a maven artifact from repository to REPL classpath. Still on the core.logic example, it becomes much easier:

$ lein2 repl
Welcome to REPL-y!
Clojure 1.3.0
    Exit: Control+D or (exit) or (quit)
Commands: (help)
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
          (sourcery function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
Examples from clojuredocs.org:
          (clojuredocs name-here)
          (clojuredocs "ns-here" "name-here")
nil
user=> (use '[cemerick.pomegranate :only (add-dependencies)])
nil
user=> (add-dependencies :coordinates '[[org.clojure/core.logic "0.6.8"]])
{[org.clojure/clojure "1.3.0"] nil, [org.clojure/core.logic "0.6.8"] #{[org.clojure/clojure "1.3.0"]}}
user=> (use  '[clojure.core.logic])
nilWARNING: == already refers to: #'clojure.core/== in namespace: user, being replaced by: #'clojure.core.logic/==
user=> (run* [q] (== q 1))
(1)

Now core.logic is right on you classpath and you are ready to use any functions under the namespace. There’s no need to create project, no need to care about where the jars stored. Just start a REPL at anywhere you want. When you finished, send EOF to the REPL. Nothing to clean up.

My jython dependency manager jip has similar feature as I described above. It does great help to me. So I have been waiting for this feature in leiningen for a long time. Thanks to leiningen guys, it finally comes.

Edit 20120323 21:39

If you want to load libraries from clojars, you should explicitly add clojars in add-dependencies:

(add-dependencies :coordinates '[[incanter "1.2.3"]]
                  :repositories (merge cemerick.pomegranate.aether/maven-central
                                       {"clojars" "http://clojars.org/repo"}))

(The example is copied from Pomegranate document.)

What’s new in slacker 0.7.0 ?

Posted on by
Reply

I just released [slacker "0.7.0"] to clojars. This is the first release after my presentation on the Clojure China Meetup. lbt05 contributed an ACL module to slacker, which is the most significant feature in this release.

The ACL module provides a simple DSL to define access rules.

(use 'slacker.acl)
(use 'slacker.server)

(defrules myrule
  (allow ["10.60.15.*"]))

(start-slacker-server ...
                      :acl myrule)

“myrule” defines a limited access control list. Only clients from IP segment 10.60.15.* could access the slacker service.

And there are also minor enhancements in this release:

  • Content compression, new content type :deflate-carb :deflate-json and :deflate-clj
  • In debug mode, server side stacktraces are printed on client
  • Zookeeper node path refined
  • New options in use-remote, :o nly and :exclude
  • Cheshire used as json library

slacker 0.7.0 will be the last version on clojure 1.2 . As aleph 0.2.1 is coming near, we will migrate to clojure 1.3 as soon as possible. If you like to taste slacker on your 1.3 application now, there is a 0.7.1-SNAPSHOT available.