Tag Archives: java

slacker 0.2.0 is out

Posted on by
Reply

Slacker 0.2.0 has been pushed to clojars today. Connection pooling and json serialization are available in this release.

Connection Pool

Generally, pooling connection is a good idea in high concurrence application. To make slacker for real world, connection pool support is a high-prioritized feature in its development. The new connection pool is backended by commons-pool which you might familiar with. To use connection pool, just create slacker client with a new function `slackerc-pool`

(def scp (slackerc-pool "localhost" 2104))

Then you can use this pool just like a single client.

Some options are available to configure the pool by your wish:

  • :max-active, max connections opened by the pool
  • :exhausted-action
    • :fail throw an exception when pool exhausted.
    • :block block current thread and wait until max-wait exceed (throw an exception)
    • :grow automatically create new connection and add it to pool
  • :max-wait max wait time before throwing an exception
  • :min-idle minimal number of pool hold idle connections

The options are inherited from GenericObjectPool, you can find detailed information from their javadoc.

JSON Serialization

slacker just added json serialization provided by clj-json. According to my test, clj-json is 1x faster than carbonite in serialization.

(def sc (slackerc "localhost" 2104 :content-type :json))

However, with json serialization, you may lost some clojure types like keyword and set in type conversion. You should be caution when using json as serialization method.

In next release, I am planning to use fastjson as json lib which provides option to write type name into json so it could be a full featured serialization for clojure. And fastjson is claimed even faster than jackson.

Performance

slacker gains high performance with its non-blocking server, serialization and direct function call. As tested on a dual 6 core server, it reaches 10000+ TPS for a single client (50 connections, 50 threads). The server just use 35% CPU so I consider it could have even more TPS if there is two or more client machines.

So if you are interested in some benchmarks, you can test it with client like this. All the requests are using synchronous call because I believe it’s the most common case you use slacker.

Next steps

Inspired by discussion in cn-clojure mailing list, I’m going to add HTTP transport for slacker. With HTTP transport, it’s easier to debug and evaluate your clojure functions, it also makes slacker available to ClojureScript.

At lst, thanks Zach Tellman for reviewing my client code.

普通青年、二逼青年与文艺青年的Java代码缩进

Posted on by
8

普通青年

while(true) {
    if (something) {
        System.out.println(something);
        break;
    }
}

特点： tab与空格混用，无其他特点。
常见于：各类代码仓库。

二逼青年

while(true)
{


    if (something)
    {
        System.out.println(something);
        break;
    }

}

特点：总担心代码不够长
见于：各类劣质技术书籍

文艺青年

while(true)
  {
  if (something)
    {
      System.out.println(something);
      break;
    }
  }

特点：普通Java青年永远不会理解的缩进，lisp程序员会心一笑
见于：Clojure源码

Convert a Python function to Java anonymous class

Posted on by
2

When calling Java with Jython, anonymous inner class might be an issue because there is no such equivalent in Python.

In GUI programming, jython made additional effort on AWT event processing. You can pass a python function as some types of event listener.

def change_text(event):
    print 'Clicked!'

button = JButton('Click Me!', actionPerformed=change_text)
frame.add(button)

Described in the Definitive Guide of Jython:

This works because Jython is able to automatically recognize events in Java code if they have corresponding addEvent()* and *removeEvent() methods. Jython takes the name of the event and makes it accessible using the nice Python syntax as long as the event methods are public.

However, it does not work with Runnable, Callable and many other interfaces designed for anonymous usage.

To solve the incompatibility, I created a small decorator that converts Python function to a Java object.

Thanks to python’s dynamic magic, we can create class in runtime and assign modified method to a particular instance. The conversion is performed once the function loaded. Also, you can pass something as arguments to constructor.

With anonymous_class decorator, the example above can be written as:

from javax.swing import JButton, JFrame
from java.awt.event import ActionListener

frame = JFrame('Hello, Jython!',
            defaultCloseOperation = JFrame.EXIT_ON_CLOSE,
            size = (300, 300)
        )

@anonymous_class(ActionListener, "actionPerformed")
def change_text(dummy, event):
    print 'Clicked!'

button = JButton('Click Me!', actionPerformed=change_text)
frame.add(button)
frame.visible = True

GPars的Actor实现

Posted on by
7

Actor是一种Continuation技术，可以在少量的线程运行大量Actor对象。Actor对象之间通过消息机制进行交互。而Actor本身线程安全，这样的模型使并发编程的复杂度降低，同时也在一定的场景下实现了可扩展性。

gpars是Java和Groovy都可以使用的并行编程库，他实现了Actor、Agent、DataFlow等模型，旨在为Groovy提供高层的并行编程模型。以下分析gpars 0.12中非阻塞Actor的实现。

@Grab("org.codehaus.gpars:gpars:0.11")
import groovyx.gpars.actor.Actors

def worker = Actors.actor {
    loop {
        react {
            reply it.reverse()
        }
    }
}

def console = Actors.actor {
    worker << "Hello GPars"
    react {
        println it
    }
}

console.join()

首先，在工厂类中Actors里会初始化一个DefaultPGroup用来封装后台的线程池、管理actors。Actors默认使用ResizeablePool，他是JDK Concurrent Framework中的ThreadPoolExecutor的封装，coreSize和maxSize不同所以称Resizeable。

Actors的工厂方法actor生成的是DefaultActor，它是非阻塞actor的默认实现。（ActorGroup:67）

DefaultActor的构造方法接受一个Groovy的闭包对象，将其封装为DefaultActorClosure对象后，调用其父类AbstractLoopingActor的initialize方法（DefaultActor：73）.

initialize方法创建一个Runnable对象AsyncMessagingCore，并将线程池传递给core对象。（AbstractLoopingActor:57）AsyncMessagingCore对象负责消息的传递和处理，是线程池处理的目标对象。

调用start启动actor后，actor会向自己发送一个start消息（AbstractLoopingActor:173）.
core获得start消息后，调用DefaultActor覆盖的handleStart方法（DefaultActor:328）。

在handleStart中，actor会调用用户传入的闭包方法。上面的例子是一种典型的用法，loop是DefaultActor中的方法，loop也并不是无限空转的，他仅在收到消息被时被触发（DefaultActor:191）react也是DefaultActor中的方法，它将nextContinuation方法设为内部闭包对象，用来处理actor接收的消息。

向Actor发送消息，是通过actor的send方法和重载的leftShift运算符进行操作。（AbstractLoopingActor:236）actor调用core的store方法，将ActorMessage压入core的队列中。入队列之后，core会检查锁对象activeUpdater，判断当前core是否在线程处理中，如果不在，则将core加入线程池中处理。activeUpdater是一个AtomicIntegerFieldUpdater对象，他的compareAndSet可以保证原子性。而通过activeUpdater也可以保证同一时刻只有一个core被线程池处理，从而使actor的线程不安全代码也线程安全地运行。

进入线程池后，core首先将自己放进threadlocal对象中，并保存当前线程的引用。然后会循环消费MessageQueue中的消息直至Queue的可处理部分为空。（AsyncMessagingCore:126）。handleMessage在AbstractLoopingActor中被覆盖，会根据消息的类型进行分发调用（前面提到的start消息就是一种）。默认的业务消息，在DefaultActorClojure中调用DefaultActor的onMessage方法处理。

onMessage中，react的闭包会被调用来处理业务。之后nextContinuation被置为null，这时loop闭包被重新调用，react闭包重新被赋给nextContinuation。这部分代码就是前面所说的loop并非空转，而是在消息处理完成后重新准备而已。

此外，core的MessageQueue的实现是DefaultMessageQueue。它使用两个LinkedList作为输入（向actor输入）队列和输出队列，当输入队列为空时，通过同步方法swap交换输入输出队列。swap是整个actor系统里唯一一个同步方法。这样的机制保证actor的core在线程池中处理时，外界仍然可以向actor发送消息，消息会在actor被调度出线程池之前全部处理掉。不过，他的前提是只有一个线程读这个队列，这个条件在actor系统里，通过core对象的activeUpdater可以有效的保证。

Actor模式采用这种onDemand方式的线程使用，允许大量的actor共存，并只有活跃的actor会占用线程，非活跃状态的actor处在dettach状态，并不消耗计算资源，取消了空转的loop。

jip 0.2 released

Posted on by
Reply

As you may know, jip is a dependency management tool for Jython/Java development. It resolves and downloads Java packages from maven-compatible repositories. jip also follows some best practices, encouraging you to use a portable and standalone environment (virtualenv) for your Jython development.

It has been four months since the initial release. In version 0.2, I made following changes for you:

  • Improved console output format
  • Correct scope dependency management inheritance
  • Snapshot management, alpha
  • Environment independent configuration
  • Bug fixes

You can find the typical usage doc on github:
https://github.com/sunng87/jip

jip-0.2 has been published to pypi so you can install or upgrade it with:
easy_install -U jip

Please do remember to use it inside virtualenv!