Don't repeat yourself: distribute jython package with jip.dist

As a new feature in jip 0.4, we can use some helpers from jip.dist to simplify package distribution. With jip.dist, you can define Java dependencies for your jython package. In an environment with jip, dependencies will be automatically installed when user uses pip to get you package.

We have two different approaches allow you to choose.

Approach 1, Define dependencies in POM

This is the standard maven way. To describe your jython package and its dependencies, create a pom.xml in your project. The directory hierarchy looks like:

├── app
│   ├── module1
│   │   ├── __init__.py
│   ├── core.py
│   ├── __init__.py
├── LICENSE
├── MANIFEST.in
├── pom.xml
├── README
└── setup.py

In pom.xml, just add dependencies as you do with Maven.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" <br="" />  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"&gt;

    ...

    <dependencies>
        <dependency>
            <groupid>org.slf4j</groupid>
            <artifactid>slf4j-api</artifactid>
            <version>1.6.1</version>
        </dependency>

        <dependency>
            <groupid>org.slf4j</groupid>
            <artifactid>slf4j-log4j12</artifactid>
            <version>1.6.1</version>
        </dependency>

        ...

    </dependencies>

    <repositories>
        <repository>
            <id>sonatype-oss-sonatype</id>
            <url>http://oss.sonatype.org/content/repositories/snapshots/</url>
        </repository>
    </repositories>
</project>

You can also define repositories in pom.xml if you use custom repository. (for example, jboss, java.net)

Remember to add pom.xml in your MANIFEST.in, to ensure the file will be packaged into source package:

include pom.xml

Approach 2, Define dependencies with Python

You may be tired with endless XML configuration. jip allows you to define dependencies with python, just like gradle with groovy.

In your setup.py, add something like:

requires_java = {
    'dependencies':[
        ## (groupdId, artifactId, version)
        ('org.slf4j', 'slf4j-api', '1.6.1'),
        ('org.slf4j', 'slf4j-log4j12', '1.6.1'),
        ('info.sunng.soldat', 'soldat', '1.0-SNAPSHOT'),
        ('org.apache.mina', 'mina-core', '2.0.2')
    ],
    'repositories':[
        ('sonatype-oss-snapshot', 'http://oss.sonatype.org/content/repositories/snapshots/')
    ]
}

Then pass it to setup(). The keyword argument require_java is jip specific.

setup(
    ...
    requires_java=requires_java,
    ...)

Use jip’s setup wrapper

To use jip’s power, the only difference is to use setup() from jip.dist instead of setuptools or distutils.

from jip.dist import setup

Then publish your jython package to Python Cheese Shop:
$ jython setup.py sdist upload

Internally, jip uses setuptools. So you can still do jython setup.py develop .

And jip 0.4 is available under MIT License. You are free to use jip.dist in your code.

For your user

You should write a guide forcing users to use your jython application within virtualenv. And install jip as a prerequisite:
$ pip install jip

Then simply install your package with pip:
$ pip install <your-package-name>

No additional step required!

So please just release your jython package with jip !

For more information:

Enhanced jip to simplify Jython module distribution

As you might notice, the installation of gefr is too complex, requiring several manual actions. User have to remember the long maven coodinator to resolve dependencies. wtf!

So I have been working whole day to simplify the process. 

Currently, gefr’s approach is to upload pom.xml to a public maven repositoy (sonatype oss). And the source is uploaded to pypi. Pip will find the source and install it. Jip will find the plain pom and resolve it. Because once pip finished the installation, the source package will be erased and jip can never find the pom. So I have to distribute them seperately.

It would be better to invoke jip right before pip exits. I did some investigation about post-install script. And lucky enough, distutils allows you to override default install command. We can use it to invoke jip, in the scope of setup scrript. It does make sense.

Another problem is the original design of .jip configuration file. From jip 0.2, .jip is available as environmen-scoped: we define some custom repositories and the whole environment shares them. But if we have multiple projects in the same environment, jip may waste time to find public artifact in a private repository. Even worse, jip may load invalid artifact from private repos. In the new design, private repositories are defined in pom.xml, as project-scoped, just like most java build tools.

With new jip, to distribute a jython package, you should write a pom in the same way of Java, specifying the dependencies and custom repositories if you have. Then modify your MANIFEST.in to include it to your source package. At last, in the setup.py, define a new install command to call jip and pass the command into the setup() . From the new approach, only ‘pip install’ is required for end user. Super easy!

Upon the new usage of jip(as a library), I am also considering to migrate from GPL to LGPL or MIT. I have little knowledge about conflicts between licenses. So if you have some ideas or concerns, feel free to let me know.

Gefr with multiple backends support

I have been working on gefr this week to add apache MINA backend for it. The goal of gefr is to provide adapters between Python’s WSGI and Java network infrastructures. In cPython world, most WSGI servers are implemented with C code(fapws, meinheld, etc.), which is not applicable on Jython platform. However, the Java world has its own reliable network foundations. So my gefr turns them into WSGI servers. The ultimate goal of gefr is to make it possible to port any of your WSGI web applications to Jython seamlessly.

There has been many attempts to use Jython in Java web development. Most of them is focusing on applying Jython in Servlet framework. This is helpful, but it doesn’t take the whole advantage of Python web programming. The most valuable thing in Python web development is its various web frameworks and agile tools. But inside servlet framework, we can hardly apply them, the only things we have is Jython’s beautiful syntax. That should not be enough.

OK. You are eager to meet gefr? Let’s get hands dirty with some setup. I just assume that you have some essential software installed in the right place. Is it true for you?

Create Jython virtual environment:

$ virtualenv -p /usr/local/bin/jython gefr-test
$ cd gefr-test
$ source bin/activate

Install jip. jip is a great tool to manage Java dependencies for your Jython application. jip>=0.3 is required in this guide, and pip will handle this for you.
$ pip install jip

Install gefr. Since gefr is a pure Python package, pip is still helpful.
$ pip install gefr

Now create a jip configuration file called .jip in your virtual home. We should add sonatype oss repository.

[repos:oss]
uri=http://oss.sonatype.org/content/repositories/snapshots/
type=remote

[repos:central]
uri=http://repo1.maven.org/maven2/
type=remote

[repos:local]
uri=/home/sun/.m2/repository/
type=local

New in jip 0.3, we can use a single command to resolve dependencies of gefr.
$ jip install-dependencies info.sunng.gefr:gefr:0.2-SNAPSHOT

OK. Then everything is ready. Create your first Jython WSGI application. Actually, there is no difference.
I just copy the example of gefr here, you can find it in the code repository.

#! /usr/bin/python

__author__="Sun Ning <classicning@gmail.com>"
__date__ ="$Jan 5, 2011 10:14:27 PM$"

import sys
from gefr.core import Gefr

def simple(environ, start_response):
    """a static app"""
    headers = []
    headers.append(('Content-Type', "text/html"))
    start_response("200 OK", headers)

    return ["<h1>it works.</h1>"]

if __name__ == "__main__":
    from optparse import OptionParser
    parser = OptionParser()
    parser.add_option('-b', '--backend', dest='backend', help='server backend')
    (options, args) = parser.parse_args()

    if options.backend == 'soldat':
        from gefr.backends.soldat import SoldatBackend as backend
    elif options.backend == 'mina':
        from gefr.backends.mina import MinaBackend as backend
    else:
        print 'backend %s not supported.' % options.backend
        sys.exit(1)

    g = Gefr(simple, backend)
    g.start()

With the command line options, you can control which backend to be used. Now start it with:
$ jython-all example.py -b soldat
or
$ jython-all example.py -b mina

Soldat is another network framework written by myself. And MINA, you must know. Open your browser, redirect it to http://localhost:8080 , now you see it works.

Sure, that’s not enough for your needs. Let’s have an advanced example. Among all the web frameworks in Python, I like bottle most. So we just build a simple application with GET and POST in bottle.

$ pip install bottle
$ pip install mako
$ mkdir bottleapp
$ cd bottleapp

The bottle application looks like:
echo.py

# -*- coding:utf-8 -*-

from bottle import Bottle, mako_view, request
from gefr.core import Gefr
from gefr.backends.mina import MinaBackend

bapp = Bottle()

@bapp.get('/')
@mako_view('edit')
def display_form():
    return dict(ip=request.environ.get('REMOTE_ADDR'))

@bapp.post('/')
@mako_view('show')
def display_content():
    content = request.POST.get('content')
    return dict(content=content)

Gefr(bapp, MinaBackend).start()

The simple mako templates are:

edit.tpl

<html>
<head>
    <title>Bottle on MINA</title>
</head>
<body>
    <h1>Bottle on MINA</h1>
    <p>User from ${ip}, leave your words please:</p>
    <form action="/" method="post">
        <textarea cols="60" rows="20" name="content"></textarea>
        <p><input type="submit" /></p>
    </form>
</body>
</html>

show.tpl

<html>
    <head>
        <title>Bottle on MINA</title>
    </head>
    <body>
        <h1>Bottle on MINA</h1>
        <pre>${content | h}</pre>

    </body>
</html>

Start the bottle app:
$ jython-all echo.py

This is rather simple but it covers the most basic features of a web server.

I should say thank you for reaching here. Gefr is still an experimental project in early development. So there must be bugs and potential performance improvements in it. Feel free to contact me if you are also interested in it. The project is hosted on bitbucket:
https://bitbucket.org/sunng/gefr

jip 0.2 released

As you may know, jip is a dependency management tool for Jython/Java development. It resolves and downloads Java packages from maven-compatible repositories. jip also follows some best practices, encouraging you to use a portable and standalone environment (virtualenv) for your Jython development.

It has been four months since the initial release. In version 0.2, I made following changes for you:

  • Improved console output format
  • Correct scope dependency management inheritance
  • Snapshot management, alpha
  • Environment independent configuration
  • Bug fixes

You can find the typical usage doc on github:
https://github.com/sunng87/jip

jip-0.2 has been published to pypi so you can install or upgrade it with:
easy_install -U jip

Please do remember to use it inside virtualenv!

gefr API updates

早上收到jythonet作者的一封邮件,受到启发,我打算扩展一下gefr这个WSGI adapter。原本gefr是写给soldat用来测试的,结果发现现在出现了买椟还珠的效果,既然大家更加关注这个,我决定多花一些时间在上面。不过话说回来gefr确实是Jython Web程序的新思路,过去大家都在想怎样把Jython放进Servlet中,gefr的思路是把Java的服务器实现放到WSGI后面。未来会有gefr-netty, gefr-mina, gefr-servlet等等出现,如果真的存在一个可靠的backend,也许这种思路也不失为一个办法,毕竟python的web框架更吸引人,而java的基础设施相对可靠,各取所长。

根据这个目的,现在gefr 0.2的API更新为:

即在创建Gefr的时候需要告知backend类型,目前还只有soldat一种后端。未来也可能通过自动检测让这个参数成为可选。

gefr 0.2已经在开发中,您可以通过这里关注:
https://bitbucket.org/sunng/gefr/overview