More about Crunchy running under Python3.0a1

Just in time for the new year, and after a major reorganization of Crunchy's code to make it easier to write more unit tests for it, I did manage to get all of Crunchy's core functionality working under Python 3.0a1. Crunchy can now import html files that are not conforming to strict xml convention (i.e. not having closing tags for some elements) and thus create problems for ElementTree. The way it does that is to launch an external process under Python 2.5 (or whatever default version that is invoked via typing "python" at a terminal - but it has to be a 2.x version for it to work). This process imports the file, "cleans" it (using a combination of BeautifulSoup, ElementTree and Crunchy's security module), and saves it locally. After a delay, Crunchy loads up the cleaned up file and display it. In this process, most of the styling is unfortunately removed. :-(

However, the good news is that I was able to load up the official 3.0 Python tutorial (work in progress) and try it out using Crunchy. I did find out one limitation of using Crunchy to do so. Crunchy encodes the Python output using utf-8 before forwarding to the browser. So, instead of having things like b'Andr\xc3\xa9' appearing on the screen, it would be converted to André. Thus, Crunchy is not a very good platform to teach about encoding/decoding of strings. For other aspects though, it is an ideal tool (if I may say so) for going through the Python tutorial: there is no need to switch back and forth between the browser and a separate Python environment to try things out. I still hope to have the time and energy to go through the entire 3.0 tutorial (something I have never done before, for 2.x) and see if I can find any bugs or come up with useful suggestions.

All I have left to do for the next release is to write up some documentation/tutorial on the new Turtle module and on launching Crunchy under Python3.0a1. With a bit of luck, this will all be finished before the end of the year.

In addition to the code reorganization mentioned above, I did fix a few bugs and made an improvement on Crunchy's Borg interpreters. For those that aren't familiar with it, Crunchy allows to embed a number of interpreters (html input box communicating with the Python backend) within a single page. These interpreters can either be isolated one from another (meaning that a variable defined in one interpreter is only known by that interpreter) or can share a common environment (aka Borg interpreters). Normally, in a single user mode, using a single open tab in Firefox, every time a new page is displayed, the Borg interpreters are effectively reset (the old ones are garbage collected and the new ones are created from an empty slate).

Previously, if one were to have multiple users (or multiple tabs open from the same user) on the same Crunchy server, all Borg interpreters ended up sharing the exact same state. This is not very convenient if two users are trying to use the same variable names! I had planned to address this "feature" at some point after the 1.0 release but was forced into it earlier due to the 3.0a1 work. The reason is the following.

To create "Borg interpreters", I was using the Borg idiom invented by Alex Martelli. It goes as follows:

class Borg(object):
    '''Borg Idiom, from the Python Cookbook, 2nd Edition, p:273

    Derive a class form this; all instances of that class will share the
    same state, provided that they don't override __new__; otherwise,
    remember to use Borg.__new__ within the overriden class.
    '''
    _shared_state = {}
    def __new__(cls, *a, **k):
        obj = object.__new__(cls, *a, **k)
        obj.__dict__ = cls._shared_state
        return obj

When using this idiom in a standard program under 3.0a1/2, a deprecation warning is raised about object.__new__() not taking any argument. When I was trying to make use of this idiom in Crunchy running under 3.0a1/2, the deprecation warning was actually replaced by an exception. Rather than trying to silence this exception, I decided to take a different approach and used instead the following:

class BorgGroups(object):
    '''Inspired by the Borg Idiom, from the Python Cookbook, 2nd Edition, p:273
    to deal with multiple Borg groups (one per crunchy page)
    while being compatible with Python 3.0a1/2.
    Derived class must use a super() call to work with this properly.
    '''
    _shared_states = {}
    def __init__(self, group="Borg"):
        if group not in self._shared_states:
            self._shared_states[group] = {}
        self.__dict__ = self._shared_states[group]

# The following BorgConsole class is defined such that all instances
# of an interpreter on a same html page share the same environment.

class BorgConsole(BorgGroups, SingleConsole):
    '''Every BorgConsole share a common state'''
    def __init__(self, locals={}, filename="Crunchy console", group="Borg"):
        super(BorgConsole, self).__init__(group=group)
        SingleConsole.__init__(self, locals, filename=filename)

The "group" variable is taken to be a unique id generated by Crunchy each time it processes a given html page. Thus, each page loaded by a different user (or the same user, at a different time) from the same Crunchy server will result in a unique set of Borg interpreters.

To be fair, I must admit that I did not come up with this solution totally on my own. A while ago, I asked for something like this on comp.lang.python. (Those interested in the details should search for "Borg rebellion".) I just derived the above solution from some suggestions made at that time.

Finally, in addition to all this, I found out a bug in code.py for Python 3.0a1/2. I tried to send an email to the python-3000 mailing list about it, but it was held up, waiting for a moderator approval for a few days. So, I canceled it and filed a bug report instead (which I should have done in the first place) on the bug tracker. I still haven't seen any follow up - perhaps due to the title I gave it. The bug is actually very easy to fix - three lines of code need to be replaced by a single one. The solution is related to my only other "official" contribution to Python to date. Hopefully, by this time next year, I'll have learned enough to contribute more to Python.