Thoughts on Google Summer of Code 2008 - part 1

In just a few days, Google will make some announcements that will please many hundreds of students and disappoint even more. I think we should all focus on the positive side when the announcements are made. It is, after all, a fantastic thing that a company is spending millions of dollars so that some students get the chance to program on Open Source software as a summer job.

Just think of it. Who could have predicted, just five years ago, that a company would spend that kind of money on students who would work on someone else's project?

This is amazing - and many are now taking it for granted.

I find it great that the Python Software Foundation is an organization that can mentor SoC students. With the excellent supporting work of James Tauber as coordinator, many promising students are going to be paired with a mentor, hopefully leading to great projects to be completed this summer.

I have seen some grumblings on some SoC related lists that have made me thought about some of the "problems" I have seen. Note that these are very minor compared with the strong positive points. I will be discussing those in part 2, after the official announcements are made.

• Because the PSF is an umbrella organization, most students work on different projects, unrelated with each other. As a result, they tend to have limited interactions with the greater Python community. I think there should be a "meeting place" where all the students would meet - perhaps a mailing list to which they have all to contribute once a week, sharing their progress, etc.
• Not enough positive publicity is given to "successful students", i.e. those that continue to contribute to the Python community after the summer is over. For Crunchy, it has been one success (Johannes Woolard) out of a total of 3 students over the past 2 years. I don't know of many other success stories from other projects ... Alex Holkner comes to mind ... but I feel I should know more names of successful students. (I know there's another Alex or Alessandro who has contributed to the Python core and was involved with GHOP....)
  
• With the exception of a few people like James Tauber and Titus Brown with whom I have had a few email exchanges, I do not feel that as a past/potential mentor I am as much part of a community as I feel should be the case. There is a mentor discussion list, but it does not seem to be the kind of place to generate community building discussions.
  

In terms of projects submitted, I would describe them to belong in the following categories:

1. Contributions to the "standard" core (cpython code, or standard library)

2 a. Contributions to "non-standard core" (e.g. Jython, PyPy, TinyPy?)
2 b. Contributions to 3rd party libraries (e.g. Numpy, Pygame)
2 c. Contributions to major projects whose end users have to use Python (e.g. SAGE)
2 d. Contributions to projects that can be used to teach Python [Crunchy, of course ;-), but there are others ... that will be for part 2]

3 a. Contributions that propose some new "standards" for Python programmers, never discussed before in the Python community.
3 b. Projects that happen to be written in Python, but whose end users are not exposed (or minimally exposed) to Python.
3 c. Projects that are not written in Python, that may or may not be usable in all OS, and that aren't more useful to Python programmers than they would be to people using other languages.
3 d. et cetera

Assuming that all projects are well-thought of (which is not always the case), I feel that:

• Projects in category 1 deserve to be fully supported. The Python community need more capable people contributing to the core to prevent burnout for the current contributors. Perhaps, in a few years, after working some more on Crunchy, I'll feel capable of joining that group and contributing effectively (and have the time to do so).
• Projects in categories 2 a-d are worthy of support. There are of course more such projects submitted than can possibly be supported, so some difficult choices had to be made. (Kudos to James for guiding this process.) Many people are going to be disappointed, but this was unavoidable.
  
• Projects in categories 3 a-d are a puzzle to me. I don't understand their appeal for the PSF (and I know I am not the only one), but it seems that very few people are willing to take a public stance on this and debate the issue. Note that this comment is made as an observation on the discussions that took place so far and does not necessarily reflect on any decision that has been made.
  

This is it for the negative comments. I can't wait for the announcements from Google to focus entirely on the more positive side.

Firefox 3b5: the pain of using the bleeding edge

After seeing so many positive reviews of the upcoming Firefox 3, I decided to try the latest beta (5) version. It seems indeed to be fast when dealing with complex javascript. While there are a few features I am not too keen about [1], I liked the extra speed (and the reduced RAM usage) so much that I have been using it almost exclusively. That is until now, since I can't rely on it to test Crunchy. Update: this is no longer true, thanks to a reader's comment. The fix was to move the onblur event to the file input, indicated by HERE.

To load a local html file [2] into Crunchy, a two-step process has to be used due to normal javascript security:

<form name="browser_local"
onblur="document.submit_local.url.value =
    document.browser_local.filename.value">
<input name="filename"  type="file" HERE >
</form>

<form action="/local" method="get" name="submit_local">
 <input name="url" type="hidden">
 <input class="crunchy" type="submit">
        value="Load local html tutorial" />
</form>

The first form allows to browse the local drive for a particular file. The second one sends the chosen file's path to the browser as an argument to the "/local" action, something like /local?url=file_path. Unfortunately, when using Firefox 3 beta 5, no argument is passed and we get /local?url= instead. And of course no file can be loaded.

This file browser feature is not something I test regularly when working on Crunchy, nor is it something that can be tested via standard Python unit tests. [3] When I noticed the new bug, it never crossed my mind that this could be a "new Firefox feature" and thought it was something I had broken in Crunchy's code. [4] It was only after I tried a few old releases of Crunchy (to figure out when "I" broke the code) that I figured out that the problem was not due to anything I wrote.

I have not been able to find any note about this new behavior of Firefox. Since this is still a beta, I guess I'll have to wait until the final Firefox 3 release to figure out if I need to change the way I load files. [5]

====
[1] One change I don't like is the rather gaudy auto-suggest list when typing a url.

[2] The same method is used to load reStructuredText files and others.

[3] I really need to investigate twill for this.

[4] One more reason to have a complete unit test coverage. Since I don't, I automatically assumed it was something I had done.

[5] If anyone has any lead as to how to do so reliably in Firefox 3b5 as well as with other browsers, I'd be keen to hear about it.

Shell meme

I'm responding to peer pressure. I pretty much only use the shell for one thing and rarely restart it...

andre$ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
322 python
76 cd
26 ls
11 grep
8 pwd
8 find
5 sudo
3 rm
3 def
2 svn

Using Clone Digger

There's a new tool available for Python programmers: Clone Digger. While it has not been officially released, it is available from the svn repository. Clone Digger finds code duplications in a given project, and creates a fairly comprehensive report (html file). Seeing the duplications on a screen is a powerful motivation for refactoring.

Check it out!

Update:

Just to make it clear: I had nothing to do with this project; I just found out about it via the gsoc-python-mentors list.

Inspiration and persistence

While mindlessly wasting time instead of programming selectively reading the internet, I came across this gem by Seth Godin which I reproduce in its entirety:

Persistence

Persistence isn't using the same tactics over and over. That's just annoying.

Persistence is having the same goal over and over.

That's it.

A wiser person would most likely leave it at that. However, this lead me thinking of my goals when it comes to programming which I thought I should write down if only to help me reflect upon them again at a later time. I can sum them up as follows:

1. Do something that is fun but that gives me some sense of accomplishment rather than just wasting time.
   
2. Find ways to make it easier for others to learn programming (in Python).

In doing so, I have found myself oscillating between two extremes:

1. Trying to follow the "release early, release often" philosophy.
2. Trying to get everything "just perfect" before releasing anything.
   

Trying to get things "just perfect" is something that can lead to procrastination and delays. As an example, rur-ple's version 1.0 release candidate 3 has not been updated since July 2007. The next version should be the final 1.0 ... but somehow, I am not happy with many details and I'd like to get everything right for 1.0. Too often I read about (usually commercial) software which is officially released and is considered by its users to be a Beta version. All open source programmers I have met have a sense of pride in their work that I share. So I postpone the final release and end up working on something else...

I went the other way with a little utility called lightning compiler (now at version 2.1), whose version 1.0 was released as a recipe in the online Python cookbook. Much of the rapid evolution of lightning compiler came from user feedback, as expected from the "release early, release often" philosophy. Yet, following the same philosophy generated relatively little feedback for rur-ple of for crunchy to date. I did get some feedback for rur-ple which has been used at an elementary school in Austria, at a high school and a university in the U.S., among others, but it has often been very indirect.

Still, I am persistent. Following Seth Godin's definition of persistence, my second goal written above can be described as finding a solution as to Why Johnny can't code. Or, as I have written elsewhere

My goal is to provide an introduction to programming which is as "smooth" as possible. We sometime hear the phrase "steep learning curve" to characterize some difficult to grasp concept. I think it is important to have as few "steep learning curves" as possible in the learning process. GvR [Guido van Robot] uses a slightly easier syntax than Python ... but at the expense of having a "step-like learning curve" when one wants to go from GvR's world to Python programming. Since Rur-ple uses Python, there is no transition to speak of.

Both rur-ple and Crunchy, and to a lesser extent lightning compiler (which has been incorporated within rur-ple) have been inspired by that goal.

However, sometimes I stray from that goal. For example, inspired by an earlier post on Georg Brand's remarkable Sphinx, Crunchy now includes a prototype for an automated documentation testing framework along the lines of sphinx.ext.doctest which was released yesterday. My intention is to update Crunchy's implementation so that it can be totally compatible with Sphinx's. And while I believe that this is a neat (and fun!) thing to include in Crunchy, it only very indirectly contribute to my overall goal and ends up delaying the 1.0 release for Crunchy.

Blogging too can be a distraction. However, it is my hope that it may generate a few comments that will contribute to inspire me to make Crunchy even more useful.

Success is the result of inspiration and persistence.

Crunchy: Pycon 2008 release

Crunchy is getting really close to a 1.0 version. To mark the Pycon 2008 event (that I won't be able to attend), I just did a new release (0.9.9). It has a few new goodies that I won't list here, leaving Johannes do the demonstration. As for me, I am heading down South for a vacation with my kids.

Note: the opening Crunchy page indicates that this is version 0.9.8.6 - which is incorrect.

What is left to be done for version 1.0 is cleaning up the existing documentation (proofreading, proofreading, proofreading) and adding a few more pages to it. New features will have to wait until after 1.0.... unless we get feedback from Pycon attendees for "must have" features that we could implement quickly.

As far as I know, there are no bugs (famous last words). If you find any, please let us know.

Pycon and Crunchy

This year's Pycon program looks very interesting. I wish I could be there but, alas, the timing was just wrong for me this year. This is doubly disappointing as I would have been able to meet with Johannes Woolard in the flesh. Yes, forget Guido van Rossum, Alex Martelli and other famous names: the one person I wanted to meet is Johannes. For more than a year an a half, I have had the pleasure of collaborating with Johannes on Crunchy, without ever meeting him. This year, Johannes will be the one showing Crunchy off. I'm sure he'll do a great job.

And, if anyone is looking to hire a bright, young, hard-working programmer, Johannes will graduate from Oxford this year.