Rur-ple: double the number of languages

While I haven't had time to do programming in the past few months, some Rur-ple users have been busy and submitted new material. The upcoming 1.0 release of Rur-ple will include 3 new languages [Turkish (including a translation of most lessons), German and Welsh] as well as previously included languages [English (with all lessons), French and Spanish].

Inspired by the user contributions, I spent a few hours today re-writing the localization code. To add a new language now only requires the following:

1. Adding one line in a Python file (translation.py), that specify the language and language code, as well as the name of the .po file.
   
2. Creating a .po file containing the string translations.
3. Creating an html file (rur.htm) used as a Welcome Page in that language.

The new version will also save the last language selected and will use it upon re-starting Rur-ple.

Securing Crunchy

Following my last post, some alert readers pointed out that Crunchy could be a security risk for its users. Crunchy acts as a web server, fetching html-based Python tutorials and displaying them in your browser with controls allowing to execute the Python code. This is meant as a good thing... but it might not be if the code is allowed to run automatically when the page is loaded. In fact, this might have occurred with previous (<0.7) Johannes and I changed the way that Crunchy works so as to remove any security worries - at least, we hope so. Anyone interested can obviously look at the code; for others, here's what we do:

• Each time Crunchy starts, it generates a random session number (10 digit long).
• The javascript code used to communicate between Crunchy and the browser is fetched from a static file and modified to include the unique session number as part of each command; a new javascript file, with a filename that incorporates the session number is generated.
  
• When Crunchy fetches an html page, it processes it to remove unwanted code. This means removing all (java-)scripts and only allowing html tags that are on a "whitelist". This whitelist excludes any of the usual "onload", "oninit", etc., automatic execution.
  
• Next, Crunchy interactive elements (Python interpreter, code editor, button for Python code execution, etc.) are inserted with the javascript calls, each call given a unique identifier for that session.
• Finally, the processed page is displayed in the browser, with the server waiting for user-generated interactions.
• This procedure is repeated for any page that is loaded during that session.
  
• At the end of a session, when Crunchy is terminated, the javascript file that was generated and used in that session is deleted.

The latest version (0.7) includes this security features and many new additions [a complete list of the changes from version 0.6 is appended at the end of this message]. Currently, Johannes is working on adapting How to think like a computer scientist to use with Crunchy, as well adding new interactive features. For my part, I have been working on internationalisation issues (adding a French interface) and other configuration choices. Due to end-of-summer holidays, work on Crunchy will likely slow down for a few weeks at least but we should be able to reach version 1.0 early in the Fall.

After that, I will be ready to start implementing rur-ple inside the Crunchy environment; if there is enough interest, I might implement a turtle graphics module as well.

===== Changes from 0.6 to 0.7 ================
0. Name change
To prevent confusion with an existing program named CrunchyFrog, "Crunchy Frog" has been renamed as "Crunchy", short for "Crunchy, the Interactive Python Tutorial Maker".

1. *Security fix*
The previous versions of Crunchy allowed tutorials containing arbitrary (and hidden) javascript code to be loaded within a browser window. The new version removes any existing javascript code prior to processing for display within a browser.

Feel free to try and break this - and please report any findings back to us.

2. New functionality.

• It is possible to run external programs from within the browser; sample demos include GUI programs with 1. Tkinter, 2. pyGtk, 3. wxPython, 4. Pygame
• Automatic syntax colouring of (static) Python code, including (as an option) line numbering.
• New plotting canvas with simple to use api to draw mathematical functions
• Drawing & plotting canvas can be set to arbitrary size.
• Multiple canvas can now appear on same page.
• New addition and simplification to the sound api.
• Simplification of error message (tracebacks) more suitable for beginners.

• New plotting canvas with simple to use api to draw mathematical functions
• Drawing & plotting canvas can be set to arbitrary size.
• Multiple canvas can now appear on same page.
• New addition and simplification to the sound api.
• Simplification of error message (tracebacks) more suitable for beginners.

3. New visual design:

• Integrated menu which can be styled through custom css
• Three sample css styles now included (selectable via the browser menu)

4. New content:

• First draft sound tutorial exploring concepts of frequency, beats, harmonics (Fourier series), etc.
• New addition to the basic "Crunchy user" tutorial
• Additions to all reference documentation (sound & graphics api, vlam options, etc.)

Exciting Crunchy News

While I knew all along it could be done, I've had a lot of fun during the last two days playing with the newest, and possibly most exciting feature of "The application formerly known as Crunchy Frog", or simply "Crunchy". It is now possible, from within a Firefox windows, to click a button and launch an external Python script. (I should mention that the bulk of this latest feature was implemented by Johannes Woolard - I just had to make a few minor changes to get it working on Windows.)

I'll provide more details later, but here's the short story. You write an html document in which you insert the Python script you want to execute inside a <pre> element with some minor markup (<pre title="external copycode">); actually, you can have as many scripts you want on any given page. Crunchy processes the page, displays the Python scripts in colour (!) and makes a copy of each available for further editing inside a <textarea>. You can then just click on a button and the script is launched.

This means that Crunchy can really transform *any* Python tutorial into an interactive experience. Crunchy can load html pages from the web (not only local ones) and perform its magic ;-)

So far, I've launched Tkinter windows, wxPython apps, Pygame programs, a terminal with a Python interpreter and, the latest, a gnuplot window that ran the gnuplot demo. You can actually have a number of independent external apps going at the same time, allowing for some interesting experiments (edit the code in the <textarea> and launch the new version).

Stay tuned for more news soon, and the announcement of the next release.

Crunchy news

It's now been a month since my last blog post and a lot has happened in the meantime. We (Johannes and I) have just released a new version of Crunchy Frog. We were planning to do a release (version 0.5) a bit earlier but we kept "fixing this", "adding that little bit", "changing this", etc., until I felt it was well beyond what had been planned for the next release. So, after discussing briefly about it (at Johannes's request), we set up a roadmap for Crunchy Frog as a whole (as oppposed to just for Johannes' project) with a date set for the 1.0 release that would coincide with the end of this Summer of Code project for Johannes.

The new public release (0.6) looks very different from the previous public one (0.4). Crunchy now has its own site. Previously, Johannes had set up an svn repository which has come extremely handy. This summer of code project is supposed to be a learning experience from the student (Johannes) under the direction of a more experienced mentor (that would be me). However, I can say that it has been a huge learning experience for me. So far, I had only worked on my own project, and only very recently had learned to use an svn repository on my own computer. Working as part of a team (supposedly as the team leader [I have the final say, if need be]; in reality, it's pretty much a team of equals, each of us having different strengths) has been an experience in itself.

While we have settled on a roadmap, I can already see some additional features, not mentioned so far, that I would like to implement before the end of the summer. For those interesed in Crunchy Frog: stay tuned, there could be some very nice new stuff added soon.

The 0.6 release got some attention outside of "regular channels" and I just found out that there exists another Python project named CrunchyFrog (no space between the two words). This, I think, is most unfortunate and I am in a bit of a quandry as to what to do about the name. Note that the developper behind CrunchyFrog, who brought its existence to my attention, not only has not complained about me choosing this name, but has actually set up a link to Crunchy Frog on his own website. [This is something we'll definitely have to reciprocate; Johannes is responsible for the website and is taking a well deserved break after the last minute rush to get everything organised for the 0.6 release.] Nonetheless, the possibility of changing the name of the project has to be considered ... even though Johannes has designed a nice graphical interface based on a Frog theme.

With all this attention devoted to Crunchy Frog, Rur-ple's development has been suspended. However, just a few days ago, Michel Weinachter sent me

• a "patch" so that the user-chosen language can be remembered from one session to the next;
• an exe version, produced by py2exe, so that Windows users don't need to install Python nor wxPython to use Rur-ple;
• some simple code to make use of a clipboard within rur-ple; I have not had time to see how to make this work though;
• just today, a link to a Firefox extension useful to users of Crunchy Frog. This extension enables the "normal" use of a tab key within an html <textarea>, which comes in handy when typing Python code.

Merci Michel!

So, all in all, a busy month with lots of long nights of coding after work. I have a great job, which does not and never will require me to do any programming [sometimes, much to my chagrin.] However, as a hobby, programming in Python has really been a fantastic one. It is my hope that the programs I create will be found useful by others.

Almost there: version 0.9.9 of RUR-PLE has been released

The title of this post pretty much says it all. I just need to write a few more lessons, read over once more all the existing lessons, do once more all the suggested exercises ... and, barring any surprises, I will be ready to release version 1.0 of RUR-PLE. Thus will end an adventure that started a little less than 2 years ago, as I decided to start a new hobby and learned about Python.

Chasing an unseen bug

A few days ago I was contacted by M.H., a professor in the U.S. who teaches Introduction to Computer Programming every year and was thinking of using RUR-PLE. [Great, I thought, more user feedback to be expected.] Unfortunately, M.H. had tried the latest version and found that the code highlighting feature [described in this post] did not work.

How could I possibly find a bug that I can not reproduce? I thought... Thus began an exchange of emails that led me to find a way to reproduce the bug and, eventually, to solve it. The final solution was somewhat typical of my experience with Python: try the obvious!

These last three words are probably the best words of advice I could give to anyone who writes computer programs using Python. If you are reading this blog as a potential source of useful tips [you fool!], you should probably stop now as nothing else I will write is likely to come even close to being as useful to you.

However, if you want to get a good laugh at my expense, you are invited to continue reading.

I first started by asking M.H. what everyone always asks me when I claim to find a bug or when I don't understand the behaviour of some module and post some question on various lists.

What OS and what version of wxPython are you using?

I expected that M.H. was using Mac OS or Linux, in which case I was getting ready to diplomatically tell him that he was likely out of luck as I'm [shame on me], a Windows user. However, M.H. told me that he was using Win XP (same here), Python 2.4.3 (I'm using 2.4.2) and wxPython 2.6 (same here).

Ok. I was more stumped then ever. This was bugging me... These days, I'm working on Crunchy Frog which will, eventually, include and supersede RUR-PLE. As it is, I have a lot more ideas than time at my disposal to work on Crunchy Frog, and this "distraction" was not welcome. However, I could not ignore it.

Since I couldn't reproduce the bug, I asked M.H. if he could do me a favour and insert a print statement in the method where the line number was set for the purpose of highlighting. As I wrote the email, I had a "clever" thought. M.H. is in the U.S. He's probably using the ansi version of wxPython whereas I'm using the unicode version. I bet you something's wrong with that version... Sure enough, my first hunch was right: M.H. was using the ansi version. And the print statement was outputting the same result on his computer as it was on mine. So, if one hunch was right... I asked M.H. to install the wxPython unicode version, which he kindly agreed to.

Same result... Of course, you are no doubt thinking. However, you are not trying to quash a bug you can't even reproduce. [That's my excuse, anyway.]

At that time I was visiting my partner who, unfortunately, lives 300 km (200 mi) from where I live. Fortunately, she is extremely patient and puts up with me when I am distracted by a programming problem. I decided to install wxPython and the latest version of RUR-PLE on her computer to investigate some more.

Ooops... wxPython is now at version 2.6.3.2. I am sure the version installed on my computer is 2.6.1.0. I read over the "changes" document to see if I can find anything that I can use. There does not appear to be anything relevant. I install the new version ... sure enough, I can "see" the bug (or, should I say, I can't see anything as no highlighting is taking place).

I read the "changes" document a second (and third) time, just to be sure. No luck.

Time to dig in my code. [... Where is this object initialised?... This could have been written so much better... What was I thinking then (about a year ago)... ] After staring at the code for a while, being convinced that the highlighting information is properly set (it was working before, after all) I am convinced it is a wxPython bug that was introduced. I file a bug report and expect to leave it at that.

However, my conscience is nagging me. I decide to look at the code again, staring at the method where the highlighting updates should take place. Here it is, with a few added notes.

def update_refresh(self, robot, name):
  if 'robot' in self.robot_dict:  # (1)
      arg = self.parent.status_bar.beeper_field, self.robot_dict['robot']._beeper_bag
      event_manager.SendCustomEvent(self.parent, arg)
  time.sleep(robot.delay)  # (2)
  wx.Yield()   # (3)
  self.world.DoDrawing()  #(4)
  self.WorldDisplay.drawImage()  #(5)
  if name:
      self.WorldDisplay.scrollWorld(name)  # (6)
  self.WorldDisplay.Refresh()  # (7)
  wx.Yield()

1. The information in the status bar (number of beepers carried by robot, etc.) is updated.
   
2. The animation is paused at each step.
3. Making sure other controls of the application are responsive so that the program can be paused or stopped altogether by pressing the appropriate button, etc.
4. Update the (in-memory) image based on the latest robot action.
5. Update the screen image.
6. Prevent the robot from going off the screen.
7. Refresh the screen [Refresh() is a wxPython method.]
   

That's it. Not much there. I refresh the screen so as to ensure that the robot world, right next to the code "window", gets updated on the screen when a new line of code is executed, and should be highlighted in the code window.

Can you see the bug?

I didn't refresh the code window. It worked before though.

How do I do this... the robot world is a ScrolledWindow whereas the code window is a StyledTextControl... Where do I find the relevant documentation...

That's when I thought of the three important words I mentioned before: try the obvious. I typed in

  self.ProgramEditor.Refresh()

and launched RUR-PLE. Problem solved. I guess it was just dumb luck that this refresh statement was not required before. All I have to do now is fix the bug and update the version on sourceforge ... before I go back and rewrite the parts of the code that I didn't find easy to read just a few days ago.

Praise for Python

Sometimes, I come accross something which I really wish I could have written. Usually, I just smile, try to commit it to memory, and move on. This time, I have to quote what Jim Roskind wrote on Guido's blog:

I was initially a bit put off by the Python indentation approach (I was a grammar guy… and got used to parsing braces and parens, and didn’t like the idea of white space having so much meaning), but I soon came to love it. Eventually I came to apply one of my favorite computer science sayings to Python as an endorsement to their indentation blocking approach: “The fundamental evil in computer science is the replication of code or data.” In almost all languages I had previously worked, the indentation of code had always been a critical PART of program nesting structure (at least for the reader). As I read and wrote more Python, its use of indentation as the ONLY way to specify blocks began to look better and better. All the silly bugs related to indentation errors (misleading the human reader) were gone in Python. The redundant use in other languages of braces AS WELL AS indentation (the former to help the parser, and the latter to help the human reader) was an effective duplication of the author’s intent. That duplication in other languages, a fundamental original sin, was missing in Python. It was cool! ;-)

Yes, indeed. Python is cool. :-)