Tuesday, December 30, 2014

EasyGUI_Qt progressing well.

A short while ago, I started working on an adaptation of EasyGUI for PyQt.  This new project is called EasyGUI_Qt.  For those that are not familiar with EasyGUI, the idea is to provide a series of graphical "widgets" that can be used in procedural programs.  For example, suppose you have a program where you query a user with Python's input:

    >>> input("Enter your reponse " )

Normally, you'd see a copy of the prompt appear, waiting for the user's response.  Using EasyGUI_Qt's  analogue of input(), this is what would show instead of a prompt:

There is also a demo launcher which can be used to see all of the available widgets:

A complete description of the existing widgets is available.  EasyGUI_Qt will not have all the same widgets available as the original EasyGUI.  While the current version number is 0.4, it is, in fact, nearly complete.  I have written a roadmap indicating what's left to be done until the 1.0 release. Anyone should feel free to make comments about the proposed roadmap including making suggestions for additional widgets (with justification provided).

EasyGUI_Qt targets Python 3.3 (and PyQt4) but should support (except for some pesky unicode issues) Python 2.7 - following some requests by users.

Saturday, December 20, 2014

Review of IPython Notebook Essentials

Disclaimer: following a post on Google+ by a Packt representative, I volunteered to do a review of IPython Notebook Essentials and got a free copy as an ebook (I used the pdf format).

The verdict: Not recommended in its current state.

IPython Notebook Essentials starts very well.  It suggests to use the Anaconda 3.4 distribution (which I have on my computer - more on that later) or connecting online using Wakari.  Both methods worked fine for simple examples.

In chapter 1, we are given a quick tour of what the IPython notebook can do using the modeling of a cup of coffee as an example. The idea, on its own works well.  [As a scientist, I found the use of degree Farenheit a bit odd and reminded me of books I read that were written 50 or more years ago.]  However, while the authors used variables that follow the normal Python convention, separating names by underscore as in temp_cream, the code formatting is at times atrocious as variable names are sometimes split up after an underscore character as in the following two lines of code:

initial_temp_mix_at_shop = temp_mixture(temp_coffee, vol_coffee, temp_
cream, vol_cream)

which are, ironically enough, in bold in the text as the author wants us to focus our attention on their meaning (but no mention of the error in formatting).

While I usually prefer holding a paper book over reading an ebook on my screen, I must admit that being able to copy-paste code samples beats having to retype everything.  So, it was possible to quickly reproduce the examples by copy-pasting and fixing the typos to run the examples.

However, a much better way exists which is often used by Packt books: making code samples available for download.  The IPython notebooks have been designed for easy sharing.  Here, the author has chosen not to make use of this.  This, in my opinion, is something that is a major flaw in this case.

Chapter 2 covers in more details the notebook interface. This is undoubtably the most important chapter of the book for anyone who wishes to learn about the IPython notebook.  It covers a lot of grounds.

However, I encountered many problems, some more serious than others. The first, which is perhaps more an annoyance than a real problem, is that one example intended to show code completion using the tab key is given as follows:

print am

Python programmers will immediately recognize this as being a Python 2 example.  However, as far as I could tell (using ctrl-f "python 2") there isn't a single mention anywhere that Python 2 is used.
I happened to have the Anaconda 3.4 distribution installed (as recommended) but with Python 3.4 and not Python 2.  Python 3 is now 6 years old and there is no excuse to focus on an old, and soon to be obsolete, version of Python without at least mentioning why that choice was made. Minor syntax difference, like adding parentheses for print statements, are easily fixed; some more subtle ones are not.   Furthermore, while I had the Anaconda distribution installed, I was still using the online Wakari site to work through the examples, so that was not a major problem at that point.

While still in chapter 2,  we are invited to replace "%pylab inline" by "%pylab" and run an example again to see a plot appear in a window (I first assumed a separate browser window) instead of being shown in the document.    This does not work using the online Wakari site: the window is actually a Qt window, not something that works in a browser.  So, to try this example, I had to start my local version and recopy the code, which worked correctly.

Shortly thereafter, we are introduced to more "magic" methods and invited to try running a cython example, loading from a local file. This did not work.  The recommended "%%cython" magic method is no longer working in the latest IPython version included with the Python 3.4 Anaconda 3.4 distribution.  After a while, I found the proper way to run cython code with the latest version BUT the example provided raised a (numpy-related?) syntax error.  I copy-pasted the code from my browser to the Wakari online version and it worked correctly, confirming that I had not made an error in reproducing the code given by the author. However, I was not able to figure out the source of the error using the local version.

After finishing Chapter 2, I stopped trying to run every single examples and simply skimmed the book.

Chapter 3 focuses on producing plots with matplotlib, including animations. While not specific to the IPython notebook, this topic felt like an appropriate one to cover.

In Chapter 4, we learn about the pandas library which has very little to do with the IPython notebook.  The situation is similar with Chapter 5 which covers SciPy, Numba and NumbaPro, again three libraries that have very little to do with the notebook as such.  The choice of NumbaPro struck me as a bit odd since it is a commercial product.  True enough, it can be tried for free - however, it is not something that I would consider to be an "essential" for the IPython notebook.

I know very little more about the IPython notebook than what I have learned from this book. However, I do know that it is possible to write extensions for the IPython notebook which is something that I would have thought should be included in any book titled  "IPython Notebook Essentials", well before discussing specialized libraries such as Pandas, SciPy, Numba and NumbaPro.

There might very well be other topics more notebook specific that should be included, but I have no way to know from this book.

The book includes three appendices: a brief IPython notebook reference card, a brief review of Python, and an appendix on Numpy arrays.   Both the Reference card and the Numpy arrays appendices seem like worthwhile additions.  However, the brief review of Python seems a bit out of place.  By including code like:

def make_lorenz(sigma, r, b):
   def func(statevec, t):
      x, y, z = statevec
      return [ sigma * (y - x),
              r * x - y - x * z,
              x * y - b * z ]
   return func

in Chapter 2, the author seems to assume, and rightly so in my opinion, that the reader will be familiar with Python.  However, the appendix only covers the standard Python construct that one may find in a beginner's book intended for programmers that are familiar with other languages.  As such, the Python review appendix seems just like a filler, increasing the page count while adding nothing of value to the book. Thankfully, it is relegated to an appendix instead of being inserted in an early chapter.

In summary, about half of the book contains information of value for someone who wants to learn about the IPython notebook itself; the choice of Python 2 over Python 3 is odd, and almost inexcusable given that it is not even mentioned anywhere; the lack of downloable code samples" (mostly IPython notebooks in this case) greatly reduces the value of this book and is something that could be remedied by the author.  In fact, given the typos mentioned (where variable names are split over two lines), downloadable copies of notebooks should be made available.

As I write this review, Packt is having a sale during which ebooks are available for $5.  At that price, I would say that IPython Notebook Essentials is worth it if one wants to learn about the IPython Notebook; however, based on a quick scan of other Packt books covering the IPython notebook, I  believe that better choices exist from the same editor.

Tuesday, December 09, 2014

Reloadable user-defined library with Brython/Python

[Note: this post is a more detailed explanation of something that is briefly described in a previous post on supporting multiple human languages for Reeborg's World.]

In Reeborg's World, I want to have programmers (read: Python beginners who have never programmed before) to be able to learn about using libraries in Python.  As usual, I was looking at the simplest way to introduce the idea of libraries.  Since almost all of the programs that programmers write make use of their own definition for turning right:

def turn_right():

it makes sense to have this "reusable" function be put in a library. So, instead of a single code editor, the code editor has two tabs: one for the basic program and one for the library. Initially, I only had a Javascript version of Reeborg's World, but I still wanted to introduce the concept of using libraries. So, when I first introduced support for using a library with Javascript, I cheated.  And I continued to cheat when I added Python support.  If the programmer wanted to use the functions (or anything else) defined in their library, I required them to insert the following statement in their program.

from my_lib import *

Before running the program using Brython's exec(), I would scan the code in the editor's program tab. If I found this general import statement, I would replace the line by the entire content of the editor's library tab and execute this modified source instead.   [Note: I have since removed the idea of using a library in the Javascript version since there is no natural syntax for importing a library using Javascript.]

However, this approach had two problems:
  1. It did not support the other ways to use an import statement in Python (see below for an example).
  2. It encouraged a bad practice of including everything, polluting the program's namespace.  I already had shown the idiom "from some_lib import *" when explaining how to have Reeborg understand instruction using other human languages (such as from reeborg_fr import * for the French version, or from reeborg_es import * for the Spanish version; other translations welcome! ;-)
I wanted to encourage good programming practice, such as using

from my_lib import turn_right

One problem I had is that Brython's import mechanism is based on retrieving files on the server using ajax calls.  This by itself might not be a problem ... except that I do not want to store anything created by users on the server: Reeborg's World is meant to be run entirely inside the browser, with no login required. (The content of the editor and library tabs are saved in the browser local storage so that they are available when the user comes back to the site using the same browser with local storage enabled.)

Another problem I had is that, once a module is imported, future import statements for that module make use of the cached version.  If the programmer modifies the code in their library (tab), the corresponding module needs to be reloaded.  I need for this to be done automatically, without the programmer having to do anything special.

One solution to these problems might have been to create a special importer class that could import code directly from the library tab and add it to sys.meta_path.  Then, after a program has been run, remove all traces of the imported module (user's library) so that the next time it is executed, the import takes place all over again.

I decided instead on a different approach.  I created a simple module, called my_lib.py  (and another one, biblio.py, for the French version) and put it in Brython's site package directory.  The content of that module is simply:

from reeborg_en import *

which ensures that all normal robot commands can be used in that module.  When Reeborg's World is first loaded, I import that module so that it is cached.  Then, whenever the programmer's code needs to be executed, instead of simply having exec(src) called, the following is called instead:

def generic_translate_python(src, lib, lang_import, highlight):
    ''' Translate Python code into Javascript and execute

        src: source code in editor
        lib: language specific lib (e.g. my_lib in English, biblio in French)
             already imported in html file
        lang_import: something like "from reeborg_en import *"
    # save initial state of lib
    initial_lib_dict = {}
    for key in lib.__dict__:
        initial_lib_dict[key] = lib.__dict__[key]

    exec(library.getValue(), lib.__dict__)
    if highlight:
        src = insert_highlight_info(src)

    # remove added definitions
    new_keys = []
    for key in lib.__dict__:
        if key not in initial_lib_dict:
            lib.__dict__[key] = initial_lib_dict[key]

    for key in new_keys:
        del lib.__dict__[key]

In the above, highlight refers to some pre-processing of the code which allow to show which line of the code is executed as illustrated in two previous blog posts.  library.getValue() is a method that returns the content of the library tab.

Friday, December 05, 2014

Still baffled by the Python 2/3 discussions

I'm ... baffled...

For the past few years, I've been focused mostly on doing my own things, and not really following what was happening in the "core" Python community.   Reading this post today by Brett Cannon about the "consensus" that has apparently emerged  by the language summit at PyCon 2014 about writing code compatible for both Python 2 and 3, I was reminded about the release of version 1.0 of Crunchy 

Crunchy 1.0 is compatible with Python 2.4, 2.5, 2.6 ... and 3.1. It is also compatible with Jython 2.5 modulo some bugs when trying to work with examples containing unicode strings.

That was in 2009.   At 2.1 MB (zipped), Crunchy was not exactly a small script...

Why has it taken so long for this to become the norm?....

Due to a lack of interest in Crunchy, I have essentially not developed it much further past that point, and it is almost certainly not compatible with newer versions of Python...