Planning to drop CoffeeScript support for Reeborg's World

When I started working on Reeborg's World, I thought it would eventually be a good site to learn the basics of various programming languages, with fun puzzles to solve. However, I found that it myself including more material for Python (and Javascript) including the recent Blockly addition, with no new programming language easily added.   I also have been reading various posts about people suggesting that CoffeeScript was good to use/learn only after having had a good grasp of Javascript.  By then, programmers would find the tasks on Reeborg's World rather trivial to solve and, in the absence of a good dedicated tutorial to CoffeeScript (or other languages), the appeal of using Reeborg's World to learn CoffeeScript would be rather minimal.

So, CoffeeScript support will be dropped, thus reducing slightly the code requiring to be maintained.

Of course, anyone finding it of value could just clone the repo and recover the CoffeeScript support.

Blockly + Reeborg = fun

I've retrofitted Reeborg's World to include Blockly.  You can select Blockly from the top.  If you do so, I suggest you keep the editor open to see the generated code prior to its execution.

Note that this is a first implementation done in just a couple of days of programming.  Bugs are likely present...

Reeborg: two major steps forward, a huge step back with Firefox

UPDATE: The Firefox problem was caused by using a function argument named "watch" ... which seemed like a good name...  However, it appears to conflict with https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/watch

=======

I'm really happy with two new cool features for Reeborg's World ... but this happiness is nearly crushed by the fact that Reeborg's World is completely broken by Firefox 42.0.  It still works with Google Chrome and even works with Microsoft Edge (gasp!) ... but not with Firefox.  I tried to load a year old version of the site that did work with older version of Firefox: no go.

The two new features: it is possible to enable a "variable watch".

Of course, the speed of the animation can be controlled and even paused.

It is also possible to control Reeborg (or, write any Python code, really) using an REPL.

Revisiting an old friend, yet again

The most popular blog post I ever wrote was on a suggested alternative for type hinting in Python. This was likely due to the fact that the blog post was linked in PEP 484 which codified a syntax for type hinting.

The alternative syntax I had been suggesting was to use a clause introduced by a new keyword: where. (I had first discussed this idea more than 10 years ago.)

For example, instead of the approved type hinting syntax

def twice(i: int, next: Function[[int], int]) -> int:
    return next(next(i))

I was suggesting to use something like the following:

def twice(i, next):
    where:
        i: int
        next: Function[[int], int]
        returns: int
    return next(next(i))

Similarly, instead of using a comment to introduce type hinting for single variables

x = 3 # type: int

one would have used the following:

x = 3
where:
    x: int

However, this alternative was never considered as a serious contender for type hinting since it was not compatible with Python 2.7, unlike the accepted syntax from PEP 484.

Still, as an experiment, I decided to see if I could use the approach mentioned in my last few blog posts and use an nonstandard where clause.  And, of course, it is possible to do so, and it is surprisingly easy. :-)

The main details are to be found in three  previous blog posts. However, to summarize, suppose you with to use a where clause like the one described above which would not have an effect on the actual execution of the Python program (same as for the current type hinting described in PEP 484). All you need to do is

1. include the line

from __nonstandard__ import where_clause

in your program.

2a) if the program to be run is the main program, instead of running it via

python my_program.py

you would do

python import_nonstandard.py my_program

instead, where import_nonstandard.py is in the directory "version 5" of this repository, and the relevant where_clause.py is in "version 6".

2b) If instead of having it run as the main program, you would like to import it, then you would include an extra import statement:

import import_nonstandard
import my_program
# rest of the code

and run this program in the usual way. By importing "import_nonstandard" first, a new import hook is created which pre-processes any modules to be imported afterwards - in this case, to remove the where clause or, as I have described in previous posts, to define new keywords or even an entirely different syntax (French Python).

Note: None of the recent experiments are to be taken as serious proposals to modify Python.
Instead, they are a demonstration of what can be done (often in a surprisingly easy way)
with Python as it exists today.

French Python ?

In two previous post, I showed how it was possible to transform some source code prior to having it executed.  My original motivation was to see how one could add a new keyword ("repeat") to use as "repeat n:" and it be equivalent to "for _ in range(n):".

As part of the additional motivation for my experiment, I mentioned the following:

In one of his posts to python-ideas, Terry Jan Reddy mentioned a discussion on the idle-dev list about making idle friendlier to beginners. In one of his post, he mentioned the idea of having non-English keywords. This idea is not new. There already exists an unmaintained version with Chinese Keywords as well as a Lithuanian and Russion version. Maintaining a version based on a different language for keywords is surely not something simple ... nor I think it would be desirable. However, it might be possible to essentially achieve the same goal by using an approach I describe in the next section.

Even just adding new keywords can be quite difficult. For example, in this post, Eli Bendersky explains how one can add a new keyword to Python. "All" you ned to do is

1. Modify the grammar to add the new keyword
2. Modify the AST generation code; this requires a knowledge of C
3. Compile the AST into bytecode
4. Recompile the modified Python interpreter

Not exactly for the faint of heart...

I thought I should revisit the idea I had to see how difficult it might be to create a French Python syntax. As it turned out, it was even simpler than implementing a new keyword. In addition to the code mentioned previously, the new function needed is simply:

def transform_source_code(text):

    dictionary = {...}

    toks = tokenize.generate_tokens(StringIO(text).readline)
    result = []

    for toktype, tokvalue, _, _, _ in toks:

        if toktype == tokenize.NAME and tokvalue in dictionary:

            result.append((toktype, dictionary[tokvalue]))

        else:

            result.append((toktype, tokvalue))

    return tokenize.untokenize(result)

where the dictionary contains the equivalent (e.g. "Vrai": "True", etc.)

The one change I made from the previous version was to replace __experimental__ by __nonstandard__.

For example, here's a test program:

from __nonstandard__ import french_syntax 
de math importe pi  

imprime(pi)  
imprime("The first 5 odd integers are:") 
pour i dans intervalle(1, 11, 2):  
imprime(i)  

imprime("This should be false:", Vrai et Faux)  

si pi == 3:  
imprime("We must be in Indiana") 
ousi pi > 4:  
print("Non standard Euclidean space") 
autrement:  
print("There should be nothing about Indiana nor Euclidean space.")

The code found in version 5 of this repository. Using this approach, it would be trivial to create localized Python versions suitable to introduce absolute beginners to programming concepts. I would not do this myself, here in Canada, but I could see the appeal in some other countries especially those where the English alphabet is not very well known by young learners.

Now, if only I could figure out how to use importlib instead of imp ...

Cryptic Reeborg

Like many people, one of the main reasons I like Python so much is that Python programs are usually very readable ... at least for people that can read and understand English.   This post is definitely not about making Python programs readable in the usual sense.

I like to think of Reeborg's World as a learning environment not only for beginners, but also as a fun and visual place to experiment with various Python features.  For example, is it possible to give a different and interesting introduction to Python's "magic" methods within Reeborg's World?  By this, I mean to replace the usual English names for methods by symbols.  Here's one such quick attempt that may find its way into my Python tutorial.

Reeborg likes to collect objects.  He can add (+) an object found in his world to his collection, or leave an object behind, by removing an object from his collection (-).

Reeborg's favourite objects are (smilie face) tokens: These can be represented in a program by ":-)"  [I would have like to be able to use the unicode character ☺but it is not a valid Python identifier.]

Reeborg can move forward (>) a set number of steps (> n); he can turn perpendicular to his direction of motion (^)  either to his left (^ left) or to his right (^ right).  Reeborg can also "invert" his position and turn around (~reeborg).

Reeborg can detect if there is a wall (|) immediately in front of him (| front) or to his right (| right); he can also build a wall if needed (| build).

Reeborg can detect if he has reached a goal (>> goal) or if he has reached a point where there is a wall in front of him (>> wall) or where there is an object [for instance, a token: >> ":-)"].    Here I would have preferred to use the __matmul__ symbol @ [as in @ goal], but it is not (yet) supported by Brython.  When @ becomes supported, I may use >> as a "fast forward" indicator to vary the speed of the display update [thus >> 100  could be equivalent to think(100)].

Here's a quick implementation of the above, followed by two animated gifs illustrating the result in action.

RUR.world.__remove_default_robot()

build = "build"
front = "front"
right = "right"
left = "left"
token = ":-)"
wall = "wall"
goal = "goal"


class Cryptic(UsedRobot):
    
    def __add__(self, obj):
        if obj == token:
            self.take("token")
        else:
            self.take(obj)    
            
    def __sub__(self, obj):
        if obj == token:
            self.put("token")
        else:
            self.put(obj)
    
    def __gt__(self, n):
        for _ in range(n):
            self.move()
    
    def __or__(self, wall):
        if wall == front:
            return self.wall_in_front()
        elif wall == right:
            return self.wall_on_right()
        elif wall == build:
            self.build_wall()
        else:
            raise ReeborgError("Unknown wall action")
            
    def __rshift__(self, obj):
        while self.front_is_clear():
            self.move()
            
    def __xor__(self, direction):
        if direction == left:
            self.turn_left()
        elif direction == right:
            for _ in range(3):
                self.turn_left()
                
    def __rshift__(self, obj):
        if obj == token:
            return self.object_here()
        if obj == goal:
            return self.at_goal()

    def __invert__(self):
        self.turn_left()
        self.turn_left()

reeborg = Cryptic(1, 1)

 [Note that turn_right() and turn_around() shown as comments on the image below must be defined by the student since Reeborg only knows turn_left().]

from __experimental__ import something_new : running scripts from the command line.

EDIT: I just found out that the notation "from __experimental__ import" had already been suggested in a different context than the one I have been working on.   Perhaps I should use "__nonstandard__" instead of "__experimental__" to avoid any confusion.

---

In a post I wrote yesterday, I mentioned a way to run "experimental" code containing non-standard Python syntax (e.g. new keywords, not recognized by Python's interpreter) by using an "import hook" to convert the code into a proper Python syntax prior to executing it.   One caveat of the approach I used was that it only worked if the "experimental" code was imported.  This restriction is also present in the MacroPy project (which is something I stumbled upon and is definitely a much more substantial project than the little toy I created.)

Today, I have a new version that can effectively be run from the command line.  (I believe that the approach I use could also work for the MacroPy project).  This is version 4 found in this github repository.

I will start with a concrete example taken from the repository (file test.py); the code below contains keywords and constructs that are definitely not valid in Python.

'''This is not a valid Python module as it contains two
   non-standard keywords:  repeat and function.  However,
   by using a custom importer, and the presence of the special
   import line below, these non-standard keywords will be converted
   into valid Python syntax prior to execution.
'''
from __experimental__ import repeat_keyword, function_keyword  # magic! :-)


def normal_syntax():
    '''Creates the list [4, 4, 4] by using the normal Python syntax,
       with a for loop and a lambda-defined function.
    '''
    res = []
    g = lambda x: x**2
    for _ in range(3):
        res.append(g(2))
    return res


def experimental_syntax():
    '''Creates the list [4, 4, 4] by using an experimental syntax
       with the keywords "repeat" and "function", otherwise
       using the same algorithm as the function called "normal_syntax".
    '''
    res = []
    g = function x: x**2
    repeat 3:
        res.append(g(2))
    return res


if __name__ == '__main__':
    if normal_syntax() == experimental_syntax():
        print("Success")
    else:
        print("Failure")

If you try to run this program from the command line using "python test.py" at your command/shell prompt ... it will definitely fail.  However, using the code from the repository, you can run it via "python import_experimental.py test".  The code inside import_experimental.py, which has many more comments than I would normally write, is the following:

''' A custom Importer making use of the import hook capability

https://www.python.org/dev/peps/pep-0302/

Its purpose is to convert would-be Python module that use non-standard
syntax into a correct form prior to importing them.
'''

# imp is deprecated but I wasn't (yet) able to figure out how to use
# its replacement, importlib, to accomplish all that is needed here.
import imp
import re
import sys

MAIN = False
from_experimental = re.compile("(^from\s+__experimental__\s+import\s+)")


class ExperimentalImporter(object):
    '''According to PEP 302, an importer only requires two methods:
       find_module and load_module.
    '''

    def find_module(self, name, path=None):
        '''We don't need anything special here, so we just use the standard
           module finder which, if successful,
           returns a 3-element tuple (file, pathname, description).
           See https://docs.python.org/3/library/imp.html for details
        '''
        self.module_info = imp.find_module(name)
        return self

    def load_module(self, name):
        '''Load a module, given information returned by find_module().
        '''

        # According to PEP 302, the following is required
        # if reload() is to work properly
        if name in sys.modules:
            return sys.modules[name]

        path = self.module_info[1]  # see find_module docstring above
        module = None

        if path is not None:   # path=None is the case for some stdlib modules
            with open(path) as source_file:
                module = self.convert_experimental(name, source_file.read())

        if module is None:
            module = imp.load_module(name, *self.module_info)
        return module

    def convert_experimental(self, name, source):
        '''Used to convert the source code, and create a new module
           if one of the lines is of the form

               ^from __experimental__ import converter1 [, converter2, ...]

           (where ^ indicates the beginning of a line)
           otherwise returns None and lets the normal import take place.
           Note that this special code must be all on one physical line --
           no continuation allowed by using parentheses or the
           special \ end of line character.

           "converters" are modules which must contain a function

               transform_source_code(source)

           which returns a tranformed source.
        '''
        global MAIN
        lines = source.split('\n')

        for linenumber, line in enumerate(lines):
            if from_experimental.match(line):
                break
        else:
            return None  # normal importer will handle this

        # we started with: "from __experimental__ import converter1 [,...]"
        line = from_experimental.sub(' ', line)
        # we now have: "converter1 [,...]"
        line = line.split("#")[0]    # remove any end of line comments
        converters = line.replace(' ', '').split(',')
        # and now:  ["converter1", ...]

        # drop the "fake" import from the source code
        del lines[linenumber]
        source = '\n'.join(lines)

        for converter in converters:
            mod_name = __import__(converter)
            source = mod_name.transform_source_code(source)

        module = imp.new_module(name)
        # From PEP 302:  Note that the module object must be in sys.modules
        # before the loader executes the module code.
        # This is crucial because the module code may
        # (directly or indirectly) import itself;
        # adding it to sys.modules beforehand prevents unbounded
        # recursion in the worst case and multiple loading in the best.
        sys.modules[name] = module

        if MAIN:  # see below
            module.__name__ = "__main__"
            MAIN = False
        exec(source, module.__dict__)

        return module


sys.meta_path = [ExperimentalImporter()]

if __name__ == '__main__':
    if len(sys.argv) >= 1:
        # this program was started by
        # $ python import_experimental.py some_script
        # and we will want some_script.__name__ == "__main__"
        MAIN = True
        __import__(sys.argv[1])

One could easily write a shell script/bat file which would simplify execution to something like "my_python test"

It would be nice to remove the "imp" dependency and use the proper functions/methods from the importlib module, something which I have not been able to figure out (yet).  Anyone familiar with the importlib module is more than welcome to do it and tell me about it. ;-)

Also, writing more useful code converters than the two toy ones I created would likely be an interesting project.