Thursday, February 18, 2021

My wish for Python 4

I love Python. 

A few years after I started using it, I saw someone writing about it and using the phrase "it fits my brain": this is very much how I feel ... at least, for simple straightforward code that doesn't rely on weird metaclass constructs, or even with the added distraction of type annotations. [Yes, type annotations can be extremely useful, but they do not (currently) "fit my brain".]

I am extremely grateful to the many volunteers that work constantly to improve Python. Thanks to their efforts, Python keeps growing. I see most of the growth as positives: more users, more applications in a growing number of fields. In many universities, Python has displaced languages such as Scheme and (thankfully!) Java as the first language that students learn.  From *my* limited point of view, there is a small negative in that this growth of Python includes a growth in its syntax: when I first encountered Python (version 2.3), it had a comparatively very simple syntax which meant that it was easier to learn (in spite of some warts that were fixed in the 2 to 3 transition) compared with the latest Python version (3.10). 

I am well aware that Python doesn't use semantic version numbers: code written for version 3.x can be incompatible with code written for version 3.y. [This was also the case for the 2.x series.] As a decision has been made to use two-digit minor version numbers, there is no apparent need to think of a version 4 of Python: improvements can continue for many years while keeping 3 as the main version.  However, I wish there could be a version 4 - as I describe below.

Note that when I think of Python, I do not think of a specific implementation (such as CPython), but I think of Python as the language. I do not consider "implementation details" such as the Global Interpreter Lock (GIL) as part of the language. Yes, it would be useful if the main Python implementation could make better use of multi-core CPUs. However, that is not something that is relevant for the purpose of this post.

So, what is my wish for Python 4? ...  

Above all, going from the last 3.x version (let's call it 3.14...) to 4.0 should be done seamlessly: code written for version 3.14 should run as is in Python 4.0.

I would like for Python 4 to get inspired by Racket and introduce "dialects".  Python could even borrow the notation used by Racket (#lang dialect) as a top directive in a given module to specify the dialect used in that module.  Unlike Racket, I would limit the number of possible dialects to 4.

The main dialect would not need to be specified: it would simply be the standard Python that everyone knows and loves (or not). It would continue evolving, changing slightly as it goes from version 4.x to 4.y.

A second dialect would be an "experimental" dialect. This dialect could be use to introduce some new syntax with no guarantee whatsoever of backward (or forward) compatibility. It would allow people to experiment with proposed new syntactic constructs before deciding to incorporate them (or not) in the main dialect. I honestly think that this would help reduce some friction in the Python community as changes are proposed and adopted.  The main benefit of such a dialect would be more social than technical.

A third dialect would be a "beginner" dialect. The goal of the beginner dialect would be to make it easier to learn basic programming concepts as opposed to learning a quirky syntax to express these concepts. This beginner dialect would be, in version 4.0, a strict subset of the main dialect. It would not include type annotations, and it might perhaps also exclude the new pattern matching syntax and other syntactic constructs.  For example, using the keyword is might only be limited to checking if the object is one of the three singletons (None, True, False); other uses of is could and should be done with "== id(thing)". Based on feedback from educators, it might perhaps make sense to eventually introduce a few additional keywords and constructs not available from the main dialect, such as:

  • Having nobreak as a keyword equivalent to else in loops.
  • Having function as a keyword equivalent to lambda.
  • Having imported as a keyword with a meaning equivalent to not __name__ == "__main__".
  • Having repeat as a keyword  as is available in TygerJython and Reeborg's World for the construct repeat nb_steps:. It might perhaps be also useful to have repeat forever: as equivalent to Python's while True: . [Other possible uses of this keyword are described here.]
Finally, a fourth dialect would be a "static" dialect. This fourth dialect would always be using a syntax strictly compatible with the main dialect. In this dialect, some dynamical features of Python (such as the possibility to change the type of the object specified by a given name) would not be available so that some optimizations could be applied to increase the execution speed.  I am sure that experts would be able to suggest other restrictions that could be used to greatly increase the execution speed.  I think that such a dialect would be one that would generate the most enthusiastic response from Python users.

That being said, I doubt very much that I'll ever see Python adopting these ideas. However, it is sometimes nice to dream ...

Thursday, February 11, 2021

Friendly-traceback's www function

Today, I saw some write up that Friendly-traceback was discussed on the PythonBytes podcast. A comment made during that podcast suggested that it would be useful if an internet search could be performed, perhaps using a function named www. (Another name was mentioned). So, of course I immediately created an issue ... and implemented a first version of this function.

Thursday, February 04, 2021

Python's tug of war between beginner-friendly features and support for advanced users

Python is my favourite programming language. Since I discovered it in 2004, programming in Python became my favourite hobby. I've tried to learn a few other languages and have never found one as friendly to beginners as Python. As readers of this blog know, these days I am particularly interested in tracebacks, and I am likely paying more attention than most to Python's improvements in this area: Python is becoming more and more precise in the information that it provides to the users when something goes wrong. For example, consider these 2 examples from Python 3.7

Given the message when we try to assign a value to None, we might have expected to see the same when trying to assign a value to the keyword "pass"; instead we get a not so useful "invalid syntax". Of course, if you've been reading this blog before, you won't be surprised that Friendly-traceback can provide a bit more useful information in this case.

However, this is not the point of this post...  Let's see what kind of information Python 3.8 gives us for the first case.

As you can see, it is much more precise: this is a definite improvement.

Let's have a look at another case, using Python 3.8 again:

Again, the dreaded "invalid syntax".  However, this has been significantly improve with the latest Python version, released yesterday.

Again, much better error messages which will be so much more useful for beginners that do not use Friendly-traceback [ even though they should! ;-) ]

There has been a few other similar improvements in the latest release ... but this one example should suffice to illustrate the work done to make Python even friendlier to beginners.  However, this is unfortunately not the whole story.

To make Python useful to advanced users having to deal with large code base, Python has introduced "optional" type annotations. This is certainly something that the vast majority of professional programmers find useful - unlike hobbyists like me.  Let me illustrate this by an example inspired from a Twitter post I saw today.  First, I'll use Python 3.8:

If you know Python and are not actively using type annotations, you likely will not be surprised by the above.  Now, what happens if we try to do the same thing with Python 3.9+

No exceptions are raised! Imagine you are a beginner having written the above code: you would certainly not expect an error then when doing the following immediately after:

Unfortunately, Friendly-traceback cannot (yet!) provide any help with this.

EDIT: this might be even more confusing.


Eventually, I'll make use of the following to provide some potentially useful information.

Ideally, I would really, really like if it were possible to have truly "optional" type annotation, and a way to turn them off (and make their use generate an exception). Alas, I gather that this will never be the case, which I find most unfortunate.

Monday, January 25, 2021

Friendly-contest: we have a winner!

 Since my last post, no new issue has been filed. As the deadline has passed (8 am, AST), I have written a short program to randomly draw a winner. In my last post, I listed incorrectly the entries which I double-checked prior to writing the program, which I tested a few times before the deadline. 

The program I wrote was not the most efficient, but should be easy to understand: I created a list with one item for each valid contest entries, shuffled it and picked the first item on the list as the possible "winner". Just to ensure that I didn't make any silly mistake, I did 100,000 random draws and compared with the original distribution.

The very last of these random draws was determined to be the winner.

Here's the program:

from random import shuffle

entries = {
    "Dominik1123": 19,
    "sdementen": 6,
    "gdementen": 3,
    "tomerv": 5,
    "dcambie": 3,
    "carreau": 1,
results = {
    "Dominik1123": 0,
    "sdementen": 0,
    "gdementen": 0,
    "tomerv": 0,
    "dcambie": 0,
    "carreau": 0,

tickets = []
for name in entries:
    for number in range(entries[name]):

nb_trials = 100_000
rescale = len(tickets) / nb_trials

for i in range(nb_trials):
    results[tickets[0]] += 1

for name in results:
    results[name] *= rescale

print("entries:", entries)
print("draws  :", results)
print("The winner is:", tickets[0])

And the winner is Dominik1123.

Thanks to every one who filed an issue for the contest, or simply tried Friendly-traceback.

Sunday, January 24, 2021

Friendly-contest: 20 hours left

There is only 20 hours left in the Friendly-traceback contest: write bad code to win a prize. After a slow start, there has been quite a few submissions lately which will definitely help to improve Friendly-traceback. Some submissions included references to StackOverflow questions and were thus determined to be worth two entries.  Currently, the number of contest entries stands as follows (using Github usernames):

  • Dominik1123: 19
  • sdementen: 6
  • gdementen: 3
  • tomerv: 3
  • dcambie: 2
  • carreau: 1

The draw will be made randomly so that, while people having more entries have a better chance of winning, anyone with a valid contest entry could win.

Saturday, January 23, 2021

Friendly contest: two days left after a surge of submissions

This is just a quick update.

Yesterday, the number of valid entries jumped from 9 to 23. Many of them have given me ideas on how to make Friendly-traceback better at finding the cause of the error but a few of them are likely going to be impossible for Friendly-traceback to evaluate properly and zero in on the exact cause of the exception. 

Friday, January 22, 2021

Contest: 3 submitters, 3x3 entries, 3 days left

 The contest I announced for Friendly-traceback has resulted in a total of  nine entries so far from three different programmers.  Two of the cases submitted have already been fixed in the development version. They also gave me some ideas to explore other possible cases and I found a few additional ones that need to be fixed.

Meanwhile, on Twitter, I saw a few discussions (for example, here and here) about improvements to messages given by the Python parser for SyntaxError cases, and comparisons with Pypy which currently does a better job in some cases in providing more useful error messages. However, as recorded in this issue on the Python tracker, doing this in a useful way can be quite challenging:

Strangely enough, the apparent impossibility of giving useful error messages in some cases reassures me since I couldn't figure out an approach that would always give the right clues. Perhaps I need to worry less and keep at it with my ad-hoc approach, looking for small improvements. Like Voltaire wrote: Le mieux est l'ennemi du bien  (Perfect is the enemy of good).