Python's tug of war between beginner-friendly features and support for advanced users

Python is my favourite programming language. Since I discovered it in 2004, programming in Python became my favourite hobby. I've tried to learn a few other languages and have never found one as friendly to beginners as Python. As readers of this blog know, these days I am particularly interested in tracebacks, and I am likely paying more attention than most to Python's improvements in this area: Python is becoming more and more precise in the information that it provides to the users when something goes wrong. For example, consider these 2 examples from Python 3.7

Given the message when we try to assign a value to None, we might have expected to see the same when trying to assign a value to the keyword "pass"; instead we get a not so useful "invalid syntax". Of course, if you've been reading this blog before, you won't be surprised that Friendly-traceback can provide a bit more useful information in this case.

However, this is not the point of this post...  Let's see what kind of information Python 3.8 gives us for the first case.

As you can see, it is much more precise: this is a definite improvement.

Let's have a look at another case, using Python 3.8 again:

Again, the dreaded "invalid syntax".  However, this has been significantly improve with the latest Python version, released yesterday.

Again, much better error messages which will be so much more useful for beginners that do not use Friendly-traceback [ even though they should! ;-) ]

There has been a few other similar improvements in the latest release ... but this one example should suffice to illustrate the work done to make Python even friendlier to beginners.  However, this is unfortunately not the whole story.

To make Python useful to advanced users having to deal with large code base, Python has introduced "optional" type annotations. This is certainly something that the vast majority of professional programmers find useful - unlike hobbyists like me.  Let me illustrate this by an example inspired from a Twitter post I saw today.  First, I'll use Python 3.8:

If you know Python and are not actively using type annotations, you likely will not be surprised by the above.  Now, what happens if we try to do the same thing with Python 3.9+

No exceptions are raised! Imagine you are a beginner having written the above code: you would certainly not expect an error then when doing the following immediately after:

Unfortunately, Friendly-traceback cannot (yet!) provide any help with this.

EDIT: this might be even more confusing.

/EDIT

Eventually, I'll make use of the following to provide some potentially useful information.

Ideally, I would really, really like if it were possible to have truly "optional" type annotation, and a way to turn them off (and make their use generate an exception). Alas, I gather that this will never be the case, which I find most unfortunate.

Friendly-contest: we have a winner!

 Since my last post, no new issue has been filed. As the deadline has passed (8 am, AST), I have written a short program to randomly draw a winner. In my last post, I listed incorrectly the entries which I double-checked prior to writing the program, which I tested a few times before the deadline. 

The program I wrote was not the most efficient, but should be easy to understand: I created a list with one item for each valid contest entries, shuffled it and picked the first item on the list as the possible "winner". Just to ensure that I didn't make any silly mistake, I did 100,000 random draws and compared with the original distribution.

The very last of these random draws was determined to be the winner.

Here's the program:

from random import shuffle

entries = {
    "Dominik1123": 19,
    "sdementen": 6,
    "gdementen": 3,
    "tomerv": 5,
    "dcambie": 3,
    "carreau": 1,
}
results = {
    "Dominik1123": 0,
    "sdementen": 0,
    "gdementen": 0,
    "tomerv": 0,
    "dcambie": 0,
    "carreau": 0,
}

tickets = []
for name in entries:
    for number in range(entries[name]):
        tickets.append(name)

nb_trials = 100_000
rescale = len(tickets) / nb_trials

for i in range(nb_trials):
    shuffle(tickets)
    results[tickets[0]] += 1

for name in results:
    results[name] *= rescale


print("entries:", entries)
print("draws  :", results)
print("The winner is:", tickets[0])

And the winner is Dominik1123.

Thanks to every one who filed an issue for the contest, or simply tried Friendly-traceback.

Friendly-contest: 20 hours left

There is only 20 hours left in the Friendly-traceback contest: write bad code to win a prize. After a slow start, there has been quite a few submissions lately which will definitely help to improve Friendly-traceback. Some submissions included references to StackOverflow questions and were thus determined to be worth two entries.  Currently, the number of contest entries stands as follows (using Github usernames):

• Dominik1123: 19
• sdementen: 6
• gdementen: 3
• tomerv: 3
• dcambie: 2
• carreau: 1

The draw will be made randomly so that, while people having more entries have a better chance of winning, anyone with a valid contest entry could win.

Friendly contest: two days left after a surge of submissions

This is just a quick update.

Yesterday, the number of valid entries jumped from 9 to 23. Many of them have given me ideas on how to make Friendly-traceback better at finding the cause of the error but a few of them are likely going to be impossible for Friendly-traceback to evaluate properly and zero in on the exact cause of the exception. 

Contest: 3 submitters, 3x3 entries, 3 days left

 The contest I announced for Friendly-traceback has resulted in a total of  nine entries so far from three different programmers.  Two of the cases submitted have already been fixed in the development version. They also gave me some ideas to explore other possible cases and I found a few additional ones that need to be fixed.

Meanwhile, on Twitter, I saw a few discussions (for example, here and here) about improvements to messages given by the Python parser for SyntaxError cases, and comparisons with Pypy which currently does a better job in some cases in providing more useful error messages. However, as recorded in this issue on the Python tracker, doing this in a useful way can be quite challenging:

Strangely enough, the apparent impossibility of giving useful error messages in some cases reassures me since I couldn't figure out an approach that would always give the right clues. Perhaps I need to worry less and keep at it with my ad-hoc approach, looking for small improvements. Like Voltaire wrote: Le mieux est l'ennemi du bien  (Perfect is the enemy of good).

Don't you want to win a free book?

 At the end of Day 2 of the contest, still only one entry. If this keeps up, by next Monday there will not be a draw for a prize, and we will have a winner by default.

The submission was based on the use of __slots__. In playing around with similar cases, I found an AttributeError message that I had not seen before.  Here's a sample code.

class F:
    __slots__ = ["a"]
    b = 1

f = F()
f.b = 2

What happens if I execute this code using Friendly-traceback. Normally, there would be an explanation provided below the list of variables. Here we see nothing.

Let's inspect by using the friendly console.

I'll have to take care of this later today. Perhaps you know of other error messages specific to the use of __slots__. If so, and if you are quick enough, you could enter the contest. ;-)

Friendly contest: the race is on

tl; dr: Python was wrong ;-)

After one day, I've had one valid entry submitted to the contest I announced yesterday; I've also had two other submissions from the same contributor that I deemed to be invalid for the contest. The submissions shared a similar characteristics to a different degree: the information provided by Python in the exception message did not tell the whole story and, taken on its own, might have been considered to be misleading. 

One such cases which I did not considered to be valid for this contest was the error message UnboundLocalError: local variable 'x' referenced before assignment  given for code containing the following

def f():
    x = 1
    del x
    print(x)  # raises the exception

When the exception is raised, there is indeed no trace of variable "x". So while there was an assignment for such a variable before, after deletion it no longer exists. Reading this code, instead of an UnboundLocalError, the exception that should probably be raised at this point is NameError: name 'x' is not defined ; however, Friendly-traceback's role is not to second-guess Python but to explain what an exception means and, whenever possible, give more details using the information available when the exception was raised. I considered this case and another one to be beyond the scope of what Friendly-traceback could accomplish.

The entry that I deemed to be valid was based on the following code:

class F:
    __slots__ = ["a"]

f = F()
f.b = 1

The error message given by Python in this case is AttributeError: 'F' object has no attribute 'b'. While technically correct, the problem is not that this object has no such attribute but that it cannot have such an attribute. This information can be easily obtained when the exception is raised and the information provided by Friendly-traceback now includes the following:

The object f has no attribute named b. 
Note that object f uses __slots__ which prevents the creation of new 
attributes. The following are some of its known attributes: a.

Reminder: the contest is open for 8 more days.