Friendly-traceback: testing with Real Python

Real Python is an excellent learning resource for beginning and intermediate Python programmers that want to learn more about various Python related topics. Most of the resources of RealPython are behind a paywall, but there are many articles available for free. One of the free articles, Invalid Syntax in Python: Common Reasons for SyntaxError, is a good overview of possible causes of syntax errors when using Python. The Real Python article shows code raising exceptions due to syntax errors and provides some explanation for each case.

In this blog post, I reproduce the cases covered in the Real Python article and show the information provided by Friendly-traceback. Ideally, you should read this blog post side by side with the Real Python article, as I mostly focus on showing screen captures, with very little added explanation or background.

If you want to follow along using Friendly-traceback, make sure that you use version 0.2.34 or newer.

Missing comma: first example from the article

The article starts off showing some code leading to this rather terse and uninformative traceback.

Since the code is found in a file, we use python -m friendly_traceback theofficefacts.py to run it and obtain the following.

Misusing the Assignment Operator (=)

We only show one example here, as the others mentioned in the article would be redundant. We remind you for one last time that, if you are not doing so, you should really look at the Real Python article at the same time as you go through this rather terse blog post.

Friendly traceback provides a "hint" right after the traceback. We can get more information by asking why().

Misspelling, Missing, or Misusing Python Keywords

Identifying misspelled keywords was actually inspired by that article from Real Python.

Note that Friendly-traceback identifies "for" as being the most likely misspelled keyword, but gives other possible valid choices.

Friendly-traceback can also identify when break (and return) are used outside a loop.

To the English reader, Friendly-traceback might seem to add very little useful information. However, keep in mind that all this additional information can be translated. If you read the following and do not understand what "boucle" means, then you might get an idea of the some of the challenges faced by non-English speakers when using Python.

In some other cases, like the example given in the Real Python article, Friendly-traceback can identify a missing keyword.

As long as there is only one instance of "in" missing, Friendly-traceback can identify it properly.

Finally, two more cases where a Python keyword is not used properly.

Missing Parentheses, Brackets, and Quotes

Five examples taken from the Real Python article offered without additional comments.

Mistaking Dictionary Syntax

Using the Wrong Indentation

Real Python gives many examples. They would all be handled correctly by Friendly-traceback in a similar way as the single example we decided to use for this post.

Defining and Calling Functions

Changing Python Versions

Friendly-traceback requires Python version 3.6 or newer. Not shown here is that it can recognize that the walrus operator, :=, is not valid before Python version 3.8 and give an appropriate message.

Last example: TypeError result of a syntax error.

Let's look at the last example in the Real Python article.

The explanation given by Friendly-traceback might seem weird "the object (1, 2) was meant to be a function ...".  Often one might have assigned a name to that object, which leads to an explanation that should be seen as more reasonable.

The explanation of looking for a "missing comma" when this TypeError is raised was actually added following a suggestion by S. de Menten in the recent contest I held for Friendly-traceback.

There is more ...

Friendly-traceback includes many more cases that those shown above and mentioned in the Real Python article. However, it is limited in that it can only identify the cause of syntax errors there is a single word or symbol used incorrectly or if the error message provided by Python is more informative than the dreaded SyntaxError: invalid syntax.

My wish for Python 4

I love Python. 

A few years after I started using it, I saw someone writing about it and using the phrase "it fits my brain": this is very much how I feel ... at least, for simple straightforward code that doesn't rely on weird metaclass constructs, or even with the added distraction of type annotations. [Yes, type annotations can be extremely useful, but they do not (currently) "fit my brain".]

I am extremely grateful to the many volunteers that work constantly to improve Python. Thanks to their efforts, Python keeps growing. I see most of the growth as positives: more users, more applications in a growing number of fields. In many universities, Python has displaced languages such as Scheme and (thankfully!) Java as the first language that students learn.  From *my* limited point of view, there is a small negative in that this growth of Python includes a growth in its syntax: when I first encountered Python (version 2.3), it had a comparatively very simple syntax which meant that it was easier to learn (in spite of some warts that were fixed in the 2 to 3 transition) compared with the latest Python version (3.10). 

I am well aware that Python doesn't use semantic version numbers: code written for version 3.x can be incompatible with code written for version 3.y. [This was also the case for the 2.x series.] As a decision has been made to use two-digit minor version numbers, there is no apparent need to think of a version 4 of Python: improvements can continue for many years while keeping 3 as the main version.  However, I wish there could be a version 4 - as I describe below.

Note that when I think of Python, I do not think of a specific implementation (such as CPython), but I think of Python as the language. I do not consider "implementation details" such as the Global Interpreter Lock (GIL) as part of the language. Yes, it would be useful if the main Python implementation could make better use of multi-core CPUs. However, that is not something that is relevant for the purpose of this post.

So, what is my wish for Python 4? ...  

Above all, going from the last 3.x version (let's call it 3.14...) to 4.0 should be done seamlessly: code written for version 3.14 should run as is in Python 4.0.

I would like for Python 4 to get inspired by Racket and introduce "dialects".  Python could even borrow the notation used by Racket (#lang dialect) as a top directive in a given module to specify the dialect used in that module.  Unlike Racket, I would limit the number of possible dialects to 4.

The main dialect would not need to be specified: it would simply be the standard Python that everyone knows and loves (or not). It would continue evolving, changing slightly as it goes from version 4.x to 4.y.

A second dialect would be an "experimental" dialect. This dialect could be use to introduce some new syntax with no guarantee whatsoever of backward (or forward) compatibility. It would allow people to experiment with proposed new syntactic constructs before deciding to incorporate them (or not) in the main dialect. I honestly think that this would help reduce some friction in the Python community as changes are proposed and adopted.  The main benefit of such a dialect would be more social than technical.

A third dialect would be a "beginner" dialect. The goal of the beginner dialect would be to make it easier to learn basic programming concepts as opposed to learning a quirky syntax to express these concepts. This beginner dialect would be, in version 4.0, a strict subset of the main dialect. It would not include type annotations, and it might perhaps also exclude the new pattern matching syntax and other syntactic constructs.  For example, using the keyword is might only be limited to checking if the object is one of the three singletons (None, True, False); other uses of is could and should be done with "== id(thing)". Based on feedback from educators, it might perhaps make sense to eventually introduce a few additional keywords and constructs not available from the main dialect, such as:

• Having nobreak as a keyword equivalent to else in loops.
• Having function as a keyword equivalent to lambda.
• Having imported as a keyword with a meaning equivalent to not __name__ == "__main__".
• Having repeat as a keyword  as is available in TygerJython and Reeborg's World for the construct repeat nb_steps:. It might perhaps be also useful to have repeat forever: as equivalent to Python's while True: . [Other possible uses of this keyword are described here.]

Finally, a fourth dialect would be a "static" dialect. This fourth dialect would always be using a syntax strictly compatible with the main dialect. In this dialect, some dynamical features of Python (such as the possibility to change the type of the object specified by a given name) would not be available so that some optimizations could be applied to increase the execution speed.  I am sure that experts would be able to suggest other restrictions that could be used to greatly increase the execution speed.  I think that such a dialect would be one that would generate the most enthusiastic response from Python users.

That being said, I doubt very much that I'll ever see Python adopting these ideas. However, it is sometimes nice to dream ...

Friendly-traceback's www function

Today, I saw some write up that Friendly-traceback was discussed on the PythonBytes podcast. A comment made during that podcast suggested that it would be useful if an internet search could be performed, perhaps using a function named www. (Another name was mentioned). So, of course I immediately created an issue ... and implemented a first version of this function.

Python's tug of war between beginner-friendly features and support for advanced users

Python is my favourite programming language. Since I discovered it in 2004, programming in Python became my favourite hobby. I've tried to learn a few other languages and have never found one as friendly to beginners as Python. As readers of this blog know, these days I am particularly interested in tracebacks, and I am likely paying more attention than most to Python's improvements in this area: Python is becoming more and more precise in the information that it provides to the users when something goes wrong. For example, consider these 2 examples from Python 3.7

Given the message when we try to assign a value to None, we might have expected to see the same when trying to assign a value to the keyword "pass"; instead we get a not so useful "invalid syntax". Of course, if you've been reading this blog before, you won't be surprised that Friendly-traceback can provide a bit more useful information in this case.

However, this is not the point of this post...  Let's see what kind of information Python 3.8 gives us for the first case.

As you can see, it is much more precise: this is a definite improvement.

Let's have a look at another case, using Python 3.8 again:

Again, the dreaded "invalid syntax".  However, this has been significantly improve with the latest Python version, released yesterday.

Again, much better error messages which will be so much more useful for beginners that do not use Friendly-traceback [ even though they should! ;-) ]

There has been a few other similar improvements in the latest release ... but this one example should suffice to illustrate the work done to make Python even friendlier to beginners.  However, this is unfortunately not the whole story.

To make Python useful to advanced users having to deal with large code base, Python has introduced "optional" type annotations. This is certainly something that the vast majority of professional programmers find useful - unlike hobbyists like me.  Let me illustrate this by an example inspired from a Twitter post I saw today.  First, I'll use Python 3.8:

If you know Python and are not actively using type annotations, you likely will not be surprised by the above.  Now, what happens if we try to do the same thing with Python 3.9+

No exceptions are raised! Imagine you are a beginner having written the above code: you would certainly not expect an error then when doing the following immediately after:

Unfortunately, Friendly-traceback cannot (yet!) provide any help with this.

EDIT: this might be even more confusing.

/EDIT

Eventually, I'll make use of the following to provide some potentially useful information.

Ideally, I would really, really like if it were possible to have truly "optional" type annotation, and a way to turn them off (and make their use generate an exception). Alas, I gather that this will never be the case, which I find most unfortunate.