If you are a professional programmer, you can just stop reading as you know much more than I do, and you surely have a better, more efficient and cutting edge way of doing things right now - and you will likely use yet a different way next year, if not next month. So, you would likely find my advice to look the same as my site: dated and not using the latest and coolest techniques - in short, for you, not worth looking at. ;-)
Summary:
- If you must, choose a well-supported Javascript library and stick with it.
- Use npm for installing Javascript tools
- Avoid depending on third-party packages whenever possible
- Use npm to manage your workflow
- Supplement with your own Python or shell/batch script when needed
- Use browserify to concatenate all your Javascript files
- Use tape for unit testing
- Use faucet for formatting of unit test results
- Use QUnit for integration testing
- Optional: use Madge for identifying circular dependencies
- Optional: use Dependo to identify any overlooked module
- Optional: use JSDoc for creating an API
- Use jshint instead of jslint
Warning
This blog post is long. I've attempted to provide enough details for you to determine in each case if my use-case corresponds to yours and thus if and when my recommendation might make sense for you and your project.
The context
I started working on Reeborg's World many years ago. The first version was created as a desktop program (rur-ple) in 2004. My first primitive attempt at a web version was done around 2007. During the years I have worked on it, tools and libraries have come, evolved, and gone, to be replaced by better ones. As programming is only a hobby for me which I work on when I have some free time, I cannot afford to change the set of tools I use every year to follow the latest trend.
I've started working on the current version when using color gradients for buttons and menu bars was the latest and coolest thing - well before the current flat UI became the norm.
Admittedly, my site looks dated - but since I do not have enough time to add all the new ideas for functional improvements I want to make, investing time to modernize the look is not a priority.
The Javascript code I wrote is split over many files and has become a tangled mess - in spite of some occasional attempts at reorganizing the code, including a near-complete rewrite a few years ago.
Some of the complexity is required as I want to make it easier for would-be collaborator to add new programming language or paradigms for learners [1] or additional human language support [2]. However, it is likely that some of this tangled mess could be simplified with a significant effort.
In addition, there is more to Reeborg's World than a single site; there is also a basic programming tutorial available in three languages [3] with additional languages in the works, a Teacher's Guide [4], an API documentation for advanced features [5], and more [6]. Each of these act almost like an independent project pulling me in different directions.
In addition, there is more to Reeborg's World than a single site; there is also a basic programming tutorial available in three languages [3] with additional languages in the works, a Teacher's Guide [4], an API documentation for advanced features [5], and more [6]. Each of these act almost like an independent project pulling me in different directions.
In order to preserve my sanity, as my project slowly evolves I need some constancy and simplicity in the tools I use.
Using a well-supported library
Unlike the situation with Python, which comes "batteries included", there is no standard library for Javascript. Using a library means choosing between various alternatives, and communities.When I started this project, the main problem facing people writing Javascript code was browser incompatibilities. There was one obvious solution: use jQuery. Nowadays, it is most likely no longer needed for that purpose, but that was not the case back then.
I also knew that I wanted the ability to have floating windows for additional menus and dialogs. After examining a few choices, I settled on jQuery UI, since there was good documentation for it and an active community ... and I was already using jQuery which meant a smaller footprint than some other alternatives.
Libraries like jQuery and jQuery UI can be included with a link to a CDN (Content delivery network) which can reduce the load on the server where my project lives. I can also link to a specific version of these libraries, which means that I do not have to update code that depend on them (except if security issues are discovered).
10 years later, both libraries are still alive and well and I haven't needed to make any significant changes to any code that uses them.
Use npm for installing Javascript tools
npm is described as both a package manager for Javascript and as the world's largest software respository. I use it to install various Javascript tools I use (like tape, browserify, jsdoc, etc. which I describe below).I do not use it to install javascript libraries (big or small) called by my own code. From what I can tell, the "best/most common" practice in the Javascript world is to make use of tons of modules found on the npm repository, some of which are simply one line of code. Requiring a single module can mean in reality that the project can depend on dozens of other modules, none of them being vetted - unlike the Python standard library. Upgrade to a single modules can result in a bug affecting hundreds of other modules ... For example, one developer broke Node, Babel and thousands of projects in 11 lines of JavaScript.
When I resume working on my project after months of inactivity, I never have to worry about how any change to any such third party module could require updating my code. (Yes, there are most likely ways to mitigate such problems, but I prefer to avoid them in the first place.)
There are alternatives to npm (such as yarn, and others), but, from what I can tell, they do not offer any advantages when it comes to installing Javascript tools - a task that is performed very rarely for a given project.
Use npm to manage your workflow
When reading about Javascript, I most often saw either gulp or grunt mentioned mentioned as tools to automate tasks. From what I read, it seems that were essential to do any serious Javascript development. Each of them came had its own way to do things ... and it was not easy for me to see which would be the best fit. In the various posts I read about gulp vs grunt, npm was never mentioned as an alternative.However, as I learned more about npm, I found that, together with a very simple batch file it could do all the automation that I needed in a very, very simple way, by defining "scripts" in a file named package.json. Chaining tasks with npm scripts is a simple matter of "piping" them (with the | character). Since I had already installed npm, it became an easy choice.
Use browserify to concatenate all your Javascript files
Once my Javascript code became much too long to fit into a single file, I broke it up into various files. With Python, I would have use an import statement in individual files to take care of dependencies. With Javascript, the only method that I knew of at the beginning of my project (10 years ago) was to add individual links in my html file. As the number of Javascript files increased, it became difficult to ensure that files were inserted in the proper order to ensure that dependencies were taken care of ... In fact, it soon became almost impossible.This required a major rewrite. Fortunately, when I had to do this, some standardized way of ensuring dependencies had emerged. The simplest was to use something like
require("module_a.js");
at the top of, say module_b.js, and use some tools to concatenate the javascript files, ensuring that proper dependencies were taken care of. The simplest tool I found for this purpose is browserify, originally created, as far as I can tell, by James Halliday.
browserify can be installed using npm.
Use tape for unit testing
Sigh ... I find testing boring ... But, as my project grew larger, it became necessary to write some tests.
When I did a search on testing tools/framework for Javascript, I most often saw mentions of Chai, Jasmine, Mocha QUnit and Sinon. A recent search yields a few more potential candidates like Cucumber, Karma, etc.
The Javascript world seems to really, really like so-called Behaviour Driven Development, where writing tests can mean writing code like:
tea.should.have.property('flavors').with.lengthOf(3);
If.I.wanted.to.write.code.that.read.like.English.I.would.likely.use.Cobol.
It is only by accident that I came accross tape as a testing framework that felt "right" to me. I like my tests to look like my code. With Python, I would use assert statements to ensure that the a function produces the correct result. My favourite unit testing framework for Python is, not surprisingly, pytest.
From what I have seen, tape is the closest Javascript testing framework to pytest. Here's an actual example where I test some code which is expected to raise/throw an exception/error:
test('add_wall: invalid orientation', function (assert) {
assert.plan(2);
try {
RUR.add_wall("n", 1, 2, true);
} catch (e) {
assert.ok(e.reeborg_shouts, "reeborg_shouts");
assert.equal(e.name, "ReeborgError", "error name ok");
}
assert.end();
});
I make use of "assert.plan()" to ensure that the number of assertions tested matches my expectations.
It was only after I had used tape for a while that I found out that it was also written by James Halliday.
tape can be installed using npm.
Use faucet for formatting of unit test results
Tape's output is in the TAP format (Test Anything Protocol) which, by default, is extremely verbose. Most often, it is recommended to pipe the results into formatters which produce more readable results.
Depending on what I am doing, I use different formatters, some more verbose than others. After trying out about a dozen formatters, I now use faucet by default. faucet can be installed using npm and has been written by, ... you guessed it, James Halliday.
Use QUnit for integration testing
Unit tests are fine, but they miss problems arising from putting all the code together. I used different strategies to do integration testing, all of which seem to create almost more problems than they solved, until I stumbled upon a very easy way that just works for me. Using a Python script, I take the single html file for my site, put all the code inside an html div with display set to none, insert some qunit code and my own tests, and let everything run.
Optional: use Madge for identifying circular dependencies
To help identify potential problems with circular dependencies, I use madge, which can be installed with npm.
There is one remaining dependency in my code, which I silence by not inserting a require() call in one of my modules: when the site is initialized, I want to draw a default version of the world which I by calling functions in the drawing module when loading some images. Later, when calling the drawing module, I do need the definitions found in the module where I load the images. I could get rid of the dependencies at the cost of duplicating some code ... but since the initializing of the site and the execution of user-entered code are done in separate phases, the circular dependency does not cause any problems.
Optional: use Dependo to identify any overlooked module
The image of the tangled mess of modules shown above was created using dependo. As I was refactoring code and adding various require() statement, dependo was helpful in identifying any module not included, either because they had been accidently forgotten or because they had become irrelevant. dependo can also be installed using npm.
Optional: use JSDoc for creating an API
While I do not particularly like it, as I cannot figure out how to extend it to address my particular needs, I found that jsdoc useful to produce an API for people wanting to use advanced features in creating unusual programming tasks (aka "worlds"). When I started using it, there did not seem to be any easy way to use Sphinx to create such API. I gather that this might no longer be the case ... but it would likely require too much effort to make the change at this point.
jsdoc can also be installed using npm.
Use jshint instead of jslint
A linter can often be useful in identifying potential or real problems with some code. When I started working on this project, the only linter I knew was jslint. jshint is friendlier and more configurable to use, and is my preferred choice. And, you guessed it, jshint can be installed using npm.Last thoughts
There might very well be other tools that would be better for your own projects but, if you love Python and find yourself not overly enthusiastic at the thought of adopting the Javascript way when working on a project that requires Javascript, you might find that the tools I use match more closely the way you do things with Python. Or not.[1] Currently, programs can be written in Python, Javascript, using blockly, or in Python using a REPL.
[2] Language support can mean one of two things: either the programming library for users (like using "avance()" in French as equivalent to "move()" in English, or for the UI, or both. Currently, French and English are implemented for both, while Korean and Polish are only available for UI. Work is underway to provide Chinese support for both.
[3] The tutorial can be found here; you can change the default language using the side-bar on the right. The repository is at https://github.com/aroberge/reeborg-docs. The tutorial is currently available in French, English and Korean, with additional languages in the works.
[4] https://github.com/aroberge/reeborg-howto is a site aimed at creators of advanced tasks for Reeborg's World. It has very little content currently but will have more to be migrated from https://github.com/aroberge/reeborg-world-creation which was written as an online book (a format which I found to be unsatisfactory.)
[5] https://github.com/aroberge/reeborg-api is a documentation site for the API that creators of advanced tasks can use.
[6]
- https://github.com/aroberge/reeborg-staging is a staging version used for testing or for those that want absolute privacy as it is available as a secure site https://aroberge.github.io/reeborg-staging/
- https://github.com/aroberge/reeborg-dist is for those interested in installing their own version of Reeborg's World by simply unzipping a file, instead of cloning a repository with unnecessary files.
