It's Impossible to Write Correct JavaScript Programs

During a coding session involving JavaScript, UIs, and new asynchronous APIs, I realized that it's no longer possible to write correct programs with user interfaces any more. The JavaScript language has long had a problem with isolated language designers who tinker on their own isolated part of the system without seeing how things fit together as a whole. Though their individual contribution may be fine, when everything gets put together, it's a mess that just doesn't work right. Then, later generations of designers patch things over with hacks and fixes to "smooth things over" that just make the language more and more convoluted and complicated.

Right now, the language designers of JavaScript are all proud of their asynchronous JavaScript initiatives like promises, async, and whatnot. These "features" shouldn't be necessary at all. They are "fixes" to bad decisions that were made years earlier. It's clear that the designers of these asynchronous APIs mainly do server-side work instead of front-end work because asynchronous APIs make it next to impossible to write correct user interfaces.

In all modern UI frameworks, the UI is single-threaded and synchronous. This is necessary because UI code is actually the trickiest and hardest code to write correctly. People who write back-end code or middleware code or computation code actually have it easy. Their code can rely on well-defined interfaces and expected protocols to ensure the correctness of their algorithms. As long as your code calls the libraries correctly and follows proper sequences, then everything should work fine. By contrast, UI code interfaces with the user. The user is messy and unpredictable. The user will randomly click on things when they aren't supposed to. You might think, how hard can it be to write some code for clicking on a button? But what happens when they start clicking on different buttons with their mouse and finger at the same time? What happens when they start dragging the scrollbar in one direction while pressing the arrow keys in the opposite direction at the same time? Users will drag things with the mouse while spinning the mousewheel and then press ctrl-v on the keyboard and get angry when the UI code becomes confused and formats their hard drive. Then when you fix that problem, some other user will get angry because they were using that combination as a quick shortcut for formatting hard drives and want the old behavior back. Reasoning about the correctness of UI code is very hard, and the only thing that makes it tractable at all is that it's all synchronous. There is one event queue. You take an event from the queue and process it to completion. When you take the next event off the event queue, you don't know what it is, but it will be dependent on the new state of the UI, not the old one. You don't know what crazy thing the user is going to do next, but at least you know the state of the UI whenever the next event occurs.

Asynchronous JavaScript inconveniently breaks the model. All of these asynchronous APIs and promises are based on the idea that you start an action in another thread and then execute some sort of callback when the execution is complete. This is fine for non-UI code because you can use modularity to limit the scope of how crazily the state of the program will change from when you invoke the API and when the callback is called. It's even fine if these sorts of APIs are needed occasionally in UIs. During the rare time that an asynchronous XMLHttpRequest is needed, I can spend the day mapping out all the mischief that the user might do during that pause and writing code to deal with it when the request returns. But these asynchronous APIs are now becoming so widespread that I'm just not smart enough to be able to work out these details any more. The user clicks a button, you call an asynchronous API, then the user navigates to a different view, then the asynchronous call comes back to show its result, but all the UI elements are different now. The textbox where you wanted to show the result is no longer there. So now in your promises code, you have to write all sorts of checks to validate that the old UI actually still exists before displaying anything. But maybe the user clicked on the button twice, so the old UI still exists, but the 2nd asynchronous call returned before the 1st one, so now you need to write some custom sequencing code to make sure things are dispatched in the proper order. It's just a huge unruly mess.

The only practical solution I can find is to suppress the user interface during asynchronous calls, so that the user can't go crazy on the user interface while you're doing your work. This is a little dangerous because if you make a mistake, you might accidentally forget to unsuppress the user interface during some strange corner cases, but dealing with these corner cases is a lot easier than dealing with the corner case of the user generating random UI events while you're waiting on an asynchronous call. There was one proposal to add an "inert" attribute to html to disable all events, but that was eventually killed. Right now, the only hope for UI coders is to misuse the <dialog> tag, but very few browsers support it currently.

The annoying thing is that these things are just sad hacks that make programming more and more convoluted. Despite all the pride that the JavaScript designers have in their clever asynchronous promises API, that too is just a hack to paper over previous questionable decisions. The root cause of all these issues is the arbitrary decision that was made many years ago that there would be no multithreading in JavaScript. As a result, the only way to run something in parallel is to use the shared-nothing Web Worker system to run things in, essentially, separate processes. Although the language designers proudly proclaimed that there would be no concurrency errors because the system didn't allow shared objects or concurrency mechanisms, this system ended up being so limited that no one really used it. There were no concurrency errors in JavaScript programs because no one used any concurrency. (Language designers are now trying to "fix" Web Workers by creating a convoluted API that adds back in shared memory and concurrency primitives, but only for JavaScript code that is translated from C++.) Once JavaScript multithreading was killed, a certain old dinosaur of a browser company (no, not Microsoft, I meant dinosaur literally) discovered that their single-threaded browser kept hanging. Although every other browser maker moved to multi-process architectures that ensured that browser remained responsive regardless of the behavior of individual web pages, this single-threaded browser would become unresponsive if any tab made a long-running synchronous call. Somehow, the solution to this problem was to remove all synchronous APIs from JavaScript. And now we can't write correct UI code in JavaScript any more.

JavaScript is getting to be a big mess again. The fact that it's no longer possible to write correct user interface code any more is a clear signal that something has gone wrong. The big browser vendors need to call in some legendary language gurus to rethink the language and redirect it down a more sane path. Perhaps they need to call in some academics to do some original research work on possible better concurrency models. This has actually happened in the past, when Guy Steele was brought in for the original JavaScript standardization or when Douglas Crockford killed ES4. It looks like something like that is needed again.

Java Metaprogramming Is Widespread But Slowly Dying

Metaprogramming is one of those cool academic topics that people always talk about but never seem all that practical or relevant to real-life programming. Sure, the idea of being able to reprogram your programming language sounds really cool, but how often do you need to do it? Is it really that useful to be able to change the behavior of your programming language? How often does a programmer need to do something like that? Shouldn't you be able to do everything in the programming language itself? It seems a lot like programming in Haskell--technically cool, but totally impractical.

I've recently started realizing that metaprogramming features in programming languages aren't important for technical reasons. Metaprogramming is important for social reasons. Metaprogramming is useful because it can extend the life of a programming language. Even if language designers stop maintaining a programming language and stop updating it with the new features, metaprogramming can allow other programmers to evolve it instead. Basically, metaprogramming wrestles some of the control of a programming language away from its main language stewards to outside programmers.

One of best examples of this is Java. Traditionally, Java isn't really considered to have good metaprogramming facilities. It has some pretty powerful components though.

• It has a reflection API for querying objects at runtime. 
• It has a nice java.lang.reflect.Proxy class for creating new objects at runtime. 
• By abusing the classloading system, you can inspect the code of classes and create new classes. 
• The JVM instruction set is well-documented and fairly static, making it feasible for programs to generate new methods with new behavior. 

The main missing pieces are

• The instruction set is so big and complicated that it's cumbersome to analyze code or to generate new methods
• You can't really override any of the JVM's behaviors or object behaviors
• You can't really inspect or manipulate the running code of live objects

The crowning piece of the Java metaprogramming system though is annotations. To be honest, most of the real metaprogramming stuff is too complicated to figure out. Annotations, though, are simple. It's just a small bit of user-specified metadata that can be added to objects and methods. Its simplicity is what makes it so powerful. It's so simple to understand that many programmers have used annotations to trigger all sorts of new behaviors in Java. Annotations have been used and abused so much that their use is now widespread throughout the Java ecosystem. This type of metaprogramming is probably the most used metaprogramming facility in programming languages right now. 

I believe that metaprogramming through annotations has allowed Java to evolve and to add new features despite long periods of inactivity from its stewards. For example, during the 10 years between Java 5 and Java 8, there weren't any major new language features to the Java language. While Java was stagnating during that period, other languages like C# or Scala were evolving by leaps and bounds. Despite this, Java was still considered competitive with others in terms of productivity. One of the reasons for this is that Java's metaprogramming facilities allowed library developers to add new features to Java without having to wait for Java's stewards. Java gained many powerful new software engineering capabilities during those 10 years that put it on the leading edge of many new software practices at the time. Metaprogramming was used to add database integration, query support, better testing, mocking, output templates, and dependency injection, among others, to Java. Metaprogramming saved Java. It allowed Java to be used in ways that its original language designers didn't anticipate. It allowed Java to evolve and stay relevant when its language stewards didn't have the resources to push it forward.

What I find worrisome, though, is that the latest language developments in Java are weakening its metaprogramming facilities. Java 8 weakened metaprogramming by not providing any reflection capabilities for lambdas. Lambdas are completely opaque to programs. They cannot be inspected or modified at runtime. From a functional/object-oriented cleanliness perspective, this is "correct." If an object/function exports the right interface, it shouldn't matter what's inside of it. But from a metaprogramming perspective, this causes problems because any metaprogramming code will be blind to entire sections of the runtime. Java 9 will further weaken metaprogramming by imposing extra visibility restrictions on modules. Unlike previous versions of Java, these visibility restrictions cannot be overridden at runtime by code with elevated security privileges. From a cleanliness perspective, this is "correct." For modules to work and be clean, normal code should never be able to override visibility restrictions. The problem is that the lack of exceptions hampers metaprogramming. Metaprogramming code cannot inspect or alter the behavior of huge chunks of code because it is prevented from seeing what's happening in other modules. 

Although its great to see the Java language finally start improving again, the gradual loss of metaprogramming facilities might actually cause a long-term weakness in the language. As I mentioned earlier, I think the benefits of metaprogramming are social, not technical. It's a pressure valve that allows the broader programming community to add new behaviors to Java to suit their needs when the main language stewards are unable or unwilling to do so. With the language evolving relatively quickly at the moment, it's hard to see the benefits of metaprogramming. The loss of metaprogramming features will be felt in the future when outside developers can't extend the language with experimental new features and, as a result, the language fails to embrace new trends. The loss will be felt if there's ever another period of stagnation or conflict about the future direction of the language, and outside developers can't use metaprogramming to independently evolve the language. Hopefully, this gradual loss of metaprogramming support in Java is just a temporary problem and will not prove detrimental to the long-term health of the language.