Recapturing Joy

I've been on a journey of late, trying to find what originally drew me to computer programming, why I stopped enjoying it, and what I might do to recapture my lost joy. I finally figured some things out - things which may also be of interest to you, if you happen to be a computer programmer.

I didn't realize it at the time, but what originally drew me to programming was the potential to create beauty - not with music, or paint, or gears, or circuits, but with pure thought. Abstractions borne of abstractions, creating virtual edifices that somehow, magically, have impact upon the everyday world.

That's how it should have been. But it wasn't. Why?

The main problem seems to be that without the constraints imposed by the physical world, human beings are unfettered in creating pure, absolute shit. You see it all around you, every day. Take cell phones (the iPhone (mostly) notwithstanding). Sure, the hardware and the form factor of most cell phones leaves a little to be desired, but the worst of it by far is the software.

In the physical world, you can only make something so horrible before it is no longer a something, but simply a pile of parts, or trash. Buildings, cars, CD players... sure, they might be ugly, and they might not be as fluidly functional as they could be, but - at a minimum - they either fulfill their function or they do not. Not so with software. I once owned a cell phone which was controlled by such horrible software that, were it a building, it would have collapsed in a heap of rubble. Just to give you a taste, about once a day I had to remove the battery to reboot the phone. Reboot the phone? The thing was a fucking state machine with at most five states! It's as if the builders were trying to make the thing bad.

But of course they weren't trying to make a bad phone. They were simply trying to make a phone, with the tools they had at their disposal. And the tools they had at their disposal were probably pure shit. But why? Why is so much software pure shit? I don't mean in the "90%-of-everything-is-crap" sort of way; software seems to be much, much worse than that. My dad doesn't get angry with his car; he doesn't even get angry with his cell phone. But he gets angry with his computer, all the time. And that's just using Microsoft Word.

I believe it's because that we computer programmers too easily confuse complexity with elegance. And so the complexity grows, because... well, because it's just so much fun. And because there is no force of gravity pulling it all down to earth - we can simply grow and grow and grow the complexity, forever. Arbitrary complexity. The physical universe puts an upper bound on the amount of arbitrary complexity that will be tolerated in a building or a car. But in software there exists no negative feedback loop for complexity.

People who don't program computers are always shocked to discover that very, very little programming time is spent thinking about the real, actual problems to be solved. Rather, most of our time is spent learning about the arbitrary complexity created by other human beings. If I'm building you a web site to keep track of how many miles you jogged last week, I assure you that I'm spending precious little time thinking about jogging or tracking miles or usability or whether or not you'd like to restrict access to your logs or any other issues that are of real, actual interest to you, the user. What I'm spending most of my time doing is figuring out how why my Javascript function is working on Firefox and not IE. Or tracking down the memory leak in my app server. Or any of one billion other stupid little problems that have nothing to do with what you want, and - more importantly - from which I'll learn absolutely nothing (except some small fragment of arcane product knowledge, guaranteed to be forgotten or obsolete by next week).

The last point is important - with the tools at our disposal, experience doesn't make us computer programmers smarter; experience actually makes us dumber, because we're not learning anything in any meaningful sense of the word "learn", unless you consider the memorization of product manuals to be "learning".

In our defense, as a profession, we do seem to be gravitating towards development ecosystems that are more and more dynamic and expressive and powerful. But they're still doomed, because they can't meaningfully evolve. Witness what's happening in the Ruby community. Ruby and its community is essentially Java++ - the language is better, the development ecosystem is better, it's run by a benevolent dictator instead of a corporation... and yet it's still doomed to go the way of Java. I'll bet my kidneys on this one - In 2015, we'll all be talking about Ruby in exactly the same way we talk about Java in 2008. I know this to be true because it's the same players on the same trajectory. Ruby now is Java in 1999 - we're all excited about building tools and frameworks and servers, and it all looks and sounds and feels great. It always does when you first start building those first few layers of arbitrary complexity.

But isn't that the way of things? This is all unavoidable, isn't it? Perhaps. But I believe Lisp is the exception. And interestingly, it has nothing to do with the community (in fact, the Common Lisp community is an active inhibitor on the growth and success of Common Lisp). It's due to a geeky little aspect of Lisp, which is that there is no Lisp. It's a language-less language - Lisp is essentially without syntax, expressing itself as its own abstract syntax tree. And when you provide the magic spells "eval" and "apply", and thus enable a macro facility that lets you manipulate the language using the language itself, there simply isn't anything you can't do.

But didn't I say that this unfettered freedom was the problem? Well, yes and no. Empirically, the unfettered freedom provided by other languages leads to abstractions being expressed in terms of frameworks and libraries - edifices of arbitrary complexity. But Lisp enables one to express abstractions in terms of language, in a way that no other language really does, or even can - as Paul Graham has pointed out, any language which provided the ability to express itself in terms of its own abstract syntax tree would essentially be a dialect of Lisp. In a world where we can express ideas with language, we're no longer talking about "computer programming"; we're talking about writing, pure and simple - a difficult activity to be sure, but one with which humanity has had more success.

And so Lisp has helped me to recapture some of the joy of computer programming, by enabling me to once again be excited about working with ideas and abstractions, which in turn is enabled by the ability to create language - and with a language that almost isn't even there...