Software Boy

To: rememberingsteve@apple.com

2011-10-06T01:43:00.000-07:00

My first computer was an Apple II Plus that my dad brought home as a Christmas present when I was in 3rd grade. Although not as immediately user-friendly as computers would become, the machine that Woz built just plain worked. Always. And it defined for me a sense of taste for what technical and design excellence really meant.

So for me, Apple was the only computer that mattered, until about 1995, long after Woz and Steve Jobs had gone, and Apple products had become just another commodity item. I bought a $4000 Mac that was so unstable (System 7) that I swore off Apple products for life. By this time I was being paid to program computers, and I figured that if I was going to be miserable, I might as well be miserable on the same awful machines that everyone else used. Computers were no longer sources of imagination and joy, but just another lifeless tool we could use to turn $1 into $1.10.

I was happy to see Steve Jobs return, but I was still skeptical, even when the first iMacs and iPods made me believe that perhaps there was once again a group of people in this world that cared about making something useful and beautiful. It wasn't until Steve Jobs presented the first iPod Nano, when I had the same emotional reaction I had when I was 8, when I first saw an Apple II: "I have to have that, and I have to have it now."

Not long thereafter I bought a 17" Macbook Pro, having heard good things about Mac OS X, and having been frustrated to the point of tears with a Dell laptop that would periodically decide to ignore me as it ran its anti-virus software. I'm typing this on that same Macbook Pro, which is running just fine 4 years later.

I never met Steve Jobs. I don't even know any of the engineers at Apple. But it has mattered to me in my life knowing that somewhere in this world there is a group of people that cares about making useful and beautiful things. That technology can improve the human condition. That people can strive for excellence, and achieve it.

Sincerely,
Kurt Christensen

Learning Things of Lasting Value

2009-10-03T12:00:00.000-07:00

In the book "Seeking Wisdom", Peter Bevelin describes the ways in which we make bad decisions, why we make bad decisions, and what we can do to avoid making bad decisions. The author synthesizes lessons from a number of sources, focusing especially on Warren Buffett and Charlie Munger, the leaders of Berkshire Hathaway (a successful holding corporation of which you may have heard).

In listing the methods for improving the quality of one's thought process, Mr. Bevelin elaborates on the method of "Simplification" under the sub-heading "Focus on what you can know and that makes a difference":

"What's knowable and important? And what can be translated into useful action?"

At the ripe old age of 38, working within a technology industry that changes daily, this question is gaining in importance for me. I have lost the energy, and patience, and interest required to stay abreast of technological flavors-of-the-month. Ruby on Rails? Honestly, I only barely give a shit. Java 7? Flex? Silverlight? These things are only valuable until the powers behind them decide I should learn something newer and better, and that's only six months away.

So, to remain effective over the long haul, I am required to focus on a few things that I believe can have lasting value. And here's my list:

The fundamentals of computer science, as laid out in the book "Structure and Interpretation of Computer Programs", by Abelson and Sussman
- Why? Because everything you'll ever really need to know about computing is contained in this one book. Seriously.

Fundamental algorithms and data abstractions
- Why? Because our job as computer programmers is to think, and algorithms and data structures (along with metaphor) are the raw materials. You can't be an effective civil engineer if you don't understand the properties of concrete, and you can't be a computer programmer if you don't understand the ways in which you can structure and process data, and the processes themselves (which are really just another form of data).

The Unix operating system - how it works, and why
- Why? Because flavors of Unix - Linux in particular - have won. I have mixed feelings about this, in that it hardly seems the best we can do; however, Unix appears to be the once and future king. This may seem a ridiculous statement, given the continuing market share of Windows, but it's true. I have no special hatred for Microsoft, but they're locked on a trajectory that ends badly, much as GM was in, say, 1995.

Proficiency in at least one flavor of the Unix shell - the Bash shell, probably
- Why? Despite the ancient clunkiness of the Unix command line as a means for interacting with a computer, there is an important way in which it was - and is - decades ahead of its time: Unix gave us the notion of assembling small powerful tools into larger, more powerful tools. Other areas of computing have only barely begun to explore this design space (e.g., web mashups), but it will become more important as computing becomes more heterogeneous and complex. As a practical matter, you can get a lot done with some basic shell scripting skills, and if you know Bash, you can probably apply that knowledge to other, less powerful shells (e.g., DOS).

Proficiency with a universal text editor (e.g., vi or emacs)
- Why? Because humans interact with computers via text, and will be after I'm dead. I never learned to type properly as a teenager, because I was assured that keyboards were going to be obsolete in 20 years. Well, I'm no longer waiting for that one to become true (a note to naysayers: explain how a voice interface will work in an office environment). Consequently, the need for typing skills and proficiency with a text editor aren't going to disappear anytime soon, and for longevity's sake we should probably choose the lowest common denominator in text editing that still affords the power we need, and that's probably vi or emacs. I wish that weren't true. But it is. You can learn something else (I myself prefer IntelliJ), as long as you accept that you'll be throwing away that knowledge and learning a new tool at some point in the next few years.

Proficiency with C
- Why? Because C remains the lingua franca for communicating with the underlying hardware, and if you can't speak this language yourself, you'll always be intimidated by and limited by those who can.

Proficiency with Lisp (or some dialect, such as Scheme or Arc)
- Why? Because it's a medium for expressing thought, similar in linguistic power to mathematics. There are many other computer languages, of course, but no other language combines the ability to express functional abstractions, data abstractions, and metalinguistic abstraction. For the time-being, Ruby or Python may be an "acceptable Lisp". But beyond that, I have no burning desire to learn Ruby or Python, because they have nothing to teach me which I can't learn from Lisp.

Proficiency with SQL
- Why? Because relational databases probably aren't disappearing anytime soon, and so you'll need to write a query occasionally if you expect to get anything done. More importantly, however, SQL offers a different way of thinking about computing - in terms of sets - and thus has something to teach.

Agile values, principles and practices (at least until something better comes along)
- The word "agile" is dead. But many of the underlying values will remain, and rightly so. Chief is the idea of "failing fast" - iterating to better understand the product one is building, and to get better at building it.

In compiling this list, I realized that my age doesn't really matter - that in any endeavor, we should focus on the underlying values and principles before we get distracted with the latest fashions, because only then can we understand and intelligently evaluate the latest fashions.

So, that's my list, until someone gives me a better one. What's on your list?

How to Debug

2009-01-22T12:38:00.000-08:00

I once worked with a wonderfully talented programmer named Cuong Tran, who has an almost uncanny ability to determine the cause of bugs, and then fix those bugs. He once explained to me how to debug code: "When you're trying to solve a problem, strip away everything and make it absolutely as simple as possible, until you have something that works. Then add stuff back, bit by bit, until you see the problem." Well, obviously. And yet I keep having to learn this lesson over and over again. Most recently, the teacher was another wonderfully talented programmer (Brian Ericson), and the classroom was Redmine.

Redmine is a lightweight collaboration tool based on Rails. It includes a wiki, forums, bug tracking and more, and yet still manages to feel small and simple and easy to configure and use. At least, it felt that way, right up until I tried to get LDAP authentication happening. Following the instructions on the Redmine wiki, I was having no success, and the log file was giving precious little information, even with debug output. Why wasn't the damn thing working??

Fortunately, Redmine has two helpful personality traits: (1) It comes with its own source code, and (2) the source code is Ruby, which means we could very easily add our own debug output - or even modify behavior - to learn what was happening. And this enabled us (and by "us" I mean "Brian Ericson") to strip away everything until we had something that worked.

After some poking around, it became apparent that the trouble was in auth_source_ldap.rb. Inside this class was simply the Net::LDAP class - shrink-wrapped Ruby. So just forget about Redmine for a minute - could we simply call the search method on Net::LDAP, and get something in response? We (and by "we" I mean "I") had almost given up when we (and by "we" I mean "Brian") discovered that yes, we could - but not with the sort of configuration information that I had provided.

The Redmine wiki implied that one simply needed to provide a username and password to connect to the LDAP server (assuming the LDAP server doesn't allow anonymous access, which mine does not). However, anyone familiar with LDAP - a club which did not include me, until yesterday - knows that you need to provide some context. Specifically, you need to chant the proper LDAP incantations to tell the LDAP server where in the directory hierarchy it should search for that user. Thus, instead of this:

Name	Our LDAP
Host	ldap.our-company.com
Port	389
Account	kurtc
Password	********
Base DN	ou=People, ou=Root, dc=our-compnay, dc=com

...Redmine requires this:

Name	Our LDAP
Host	ldap.our-company.com
Port	389
Account	uid=kurtc, ou=People, ou=Root, dc=our-company, dc=com
Password	********
Base DN	ou=People, ou=Root, dc=our-compnay, dc=com

It all makes perfect sense, when you ask the right questions in the right order: In Ruby, what's the standard way of searching for user information in an LDAP server? How is Redmine using this code? And are we passing in the expected arguments?

Once we simplified the problem enough to ask the right questions, the problem was obvious: we (and by "we" I mean "I") wasn't passing in the right "Account" string, which needed to be the proper LDAP incantation, as opposed to a simple username.

So there it is - how to debug. Just strip away everything and make it absolutely as simple as possible, until you have something that works. Then add stuff back, bit by bit, until you see the problem. The details are left as an exercise for you, dear reader.

Attack of the Architects

2009-01-05T08:50:00.000-08:00

"Software architects" are often willing to destroy value in the name of generalization. As an example, I'm working with a group using VersionOne as a collaboration tool. VersionOne has a somewhat generic tagging mechanism called "Feature Group". For the most part, we use the "Feature Group" mechanism to track the "Capability Package" (i.e., a related set of high-level requirements from the product managers) from which a story was derived. But there are a (very) few teams that use the Feature Group mechanism in a different way. And so the software architects won't let us rename the field, because they prefer the general name, because they prefer generality over specificity.

The problem, of course, is that the term "Feature Group" means exactly nothing to anyone in the business. But at some undetermined point in the future, someone somewhere may wish to use "Feature Groups" for something else, at which time we'll be heroes! We kept it general, and abstract! And totally confusing for the business people in the organization who are actually trying to get something done! Yay! Hooray for architects!

Recapturing Joy

2008-12-15T22:50:00.000-08:00

I've been on a journey of late, trying to find what originally drew me to computer programming, why I stopped enjoying it, and what I might do to recapture my lost joy. I finally figured some things out - things which may also be of interest to you, if you happen to be a computer programmer.

I didn't realize it at the time, but what originally drew me to programming was the potential to create beauty - not with music, or paint, or gears, or circuits, but with pure thought. Abstractions borne of abstractions, creating virtual edifices that somehow, magically, have impact upon the everyday world.

That's how it should have been. But it wasn't. Why?

The main problem seems to be that without the constraints imposed by the physical world, human beings are unfettered in creating pure, absolute shit. You see it all around you, every day. Take cell phones (the iPhone (mostly) notwithstanding). Sure, the hardware and the form factor of most cell phones leaves a little to be desired, but the worst of it by far is the software.

In the physical world, you can only make something so horrible before it is no longer a something, but simply a pile of parts, or trash. Buildings, cars, CD players... sure, they might be ugly, and they might not be as fluidly functional as they could be, but - at a minimum - they either fulfill their function or they do not. Not so with software. I once owned a cell phone which was controlled by such horrible software that, were it a building, it would have collapsed in a heap of rubble. Just to give you a taste, about once a day I had to remove the battery to reboot the phone. Reboot the phone? The thing was a fucking state machine with at most five states! It's as if the builders were trying to make the thing bad.

But of course they weren't trying to make a bad phone. They were simply trying to make a phone, with the tools they had at their disposal. And the tools they had at their disposal were probably pure shit. But why? Why is so much software pure shit? I don't mean in the "90%-of-everything-is-crap" sort of way; software seems to be much, much worse than that. My dad doesn't get angry with his car; he doesn't even get angry with his cell phone. But he gets angry with his computer, all the time. And that's just using Microsoft Word.

I believe it's because that we computer programmers too easily confuse complexity with elegance. And so the complexity grows, because... well, because it's just so much fun. And because there is no force of gravity pulling it all down to earth - we can simply grow and grow and grow the complexity, forever. Arbitrary complexity. The physical universe puts an upper bound on the amount of arbitrary complexity that will be tolerated in a building or a car. But in software there exists no negative feedback loop for complexity.

People who don't program computers are always shocked to discover that very, very little programming time is spent thinking about the real, actual problems to be solved. Rather, most of our time is spent learning about the arbitrary complexity created by other human beings. If I'm building you a web site to keep track of how many miles you jogged last week, I assure you that I'm spending precious little time thinking about jogging or tracking miles or usability or whether or not you'd like to restrict access to your logs or any other issues that are of real, actual interest to you, the user. What I'm spending most of my time doing is figuring out how why my Javascript function is working on Firefox and not IE. Or tracking down the memory leak in my app server. Or any of one billion other stupid little problems that have nothing to do with what you want, and - more importantly - from which I'll learn absolutely nothing (except some small fragment of arcane product knowledge, guaranteed to be forgotten or obsolete by next week).

The last point is important - with the tools at our disposal, experience doesn't make us computer programmers smarter; experience actually makes us dumber, because we're not learning anything in any meaningful sense of the word "learn", unless you consider the memorization of product manuals to be "learning".

In our defense, as a profession, we do seem to be gravitating towards development ecosystems that are more and more dynamic and expressive and powerful. But they're still doomed, because they can't meaningfully evolve. Witness what's happening in the Ruby community. Ruby and its community is essentially Java++ - the language is better, the development ecosystem is better, it's run by a benevolent dictator instead of a corporation... and yet it's still doomed to go the way of Java. I'll bet my kidneys on this one - In 2015, we'll all be talking about Ruby in exactly the same way we talk about Java in 2008. I know this to be true because it's the same players on the same trajectory. Ruby now is Java in 1999 - we're all excited about building tools and frameworks and servers, and it all looks and sounds and feels great. It always does when you first start building those first few layers of arbitrary complexity.

But isn't that the way of things? This is all unavoidable, isn't it? Perhaps. But I believe Lisp is the exception. And interestingly, it has nothing to do with the community (in fact, the Common Lisp community is an active inhibitor on the growth and success of Common Lisp). It's due to a geeky little aspect of Lisp, which is that there is no Lisp. It's a language-less language - Lisp is essentially without syntax, expressing itself as its own abstract syntax tree. And when you provide the magic spells "eval" and "apply", and thus enable a macro facility that lets you manipulate the language using the language itself, there simply isn't anything you can't do.

But didn't I say that this unfettered freedom was the problem? Well, yes and no. Empirically, the unfettered freedom provided by other languages leads to abstractions being expressed in terms of frameworks and libraries - edifices of arbitrary complexity. But Lisp enables one to express abstractions in terms of language, in a way that no other language really does, or even can - as Paul Graham has pointed out, any language which provided the ability to express itself in terms of its own abstract syntax tree would essentially be a dialect of Lisp. In a world where we can express ideas with language, we're no longer talking about "computer programming"; we're talking about writing, pure and simple - a difficult activity to be sure, but one with which humanity has had more success.

And so Lisp has helped me to recapture some of the joy of computer programming, by enabling me to once again be excited about working with ideas and abstractions, which in turn is enabled by the ability to create language - and with a language that almost isn't even there...

Hope

2008-11-05T00:11:00.000-08:00

"I am asking you to believe, not just in my ability to bring about a real change in Washington, I'm asking you to believe in yours."

- President-Elect Barack Hussein Obama

Agile Languages

2008-10-10T17:00:00.000-07:00

The other day I was having lunch with a friend - a good programmer - and I asked him what he was working on. He said that was learning about Ruby, and Rails, and Rake. He talked about how he thought it would be nice to use Rake at this one client to clean up their messy build. I asked him how Rake differed from Ant, and he replied "Well, I haven't actually used Rake yet, but it looks really cool." And suddenly I felt uncomfortable.

Once upon a time, Java was new and fresh and exciting. To those of us coming from a C++ background - and who had never heard of Smalltalk - Java seemed to us like C++ done better: familiar syntax, but with garbage collection and a beautiful set of standard libraries, all packaged on top of a platform-independent runtime. Java and its associated collection of frameworks have matured to the point where you can crank out a web-based database front-end and have a pretty good idea of how long it will take, and where you'll run into problems. But because we're geeks, we prefer to tinker with new things, and a corollary of this is that we don't like to tinker with old things that we know very well, even if they're working OK. Enter Ruby.

There's a lot of momentum right now behind Ruby, especially Rails, and some of this momentum is coming from within the Agile community. To a certain extent, this makes sense - agile teams strive to deliver high-quality software in an iterative fashion with just-in-time requirements. The tools we use can help us or hinder us in this effort. And of all our tools, the most important is our programming language. It is through language that we are able to express processes and abstractions in such a way that we can automate and manipulate and understand them.

Some say that the choice of language is irrelevant, that all languages are effectively the same. But anyone who says this is either unaware of the differences between languages, or - more likely - is simply being polite, avoiding an argument. Would anyone seriously argue that assembly language is as expressive as Java? But what about the difference between Java and C#? Here the answer - if there is one - becomes more complicated, and so most of us just punt. However, if we're all excited about a new language (such as Ruby), and if we're going to be responsible with our employers' money, then we need to address two issues:

Are we switching to Ruby because it's really that much more productive, or are we switching to Ruby because we're geeks and we're bored and we've done it in Java and now we're just dying to try something new...?
If there are valid reasons to toss Java and try something truly new, should that something new be Ruby? Are we all excited about Ruby because it's really the right way to go, or is it because we're being told us to be excited about it?

When a community springs up around a new technology and reaches a critical mass, a training and consulting community springs up around it as well, because everyone smells the money. And therein lies the danger that we might make decisions for the wrong reasons: because there are now a lot of people telling us how great Ruby is who have a financial stake in whether or not we like Ruby.

I understand that Rails is a wonderful framework with which to write a web app. I also understand that there are similarly wonderful frameworks written in other languages - Django, web.py and others for Python, Seaside for Smalltalk, and so on. So why are we excited about Ruby and not about, say, Python?

I've never been paid to write an application in Ruby, or Python, or any of these other fancy dynamic languages, so I'm ill-qualified to pass judgment on any. But as an outsider, what concerns me is this: in the Java community, we didn't break a bunch of new ground. To be brutally honest, we - or, at least, I - ain't the best and the brightest. If we were, we'd be doing something more interesting than using Blub to write the nine-billionth web-based database front-end for a bank.

But here's a question: if Ruby and Rails are so great, then why is it that Google and YouTube and many of the more interesting startups gravitating instead towards Python? If I were an IT manager, this would make me curious.

There are a lot of interesting languages with active communities happening at the moment. Ruby, Python, Haskell, Erlang and Common Lisp seem to be the standouts. The hardware people tell us that foreseeable future performance gains are coming through multi-core parallelism; that instruction-level parallelism has gone about as far as it can go.

This is a very, very important fact for software developers, because it means that we will no longer be getting the speed-ups for free from the compiler and the hardware - it means that we will have to program parallelism. History shows that the software community in general isn't very good at this. In fact, we suck. Erlang, however, provides a beautiful model for executing several concurrent processes and keeping them coordinated. So if we're looking towards the future, shouldn't we be more interested in Erlang, or something like it?

Most of us are corporate IT programmers, which means that every day we spend a whole lot of someone else's money. Don't we owe it to our customers to pick our technology based on something more than a fad?

Proactive + Reactive Processes

2008-09-11T11:08:00.000-07:00

At present I'm playing the part of a shmagile coach for a medium-sized software company that's rolling out scrum processes across their entire development organization for their next big product release. We've got 38 scrum teams, so it's a non-trivial effort. One of the fundamental challenges is this: once you go away from having an up-front functional spec, how do you enable the various groups to stay abreast of what other groups are doing, without having so much communication that the signal-to-noise ratio goes to zero?

I'm finding that a good way to attack these sorts of problems is to define a proactive process, and a reactive process. The proactive process enables teams to identify obvious dependencies and shared issues up front. It's an 80% approach - you know you're not going to find everything, but you can at least get working without being willfully ignorant.

That's good, but in an agile environment you also need a reactive process - a chance for people to identify issues that were missed during planning, and to identify these issues as soon as possible - preferably as they're happening.

To illustarte this more concretely, before we began the current release cycle, the product management organization had a release roadmapping session in which they roughly mapped out which "epics" (big huge stories) were slotted into one of five "release candidates". The architecture group and some of the development team leads then spent a little time identifying technical dependencies between those epics. Again, the goal wasn't to find every dependency or hidden technical issue, but to proactively pick the low-hanging fruit.

Meanwhile, we have two different weekly "scrum of scrum" meetings attended by a representative from each of the teams. One scrum is for product managers, while the other scrum is for the dev tech leads. These forums exist primarily to enable others to learn about - and react to - changes from other teams as quickly as possible.

How well will does the proactive + reactive approach work in practice? I don't know yet - we just started our very first sprint. Stay tuned...

Meeting Sleepytime Indicator #2

2008-08-25T10:27:00.000-07:00

Meeting Sleepytime Indicator #2: When you've been in a meeting for over an hour, and someone asks a question which would actually provoke a meaningful discussion, and the presenter shuts it down by saying "I just want to get through the rest of these slides...", it's time to take a nap.

Meeting Sleepytime Indicator #1

2008-08-18T09:19:00.000-07:00

Over the course of the next few... years...(?), I'll be posting various Meeting Sleepytime Indicators (MSIs) as they are identified. When you're attending a meeting and an MSI occurs, it lets you know that it is now perfectly acceptable to stop paying attention and fall asleep, because all valuable conversation has ended. In fact, this may be the best way to engage in the conversation, because if you begin snoring it might prompt the other attendees to change the subject. So let's kick off the list!

Meeting Sleepytime Indicator #1: When two or more attendees begin debating the differences between "processes" and "procedures".

Professional vs. Profitable

2008-08-15T11:42:00.000-07:00

I recently was tasked with producing some documentation for a client. Most of my writing is conversational in tone, with the occasional tongue-in-cheek comment. The client liked the content, but asked that I change the language to make it more "professional". Which got me thinking about the word "professional". What exactly does it mean to be professional?

Entry number one from dictionary.com gives: "following an occupation as a means of livelihood or for gain". Entry number three, however, better captures the shade of meaning for "professional" as it's normally used in the workplace: "appropriate to a profession". The idea is that within a given profession there are certain things that one does to conform to the standards of that profession. But why? What are those standards, and what are they for?

Returning to definition number one, most of us practice a profession within the context of a business, and any business exists for one reason: to make money. Period. No matter what reasons a business gives by way of vague mission statements, a business exists to make a profit. Unfortunately, making a profit is hard work. More unfortunately, within the context of a large corporation - and particularly within the context of information-based corporations which employ a large number of knowledge workers - it becomes increasingly difficult to tie any one person's activities to the profitability of the company.

Enter "professionalism". Since we can't tell whether or not any one person's actions are profitable, we invent a set of customs collectively called "professionalism". Like any social ritual, professionalism involves wearing the correct costumes, saying the correct things, and following the correct protocols. These things are measurable - even if only qualitatively. Measuring professionalism thus becomes a proxy for measuring profitability.

The problem is that as an organization grows, so grows the disconnect between professionalism and profitability. So much so, in fact, that what most people call "professional" behavior tends to actually be mocked by the rank and file workers of an organization, who - on the whole - provide the lion's share of direct value to the company. It makes a certain amount of sense; the rituals of professionalism require effort, and so being "professional" necessarily leaves less time for being profitable.

What's especially terrible about this is that it's a positive feedback loop - the more authority one has within a hierarchy, the more disconnected one becomes from the activities associated with profitability, and so the more one must rely on judgments based on professionalism. Thus, senior management becomes even more disconnected as they become increasingly surrounded by like-minded experts in professionalism but not profitability.

It's not obvious what - if anything - can be done about this, except perhaps to grow an organizational culture that explicitly disdains the traditional notion of "professionalism" (or at least certain aspects of it). The next time you hear someone use the term "professional" or "professionalism", see if you can figure out why they chose to use that particular word.

More Manager Metrics

2008-08-14T15:08:00.000-07:00

Here's another manager metric: does your manager often justify decisions or processes or technology choices as "industry standard"? Your company is in an industry, but the industry is not your company. Who cares what everyone else is doing? If all the other kids are doing something, it makes it worth investigating, but it certainly isn't a rationale for doing it - particularly within the software industry. The software industry sucks so badly that "industry standard" can often be used fairly reliably as an indicator of what not to do.

Attention to Detail

2008-08-13T22:55:00.001-07:00

On and off for the last few months, I've been working on a plugin for IntelliJ (my favorite IDE and text editor) to support development in the Arc programming language, which is Paul Graham's (relatively) new dialect of Lisp. Writing code in Arc is sort of fun, because although it's young and immature, it's very small and clean. However, it's also fun to develop plugins for IntelliJ, because their plugin API - including their API for developing custom language plugins - is just fabulous. Consequently, I've spent very little time writing Arc code, and quite a lot of time working on my Arc plugin. To a hammer maker, everthing looks like a hammer, I guess.

Anyway, everything was fine until the switch from IntelliJ version 7.0 to version 8.0, in which they've introduced a fair amount of changes to the language plugin API, in order to better support languages other than Java. This is a great thing for IntelliJ, as it is now poised to become the premier IDE for Ruby, and Python, and a host of other languages in addition to Java. It's not such a great thing for me, however, in that I've had to make a variety of changes to the Arc plugin in order to get it creaking into action.

After a few late nights, I finally got most of the Arc plugin working in version 8, with one maddening exception: syntax highlighting. Now, syntax highlighting is one of the easiest things to make happen when writing an IntelliJ language plugin. You practically get it for free, because once you've written the lexer (which is your very first step, and which they also make very easy via JFlex), all you need to do is create a class which extends SyntaxHighlighterBase, specify the default colors and fonts you want for various tokens, and snap - you got yourself some syntax highlighting.

In version 8, (presumably) to better follow some sort of internal dependency injection blah blah blah, the JetBrains folks have changed the way you get a lot of these components created - instead of defining getters in your "MyLanguage extends Language" class, you add "extension" entries to your "plugin.xml" plugin configuration file. This is fine, except that the JetBrains folks haven't gotten around to updating the plugin documentation yet. At all.

But you're still not doomed, because they have been updating their own language plugins for version 8, and for their Javascript plugin, they've included all the source code! So all I should need to do is simply copy the right entry from the Javascript plugin.xml file, and I'm good to go, right?

<syntaxHighlighter language="Arc" implementation="com.bitbakery.plugin.arc.ArcSyntaxHighlighterProvider"/>

Looks right. Fire up the plugin, and I get... no syntax highlighting. So, I beg for help on the "Open API and Plugin Development" user forum over at intellij.net, and Maxim Shafirov (IntelliJ team lead!) is nice enough to respond that the correct XML attribute is "key" and not "language". So I try that, and still no luck. So I try to create the extension programmatically within the constructor of my ArcLangauge class. Still no luck. Finally, after a few hours of frustration, spread across two evenings, I decide to really look at the plugin.xml for the Javascript plugin:

<syntaxHighlighter key="JavaScript" factoryClass="com.intellij.lang.javascript.highlighting.JSSyntaxHighlighterProvider"/>

Oops. Turns out Maxim was very nicely telling me to quit blaming the plugin API changes, and pay attention to detail, because I wasn't.

When I was a freshman in college taking my first physics class, I would often find myself coming up with homework answers that didn't match those in the back of the textbook. I would bring these issues up to my professor during her office hours, and she would very patiently work through the problem with me and show me what I was doing wrong. And not once did I ever actually discover a wrong answer in the back of that textbook, although I thought I did, many, many times.

Occam's Razor is a principle which asserts that - all other things being equal - the simplest explanation is usually the correct explanation. And when you're programming computers, the simplest explanation is that it's not the other guy, it's you. So pay attention, because there are a lot more little things - and a lot more stupidity - than you might think.

It's a valuable lesson. I just wish I didn't have to keep re-learning it.

Meeting Metrics

2008-08-13T05:56:00.000-07:00

Random thoughts while attending a stupider-than-usual meeting:

Here's a barometer for managers - do they use harsh language for their own mistakes, and gentle language for others' mistakes, or vice-versa?

...and here's a meeting metric - every time someone says or does something that makes you want to crush your own soul, put a hash mark down on your meeting notes. You can then measure the pain of a meeting quantitatively in units of soul crushings per hour (SC/hr). Over time, you can then look for corollations between SC/hr and meeting attendees, topics or facilitators.

Hello, world!

2008-08-12T12:41:00.000-07:00

Jason Bock keeps bothering me to start making my stupid ideas more public, presumably as part of an elaborate scheme to get me to say enough stupid things in public that I eventually find it impossible to find consulting work. Yes, if there's one thing Jason is about, it's finding a way to destroy me and my family. I'm not sure why.

But since I find Jason strangely persuasive, here it is - my blog, relentlessly driving me to a world of humiliation and starvation, seemingly avoidable, and yet completely unavoidable. It's like watching a Greek tragedy.

To kick this off, I'll answer the same set of questions that Jason himself answered...

How old were you when you started programming?

I took a computer programming summer school class in 1981, at the age of 10. I really, really hated it. I dropped out, and never looked back. Then in college I decided I wanted to be a physicist. I sucked at that, but I did enjoy wiring up cards to control experiments. So then I became enamored with electrical engineering. I sucked at that as well, but I did enjoy writing the device drivers for the cards we were making. So then I tried computer science, which I really enjoyed. I sucked at that too, but not as much. And people were (and still are) willing to pay money for sucky programmers, so I rolled with it.

What was your first programming language?

Technically, Apple Basic, but really I'd have to say Fortran 77, in 1990.

What was the first real program you wrote?

As Jason said, "depends on what you mean by 'real'". I guess I'd say some educational video games I did at JVC Digital Arts Studio in 1997. Everything before that was almost too simple.

If you knew then what you know now would you have started programming?

Yes, but instead of doing the C --> C++ --> Java/.NET route, I would've become a Unix-y/Lisp-y snob, and only taken certain kinds of gigs. I'd be smarter and happier at work had I done this.

If there is one thing you learned along the way that you would tell new developers, what would it be?

I'd echo Jason's sentiment: "learn core concepts". I've learned and thrown away about three technology stacks in my career so far, but if I knew a lot of algorithms and data structures and the Bash shell and Sed and Lisp and maybe C really, really well, I'd have something to hang my hat on. Everything else takes too much work and doesn't last long enough to add lasting value to your brain.

What’s the most fun you’ve ever had … programming?

The project I was on with Jason Bock was actually a ton of fun for a few months. I also had an insanely good experience at JVC Digital Arts Studio, and at a place called Gearworks. It's cool to get gigs where everyone is way smarter than you.