What I Learned About Incompetence

I've been thinking a lot about another unacknowledged 20th-century classic, Gerald Weinberg's The Psychology of Computer Programming (1971). In my view, the book's title is misleading, snce it has little actually to say about either psychology or compuyter programming. The reviews at the link, in fact, suggest it focuses on organizational behavior.

I read it not long after it came out, around the time I refocused my career into IT. My main takeaway was his point that in a typical tech organization, one or two people have most of the necessary knowledge and skills, while everyone else does very little and refers any critical problems to the key people.

In other words, "You're getting an 808? Check with Herb, he knows what do do." But as a practical matter, whether it's an 808, a blue screen of death, a system crash, a power outage, or a user error, Herb is the guy who has all the experience and job knowledge with the system and knows how to fix things. Everyone else is basically there to refer things to Herb. This, as I think Weinberg understood, has nothing to do with either psychology or programming.

I spent some part of my career working in IT for a large utility. I got a good deal of education on these issues there. I was working on and off as a 1099 contractor, not a W-2 employee, although I was an employee there. The company was going through one of those reorgs where everyone basically has to reapply for their own jobs, which shold have in theory resulted in a reborn company where the deadwood had been trimmed away. Fat chance.

The exhilirating thing about working as a contractor is that you're coming into an organization based on an expectation that you're actually competent in a particular area, basically a temporary Herb, or maybe a little more like the Western gunslinger who's hired to do a particular dirty job and move on. I remember walking down a corridor behind two contractors who'd been with that utility for a while, and I overheard their conversation about how the reorg was proceeding.

"What I don't understand," said one to the other, "is that in case after case, they're taking the absolutely most valuable people in each area, and those are the ones they're pushing out."

A few weeks later, they called me into the little room where they told me there would not be a place for me in their dynamic new organization and escorted me out of the building. (I handed them my pager with a smile.) They referred me to the placement consultants, who were set up in a nearby hotel. When I went in to see a lady, she said, "Well, this is a surprise. I saw your resume, and I was convinced you would definitely not be one of those who'd be asked to leave." At least I'd already had an indication of how things were happening, huh?

The bottom line is that in most organization, incompetence is baked into the system. Even if the organization goes through the motions of reform, the incompetents actually form a powerful consituency that's able to perpetuate itself.

You might ask, "Wait a moment. How can a company survive if it screens out all the competent people and just keeps incompetents? Won't this catch up with them?"

That's one of the points in The Big Short. The system will hang on, seemingly forever. Incompetence is a huge constituency all its own. I'm not sure if Gerald Weinberg fully understood his own point.