tech debt

On Forests and Trees

When an English speaker is drowning in details that make the big picture hard to see, she might complain, “I can’t see the forest for the trees.”

image credit: Miguel Virkkunen Carvalho (Flickr)

It’s an odd expression, partly ironic and partly humorous. When I hear it, I sometimes think of my sister, who, after moving from Indiana to Utah, complained that the mountains were getting in the way of her view. (Her tongue was firmly in her cheek… :-)

The expression also describes an important problem of software engineering–one that a lot of engineers don’t understand well enough. It’s a problem with generalization.
Continue reading →

How to make a const-correct codebase in 4300 easy steps

One of the codebases that I work on is theoretically C++, but if you peer under the hood, it looks more like 1990-vintage C. It’s 500 KLOC of almost purely procedural code, with lots of structs and few true objects. More dusty and brittle than I’d like.

Image credit: Tim Gillin (Flickr)

I am not a C++ bigot; I first began to do serious, professional coding in C, not long after this codebase got its start. I understand the C-isms pretty well. And although I think Linus got carried away in his rant about the ugliness of C++, I can appreciate the ways that lean C sometimes makes its descendant look ugly and inefficient. (Though C++11 and 14 are making this less true…)

This means that I don’t consider the C-like style of this particular codebase a fatal flaw, in and of itself.

However, since my early coding adventures I’ve been converted to the advantages of OOP for complex projects and large teams, and I’ve also accumulated a lot of battlescars around multithreading. Our codebase needs OOP to solve some encapsulation antipatterns, and it needs RAII and C++11-style mutexing in the worst way. Its old, single-threaded mindset makes far too many things far too slow and error-prone.

A few months ago, we decided to make an investment to solve these problems.

To do it right, I had the team add a story to our scrum backlog about making the codebase const-correct. And therein lies a tale worth recounting…

Continue reading →

Learned Helplessness, Rats, and People Power

In the 1950s, researchers at Johns Hopkins conducted some very troubling experiments. They caught wild rats and squeezed them in their hands until they stopped struggling, teaching them that nothing they did would let them escape the crushing grip of their human captors. Then they dropped the rats in a bucket of water and watched them swim.

Now, wild rats are superb swimmers. On average, rats that had not received the squeeze treatment lasted around 60 hours in the bucket before they gave up from exhaustion and allowed themselves to drown. One unsqueezed rat swam for 81 hours.

A later rats-in-bucket experiment (not quite so brutal). Photo credit: MBK (Marjie) (Flickr).

The average squeezed rat sank after 30 minutes.

In the 1960s and 1970s, Martin Seligman became interested in this phenomenon–he called it “learned helplessness“–and he was able to trigger similar “giving up” behavior in dogs and other animals. He theorized that human depression Continue reading →

All I Really Need To Know I Didn’t Learn In Compugarten

I’m glad newly minted software engineers are exposed to data structures, compilers, concurrency, graph theory, assembly language, and the other goodies that constitute a computer science curriculum. All that stuff is important.

But it’s not enough.

Not all classroom material for CS folks should be technical. Photo credit: uniinnsbruck (Flickr).

Since I’m half way to curmudgeon-hood, I frequently find myself lamenting educational blindspots in the young. I’ve even toyed with the idea of teaching at the nearest university, some day when I Have More Time™. If academia would take me, my lesson plans might cover some of the following topics:

Why Mental Models Matter

As they leave school and embark on professional adventures, naive engineers believe their purpose is more or less summed up by this equation:

product = software = code

As they get deeper into their careers, good engineers gradually realize that the raw code baked into a product is not everything. They come to appreciate the role that support folks and tech writers, marketers and professional services play in delivering value to the customer. Eventually many arrive at :

product = (software = code) + augment

I’d put this equation into words as follows: the purpose of dev teams is to create products, which consist of software (a synonym for code) plus auxiliary offerings like support, documentation, and services.

Equations capture mental models… Image credit: xkcd

This is the level of sophistication at which much of the software industry operates. It is taught by academia (at least, if you listen to business professors), and it’s the philosophy that underpins lots of outsourcing decisions, as well as strategic mergers and acquisitions.

I think the second equation is better than the first, but it’s still woefully inadequate.

Easy Critiques

For one thing, it ignores the interrelationships among software, hardware, enabling ecosystems, and customer communities. Products don’t exist in isolation; they are part of an embedded system made possible (and relevant) by societal conventions and other technologies. “Microsoft Word” and “Adobe Photoshop” are not “products” for Kalahari bushmen.

For another, software is more than code. Notice the subtitle of my blog… Software includes people as a fundamental ingredient. In the shadows of every architecture diagram is an assumed human being (or an army of them), providing input or accepting output. How else do we think our systems will be installed, configured, optimized? How will our databases get populated, our backups get mounted, our e-books get typeset, or our web searches get chosen? (See my posts about people in architecture and role-playing in design.)

Both of those critiques are important, I think. But today I have a different bone to pick.

The Deeper Issue

Whenever we put “product” at the front of equations that describe our industry’s output, we make the implicit assumption that product is the major–or even the entire–output of tech companies. This assumption is ubiquitous and almost never articulated, let alone challenged. Ask a tech buddy about what his company does; he’ll say something like “We build products that ___.”

Of course, tech companies do build products–or solve customer problems by delivering products and services, if you want to make economists happy. But they also create another output, and I think this neglected stepchild deserves far more attention.

Besides products, tech companies produce and propagate mental models. Or in other words, they enable and shape our view of the world.

Photo credit: daveelf (Flickr)

These mental models of the world matter. They–not products–are the nuggets of gold for which we prospect. Ask Galileo.

How much of popular culture is built on scaffolding provided by an idea that used to exist only in the mind of an engineer? Engineers didn’t just dream up plasma TVs or radios; they enabled the very idea of broadcasting. They didn’t just figure out how to download files from the internet; they convinced us to think of data blobs in terms of files and folders in the first place. They didn’t just populate the App Store; they thought the concept of “app” into existence. I could go on and on with examples, but I’ll leave that as an exercise for the reader.

As I said in my post the other day about comments, the mental models created by engineers are the most valuable output of the tech industry.

MVP

Products are directly sellable, and we have to have them. But products without mental models are pretty darn useless. If you doubt me, try using a sophisticated piece of software without any idea how to think about its problem domain. If you know nothing about accounting, try to use Great Plains to be a bookkeeper. If you know nothing about graphics, try airbrushing an image in Photoshop. If you know nothing about HPC, try keeping Cray’s latest supercomputer busy doing protein folding.

Code is important, but without a mental model of how that code works, it’s not much of a foundation for a product. (This is why outsourcing that doesn’t involve bi-directional knowledge transfer is usually foolish, and why acquiring a company and RIFing all its employees nets the acquirer a lot less than they bargained for.)

Patents look nice in a war chest, but it’s sophisticated mental models, not patents, that are the prerequisite of innovation.

Implications

If you understand that tech companies produce mental models, then certain issues take on new significance.

Tech debt isn’t just insidious because it makes code ugly. A kludge lets us get by with a flawed, ill-developed mental model of a problem domain–and if we build on that model, eventually we create a house of cards. Bad mental models bite us, sooner or later.

Competition in a turbulent market is often decided by who has the better mental model. “Better” might mean the one closer to the predilections of the customer, or the one that has better long-term applicability.

Usability is all about conveying a mental model with minimum effort on the part of the receiver–and then using that model consistently and easily.

A product that doesn’t improve the mental model of the customer (e.g., by pruning unnecessary clutter, by visualizing connections that were previously impossible to see, by accounting for a neglected issue that’s been a thorn in the side) is not innovative, no matter which features it touts. It is providing little of value, and will end up on the dust heap of history.

Action Item

Take a minute to ponder how much of your passion and talent is actually centered on the “other” output from product development. What contribution have you made to a helpful mental model for a customer? Where have you invented a term that resonated, or formalized a process that used to be chaos? 

Related articles