Software Architecture

On pains and brains

A while back, I wrote a post on why software should feel pain. Since then, I’ve had that lesson reinforced in my mind, and I’ve also understood some nuances that weren’t obvious to me before, so I’m revisiting the topic.

The Reinforcer

What brought this topic back to my mind was a root cause analysis I did to diagnose a recent system failure. I’ll spare you the gory details, but here’s what happened in a nutshell: a daemon got bad data files and began behaving strangely as a result. The replication process for its data files had been impaired because the app producing the data files finished much later than normal. That app in turn was impacted by anomalous network brownouts which began with a partly damaged network cable.

The Obvious but Naive Lesson

The final step in my root cause analysis was to make recommendations, and I was quick to offer some: the daemon should double-check the integrity of its data file; the originating app should monitor its timing and complain about anomalies.

The more I thought about it, however, the more unhappy I became. Surely, such monitoring is a good idea. So why did I not believe my recommendations would really make things better?

Descartes-reflex

The pain pathway is more than nerves in the toes; it runs all the way back to the brain. From René Descartes’s Treatise of Man. (Wikimedia Commons)

Continue reading →

Taming Side Agreements

When I was a technical director at Symantec, I had to formally certify at the end of each quarter that I had not entered into any “side agreements” with customers.

A side agreement is any arrangement that takes place out-of-band, off-the-books, or using private channels not normally examined by accountants. In business, they are usually a bad thing; they can be used to build Enron- or Madoff-style house-of-cards revenue pipelines that are gleaming and glittery at first glance, but that are ripe for collapse because they’re full of hidden caveats and preconditions.

The former Enron towner, now owned by Chevron. Image credit: DaveWilsonPhotography (Flickr)

The problem of side agreements might not impinge on the consciousness of software engineers much, except when they grumble that sales or execs or product management is “selling the roadmap” instead of shipping features. But would you believe me if I said that engineers perpetrate their own Enron-esque side agreements all the time?

Continue reading →

Add some more extra redundancy again

It’s the season for coughs and sniffles, and last week I took my turn. I went to bed one night with a stuffy nose, and it got me thinking about software.

What’s the connection between sniffles and software, you ask?

Let’s talk redundancy. It’s a familiar technique in software design, but I believe we compartmentalize it too much under the special topic of “high availability”–as if only when that’s an explicit requirement do we need to pay any attention.

Redundancy can be a big deal. Image credit: ydant (Flickr)

Redundancy in nature

Mother Nature’s use of redundancy is so pervasive that we may not even realize it’s at work. We could learn a thing or two from how she weaves it–seamlessly, consistently, tenaciously–into the tapestry of life.

Redundancy had everything to do with the fact that I didn’t asphyxiate as I slept with a cold. People have more than one sinus, so sleeping with a few of them plugged up isn’t life-threatening. If nose breathing isn’t an option, we can always open our mouths. We have two lungs, not one–and each consists of huge numbers of alveoli that does part of the work of exchanging oxygen and carbon dioxide. Continue reading →

What are your software’s vital signs?

Most software has a profoundly inadequate concept of “health.” In order for applications to run, they must:

  • have adequate resources (RAM, disk, network, CPU)
  • receive cooperation from services exposed by the operating system or by network endpoints
  • be adequately and correctly configured
  • not be hacked
  • acquire delegated privileges from users

… and so forth. And yet, most software that I’ve encountered in my career does little to see whether it’s working properly and has what it needs. Sure, it may log a catastrophic error if the disk fills up, but it makes no effort to see the problem coming or to plan more graceful recovery than a crash.

In my most recent post on cloudifying your software, I explore how cloud computing is magnifying the need to understand and to regularly check your software’s vital signs. Head on over to adaptivecomputing.com/blog and check it out.

Checking vitals isn’t just for healthcare… Photo credit: U.S. Pacific Fleet (Flickr)

Stay tuned for further installments of this series each Friday. As I said in Part 1, I believe that a competence with cloud–cloud-oriented programming, if you will–will be a checkbox on future tech resumes.

The third half of computational economics

If you look up “computational economics” on wikipedia, you’ll find out all about software models that economists use to study game theory, recessions, scarcity, and so forth.

Tweak your search terms a bit, and google takes you to discussions about the economics of the computer industry–how Moore’s Law plays out in changing prices for compute power, why cloud computing and cheap GPUs are changing how much we expect to pay, how the mobile revolution is killing traditional PCs, what the job market looks like for us software geeks.

That’s all well and good.

But there is a third half of the computer+economics interaction that I don’t hear anybody talking about.

3-fingers

My buddy Ken Ebert likes to joke about incomplete thinking by saying, “There are 2 aspects of the issue…” — while he raises three fingers. :-) Interestingly, this three-fingered gesture is a symbol of sustainable development, which connects nicely to our theme of economics. Photo credit: \!/_PeacePlusOne (Flickr)

Continue reading →