Build vs. Buy ... Again?

I didn't anticipate talking about work on this blog. Mostly because of some vague fear brought about by those "Privacy in the Facebook Age" articles. Of course, I have always used my real name on the Internet since the days of Usenet groups. So, if any potential employer does happen to search for my name, they will likely learn all of my darkest secrets. Namely, that I am a D&D nerd, that I once wrote an amazingly unknown PnP RPG many years ago (which I still play) and that I have an unseemly attraction to video games.

I can live with that. Being a D&D nerd is one thing. Ranting about your company's business practices, or your boss, or co-workers in a public forum is another.

So, this isn't a rant or complaint, but there is an issue that regularly comes up for discussion in my company (and I am sure it does in many companies) and it tweaks me enough to feel the need to comment. That issue is the age-old question of Build vs. Buy. When you need a new system for data cleansing, or tracking shipments, or maintaining contact lists, do you buy a "turnkey" system, or do you get your developers started on a new project?

From my perspective as a software developer, I fall heavily into the "Build" camp. I am a developer and I want to develop software, not install, configure, customize and nanny software someone else wrote. On the other hand, software development is fraught with peril and the lore is littered with stories of companies spending years and millions of dollars on failed "enterprise" software projects... So if you are an exec, I can see the allure of buying a package off the shelf and it "just running".

I've noticed that when I talk to folks (in my company and others) about their buy decisions, they often say things like... "We had to have a product NOW, not in 8 months" or "We simply don't have the development staff to deal with a project this big" or "The cost of developing that software would have been to high."

How expensive is the package being considered? How fast do we have to get to market? How complex is the problem we are trying to solve? Are our developers up to the task of producing this software? Do we even have enough developers to produce the software?

Obviously, there is no single correct answer. Every case is complex. However, from years of experience watching how these decisions play out, I have come to this conclusion:

If the system you are considering is part of your core business competency, or is part of a package you want to sell or present to the public, then 9 times out of 10, you should start with the idea of building that system, and fall back on buying as a last resort. Here's why:

Turnkey Solutions Usually Aren't -- One of the big draws of buying pieces of a mission-critical solution is that you can get it up and running more quickly than if you spent the time building it. Sometimes this is true; you install the new package, do a couple quick configuration tweaks and it is producing worthwhile results.

Unfortunately, more often than not the story is install the package, do a couple of quick configuration tweaks, have problems with the security settings on your network, do some more tweaks, call customer support, they blame Microsoft, call Microsoft, they blame the software vendor, post on a forum, retweak the configuration, things seem to be working, but the results aren't as useful as you originally thought, now you have to customize the package, in some goofy proprietary scripting language, why couldn't they just allowed you to write a C# assembly, oh well, customization is done, results are good, users are complaining that system is slow, do some profiling, it is the package's weird batching system, call software vendor, they send consultant out...

Whether you build or buy, there is almost never instant gratification in the software world. You might balk at the idea of giving a team of developers 4 months to deliver a piece of functionality that you need today, but keep in mind that most off-the-shelf packages require lots of setup, configuration, and often outright customization before they can deliver useful results. I call this the "hidden work" of buying, because while you might meticulously track the time your developers take to build in-house solutions, few companies give the same level of scrutiny to the fact that Bob from the data team spent 6 months playing nursemaid to that QuikSolv package you bought expecting instant results.

Obviously, if there's hidden work associated with turning the key on your spiffy new solution, there must also be hidden costs. Whether it is the $250/hour consulting you have to purchase to get the solution customized enough to work in your environment, ongoing support costs, or the fact that years later, your Senior Developer still has to babysit the thing when it is at peak load, you're going to pay a heck of a lot more than sticker price for packages you buy.

You Are Buying What Someone Else Needs -- Another thing to consider when buying a software package is that you are just one customer out of many and your co-customers have different needs than you. You are splitting the development costs with these co-customers, but you are also splitting the development resources and paying for features that they need, but you don't.

In my experience, this is evidenced in two main ways. The first is the "Everything to Everyone" package. You just needed a sleek, web-based grid, so your company purchases "Ulti-Grid" based on a demo page and a quick trial. Once you start digging deeper into Ulti-Grid, you realize that it has this horrible, 100-page API and a host of features you will never use... and that ultimately make the features you will use more fragile. Whereas you just needed a web-based, grid with some shift/ctrl click capabilities, you got the titanic AJAX-enabled, super-scripty, WPF, WCF and WTF-capable grid from Hell.

Oh yeah, and when they update the package, all of your code breaks :P

The other issue is when you need support, or a hotfix, or would just like to see a set of features implemented, you aren't at the top of the queue. To be fair, most companies are pretty good at handling critical issues quickly when dealing with production systems, but if you find an issue during development, especially if it is a subtle issue, or is performance-related as opposed to just an outright exception... good luck.

Even Worse, You Aren't Buying What You Need -- If buying a package that has too many features doesn't bother you, then how do you feel about buying a package that fails to meet your needs? Software development is often a moving target. The business' needs change and our understanding of the problem changes. When that happens, don't be surprised if your plug and play package is all of a sudden lacking key features needed to integrate it with the rest of the system and then you are stuck waiting for a new feature release, adding complexity to the parts of the software you ARE building, and/or customizing the package... which is great business for the $250/hr consultants, but stinks for you.

Sometimes You Need To Know How To Make a Wheel -- The old saying goes, "Don't reinvent the wheel." Obviously, there are a lot of cases where this makes sense. Reusing code is the prime example and purchasing a package as opposed to building one is just another form of code reuse.

However, Jeff Atwood had an interesting take on this on his blog which was, "Don't reinvent the wheel unless you want to learn about wheels." To me, this makes a lot of sense, even in a cost and time-conscious business environment, there are skills and techniques your programmers need to have and they can't get those skills by taking classes, or reading, or implementing someone else's solution -- they need to build software. So if you want your software to have a nicer UI, you better start letting your developers build UI. If you want better data processing, you should give them a chance to build the data process and so on.

The decision to build or buy a piece of software is hard and it is often much easier for a non-technical manager to understand the benefits of buying a package (and the risks of building) as opposed to the converse. Do I believe you should build everything? No -- there truly isn't enough time or money to build every component in a large software project. However, the next time your company is faced with this decision, I hope that the hidden pitfalls of "buy" are exposed and "build" gets a fair shake.