Generally, it’s Wiser to Buy than Build
Most of us started working in IT because we love to build stuff. After a while, we started managing people that love to build stuff. For good and ill, it turned out the people we manage possess the same traits that we took pride in ourselves: fearlessness in the face of complexity, the insatiable need to understand how a thing works, and the willingness to sacrifice limitless hours on the altar of a problem. These traits comprise both the core strength of a good IT shop, and the primary challenge of managing it.
The scope of capabilities that you are responsible for delivering to the organization is often far beyond the range of subjects that a single human being can have detailed knowledge of. That’s why it’s nice to have such great people working for you. At least there’s somebody you can turn to.
Just look at that shady vendor offering you the analytical capability that you are going to be evaluated on delivering this year. If your BI developer Avery hadn’t told you after the demo that they were sure that they could build exactly the same thing being presented, you would have been stuck with a bill equivalent to a year’s salary and had to jump through purchasing and legal hoops. Thank goodness.
Thank Avery. A real straight shooter, reminds you of a young you. Loves to explore any new technology, willing to put in the hours to tackle the most intractable problems. In fact, Avery was just talking about wanting to learn about the new analytical capabilities in RiskSoft, and had an idea about a POC a couple of months ago.
Therein lies the risk. We are surrounded by technologists, and it will always be thus. The traits in those that we seek counsel from are weighted toward the decision to build. Sometimes, it’s the right answer, but remember, when you ask a technologist, it’s almost always the answer.
Is it Worth the Risk?
The next stage in maturity as a manager of technology is to realize that we are managing technologists and to realize that we are technologists ourselves.
Complexity
We are fearless in the face of complexity. In the bliss of youth, that fearlessness let us build the impossible. The failed experiments are charming remnants of misspent summers that litter our garages, clog our hard drives, and haunt our EC2 image backlogs (depending on age). As a mature manager, we are no longer blind to complexity risk and seek both to reduce it and to pull our colleagues out of it when so mired.
How Does it Work?
We simply can’t not know. What is even more tempting, we know we can know. I’ve forced my own children to take the lid off the back of the toilet to witness the simplicity of the invention upon which civilization rests. It’s the knowing that every piece of technology is a simple, often ugly idea wrapped in a clean white cover that launches a lifetime of invention. But remember, that’s the background of everyone you work with. Every single one of them is itching to take that remote apart. Your job is to make sure at least one of them can put it back together again.
The Hours
If everything just worked the way it was supposed to, it would take half the time. Technology is fixing. Most of us have learned to double our own estimates when looking at a new problem. What we have learned after years of being sliced by the cutting edge is that there are many species of unknowns. Remember to double the number when a young technologist brings you an estimate. They may not yet have been properly scarred. Remember, they are the young you, not you.
Build vs. Buy
There are times to build and times to buy. Step 1 is to understand your situation. Is the capability you are trying to deliver unique to your department, your company, your industry, or your function? The more specific case, the less likely it is that somebody has created a product to solve your particular problem. But think about how specific that has to be before there’s somebody who tries. Have you had tangerine infused double IPA? It’s fantastic.
Step 2 is to evaluate the market. What you’re seeking is a Commercially Available Off the Shelf (COTS) solution. There will be a range of solutions that range in maturity from infantile compared to your internal processes to something mature enough that you can learn from. Your leadership in the subject area will help determine what you should expect from the market. Evaluate the vendors accordingly.
Step 3 is the evaluation yourself. Is this the area that I want to spend precious development resources? If so, this had better be top priority/Sustainable Competitive Advantage area. Even so, can I leverage existing resources to get there? Build vs. buy is not necessarily binary.
Speed to Value
With analytics, this is typically the primary driver in the business case. There is a cold, clear, calculating way to build that case and an even less enjoyable emotional way. The former estimates the value of a better decision in the tactical and operational areas that the analytical capabilities evaluate. In all but a few cases, these benefits to the business eclipse any purchase or investment required to produce the capabilities. Every day wasted is money lost in a considerable proportion of the project cost. The emotional version of the business decision typically involves an executive who loudly protests flying blind in this rinky-dink operation. Both approaches result in the same expletives describing the eventualities of not getting this done right now.
Build Risks
Separate from the when can this get done is the evaluation of how likely this is to get done at all. Too often, projects are evaluated on a dollar-for-dollar basis on the assumption that upon the passage of time and the investment of dollars that success occurs. Because probabilities are difficult to estimate precisely, let us only concern ourselves with risks that are engaged only if a build decision has been made.
Platform Risk
Analytical solutions have a lot of moving pieces. The boring, but required stuff includes: the interaction with the production data, the ETL, the database, the calculations, the visualization platform, the visualizations, the hardware and OS each are hosted on – whether it be bare metal, virtual, private cloud, public cloud, did I forget anything? Probably, and I do this for a living. Then there are the exotics: is there a multi-dimensional database, a NoSQL database, an edge database, streaming analytics, AI/ML models, custom visualizations, a new thing that just rolled out yesterday?
Each of these steps is an opportunity to make a wrong decision. If you’re in a common or long-standing situation, nearly all of those wrong decisions have already been made by somebody else. Technological entrepreneurism is a Darwinian dynamic. There are enough people sick of their boss and sufficient bored capital such that every idea has been tried, no matter how dumb, middling, or brilliant. The evaluation of which is which is made only by survival and always in retrospect – the boasting of the fortunate notwithstanding.
The surviving products in the marketplace are the ones that have lucked into the right combination. Thankfully, you do not see all of the mistakes that have been made, because they are no longer on offer (with caveats*). Learn from them or take advantage of the bullets already taken. The costs of venture capital lost on other firms do not figure into the price of the surviving products – that savings is passed directly to you, the customer.
* Zombie products may continue to stagger through the marketplace, usually from large vendors that can afford to feed them enough brains. Vaporware has not necessarily been vetted for platform risk. Please do yourselves and the industry a favor and weed out vaporware by asking for references or POC’s if you are the launch customer.
Functionality
Functionality risk is the chance that the platform chosen or combination thereof will not deliver a minimally viable product (MVP). Consider the issues with compatibility of each of the components listed above, both stated and practical (rarely similar). How many times will decisions need to be rolled back before the proper solution is reached?
There is a non-zero percentage chance that the cost and/or duration of the experimentation required to arrive at the proper combination will lead to the termination of the project. Low to mid double-digits is probably a fair empirical estimate to this observer, but it certainly varies vastly by team.
Obsolescence
Obsolescence is the risk that the platform chosen will not survive long enough to return value on the initial investment. What is the likelihood that the existing platform will continue to provide capabilities that exceed or are comparable to existing and future competitors? How does one answer that with certainty? May I recommend leaning on inequities?
In certain industries and functions, scale is a killer that can make up for a great number of bad decisions. Choose the platform that has both the wherewithal and inclination to invest in new capabilities at a pace greater than its rivals. Rivals will continue to emerge in individual functions, Edge DB, cloud MDDB, etc. Look for an end-to-end solution with best of breed components in the path to value for your organization. If they don’t work together, they don’t work at all.
Project Risk
Suppose that an appropriate risk discount is applied to platform risk or that such risk is mitigated by previous experience. What then is the remaining risk that the project does not reach completion or does not deliver a sufficiently compelling MVP?
Does not Complete
Given the word “go” for build in the build vs. buy binary, what are the chances of project success? We have incorporated the risks involved with platform selection above, but not for the host of talents required to achieve a valuable product. Is there sufficient familiarity with the technology to both know what is possible from it and to create working code? Are there analysts on hand with sufficient familiarity with business operations to properly gather requirements, and to translate that language for the technologists? Are the integration skills present in the team achieve the finesse with interactions between components of the platform chosen? Do any of these issues raise a non-zero percentage chance for failure? Do any of us believe this is a complete list?
During the entire period of development, the entire investment of money, people, and organizational attention can be lost due to loss of sponsorship, loss of funding, loss of critical skills. Reduce the time in development. Agile is the watchword. Work in progress is the enemy.
Moon shot in common parlance, means exactly the opposite of what happened. Let us not forget that before the Eagle had landed, there were Goddard rockets prototyped in 1915, empty rocket launches, a satellite, a dog, a monkey, a person, two people, Earth orbits, moon orbits, and a bunch of problems along the way that were bad, but not fatal to the project because significant, measurable progress had been made. Let’s take our moon shots in the same way and use the benefits from achievable wins to fund the next step of the project.
Does not Deliver
Even if the project gets to complete, navigating the sea of risks above, often the end result underwhelms. The typical cause for this is lack of vision. This sounds derogatory, but as stated above, predicting the future has proven difficult in most organizations. The items that cripple the project are the unforeseen issues that are encountered along the way. A more abundant resource than vision is having done exactly the same thing before. If that resource is available to you, avail yourself of it.
Quicksand
The most common scenario, and the most difficult is this. We took a shot on Avery, who is our best, but it didn’t work out. We lost a little money, a little time, and a little reputation. Do we risk losing a little more money, a little more time, more of our reputation to break even? This is the moment, when we should pause, and, though I do not say it often, take a little advice from Robert Plant:
We could re-evaluate. That off the shelf solution that already solved the platform risk and has value out of the box that would allow us to hit our timelines may not be so bad. If we miss the second time, and we’re down quite a bit, are we likely to make the big gamble, knowing that now gambling big might be our only chance to save our reputations?
The decision is so common that most trading operations have systems in place to check the behavior. Just ask Barings bank (oh wait, you can’t). The truth is, we all double down, and we do it for all the same reasons that we have achieved whatever success that we have achieved so far, and for the same reasons that we trust Avery. We place enormous confidence in the ability to handle complexity, to understand all the moving pieces, and to rescue ourselves with heroic effort. Though we may be wistful for the adventure and the age, which of us would wish this on their worst enemy. Be the adult in the room, evaluate the risks and choose the surest path to victory. That’s what you are there for.
Northcraft Analytics is here for you if you want to walk the direct path. We have spent centuries of technical labor navigating these risks with our clients and would love to save you the pain involved learning some very hard lessons. That’s what we are here for.
Some decisions take an enormous amount of resources to defend and are very difficult to escape from. I’m looking at you Australia . . . and a few build green lights that will remain nameless.