Tuesday, May 02, 2006

ROI Gibberish and A Video to Illustrate It

I have argued before in my blog that for technology, ROI and TCO calcs are accounting gibberish that companies use to justify the costs to acquire new technology, but which are largely useless because they don't lead to meaningful information. I have a couple of components to my argument against ROI and TCO calculations for technology buys:

1. The actual dollar value of benefit gains are wrapped around assumptions that have varying probabiliites of accuracy, where "accuracy" is defined as how closely the assumptions map to the actual values of costs and benefits (and often the total cost and value picture isn't known at the time the ROI and TCO calcs are made). Further, these calculations tend to only look at benefits. They do not look at, for example, the loss of productivity that happens during implementation and during the user's learning experience as well as other factors that could potentially minimize the net benefits associated with the project.

2. ROI and TCO calcs are most accurate when direct costs are used. They are less accurate when indirect costs are applied. For costs like software licensing fees and hardware acquisition, direct costs are pretty solid. But there are other costs like hourly consulting fees that can easily go over budget. Further, these calcs can be hampered by the opportunity cost of the support provided by existing staff to the technology project in question: when staff is supporting the project, they cannot do their regular jobs. For disciplined companies, the cost of validating the ROI after the payoff period has expired adds costs to the project and diminishes the expected return (see item 3).

3. Most companies do not bother to measure the actual ROI of a project after implementation and after the projected payoff period. This implies that the ROI and TCO calculations are merely exercises to mollify the budget-keepers. If a company truly valued its capital and the need to satisfy a capital hurdle rate (ours is 12%), then they would also have the discipline to validate the ROI after the payoff period has passed. To me, the fact that most companies don't do this means that the ROI and TCO calcs are either acts of obeisance or they are simply traditional, yet vacuous corporate exercises.

4. Individuals who want the project approved adjust their assumptions not to reflect reality but to get the numbers needed to get project approval. It simply involves working the numbers backwards.

Check out this four minute cNet video on performing the TCO calculation. It's a wonderful illustration of my objections to these common exercises. There are two key problems with the claims made in this particular presentation.

Common metric for decisions on all potential projects: She asserts that the value of the ROI calc is that it allows a business to evaluate all potential cash outlays by using the same metric: namely, the value of expected benefit in relation to the up-front costs.

This is not true in all cases and is particularly sketchy with technology purchases. This is because the benefits of technology and the dollar value of those benefits are difficult to measure. In contrast, the ROI of a forklift is more tangible because the dollar value of benefits are easier to measure (e.g. increases ability to move inventory to assembly, which helps increase inventory turns, etc.).

She completely glosses over the value of benefits with a straight face: In her example, she says something like, "We are going to experience an increase in productivity that will drive revenue gains of $80,000/yr for three years."

This is exactly the kind of statement that completely undermines the value of ROI calcs. She doesn't describe how to go about calculating the value of the expected benefits. Specifically, how will the expected benefits translate into measureable dollars? The dollar value of benefits magically appear in ROI spreadsheets as functions of assumptions about those benefits and no one ever challenges the ROI calculation even though everyone knows the assumptions are usually innacurate and incomplete.

The reality is that most companies do not have a clear idea of the dollar value of benefits with technology acquisitions because tech buys are largely intangible. Further, the broader the scope of a tech buy, the more systemic its effects will be. This means that implementing an ERP application for a whole company will touch more business processes with more potential for broad benefit or catastrophic disruption than say implementing AutoCAD in the engineering department. The intangibility of tech buys can be made more concrete by conducting baseline analyses of the costs of business processes and redesigning business processes to derive benefit out of the tech buy. Of course, most companies don't have the discipline for this kind of validation.

Some tech spends have variable benefit value based on whether the technology is fully utilized. For example, if your company uses Office primarily for memos, localized number crunching (i.e. individuals running their calculations without regard to other departments) and databases, there isn't likely a whole lot of value in upgrading from Office 2003 to Office 2007. But if you take Office 2007 and integrate it with SQL 2005, SharePoint and Microsoft Business Scorecard Manager to distribute key performance indicator metrics to the company, you will experience incredible value for the cost of upgrading to Office 2007.

The bottom line is this: for tech spends, it is difficult to accurately measure the dollar value of expected benefits. ROI calculations are hampered by this difficulty. TCO calcs are hampered by the presence of direct and indirect costs, as well as costs that are ignored for various reasons. For example, cold room electrical consumption may be viewed as an overhead cost, which makes it difficult to evaluate the cost of electricity as a decision-making factor in choosing one server or another. This is why Sun's marketing strategy to sell SunFires on the basis of lower power consumption and lower heat throw is not likely to connect with a lot of companies. I address this issue heretoward the end of the article.

The bottom line is that in many, if not most cases, decision-makers buy software solutions because they have an intuitive or hopeful sense that the software spend will generate real process and dollar benefit. I have seen decision-makers tweak assumptions known to be erroneous simply to get the ROI over the hurdle rate so they can get their project approved. In other words, they adjust the numbers at the back end of the calculation in order to justify their desire and hope.

Risk analysis then is also intuitive and is measured by comparing the risk of failure with the comfort level among decision-makers that they have made a good selection and that the project will, in the end, be successful at delivering value.

Technology is bought because of emotion. It's not a purely financial decision.

As much as accountants would like it to be otherwise, technology spends are primarily emotional. Technology is bought because of intuition, desire, coveteousness, a need to appear significant and a deep hope that something can make business pain go away.