ROI and TCO works for things like forklifts, a new assembly line, tools and company cars. You can calculate how a new forklift will increase the number of trips possible for inventory movement and how those additinoal trips can reduce production cycle times. This will impact labor rates and possibly inventory turns, since inventory can be processed faster. These all drive hard, measurable costs.
In contrast is measuring ROI and TCO for software. Software has intangible benefits and therefore require tricky if not meaningless calculations for ROI and TCO.
While it is possible to know most of the hard up-front costs of software (licensing, hardware, training) the back end benefits are not so easily measured. The whole idea behind an ROI calculation is that if you cannot gain a certain return on the investment then it makes sense to spend your money on something else. For example, the company I work for uses a hurdle rate of 12%: if capital expenditure doesn't yield a return greater than 12%, we don't do it.
For ROI to work, you have to ask specifically how you are going to measure the intangible benefits of purchasing software. For example, how do you measure the ROI of upgrading from Office 97 to Office 2003? There are significant technological differences between the two versions of Office suites but the question is: how many companies are actually going to use and derive measureable benefit from the additional capabilities? Further, how long will it take companies to implement the additional technologies like SharePoint and Exchange 2003 necessary to use some of the cooler capabilities of Office 2003?
In order to measure productivity gains from using software, you must first measure a baseline of productivity from using Office 97. This means you need to look at the most common and most valuable Office functions and calculate the cost and benefit of using Office 97 for those functions. Then, after you calculate the projected ROI (let's say it's a 20 month return on investment), you need to perform the same tasks using Office 2003 and compare the costs and benefits to the Office 97 baseline.
Unless you do a comparitive analysis between a baseline of Office 97 and an actual measurement of Office 2003 productivity, the ROI calculation was nothing more than corporate hoop-jumping designed to satisfy the accounting department and, if the expenditure is big enough, the board of directors. Without a comparitive analysis, ROI has zero meaning because the assumptions used in the ROI cannot be validated 20 months later.. Similarly, the comparitive analysis should be done with the purchase of a forklift, but because a forklift ROI can be meaningfully calculated, the comparitive analysis is usually not conducted. Because of the tangibility of the value of a new forklift, the ROI calculations are usually obvious and end up mapping to real world costs and benefits.
Total Cost of Ownership is a similarly silly calculation in the world of software and hardware. TCO attempts to take a look at both front end acquisition and implementation costs and combine them with the costs to maintain the system over a certain period of time, usually the break-even point for the ROI. ROI and TCO calculations have been used over the past few years to justify the claim that TCO for Linux is cheaper than Windows. The zero dollar acquision costs weighted the TCO calcs in favor of Linux because in comparison to Windows, there is a huge difference: zero dollars to something a lot greater than zero dollars.
The problem with Linux has been unforeseen back end costs, like a lack of Linux talent in comparison to Windows, a lack of 3rd party applications developed for Linux in comparison to Windows, a lack of standards for Linux competence in comparison to Windows' MCP, MCSE, MCSD, etc. (These are opportunity costs rather than hard costs but they illustrate the intangible nature of measuring factors that drive total cost. You could argue though that opportunity costs have real dollar costs but measuring those costs is challenging). Linix has higher back end costs because of the difficulties of fully integrating Linux within an existing infrastructure. There were also likely higher costs because Linux's low-cost to acquire enabled hobbyists to develop skills in Linux, but without clean pathways of certification, quantifying an individual's skills was more challenging. Incomplete Linux skills can drive the cost of ownership of Linux higher than would be anticipated. Because Linux infrastructures are less ubiquitous than Windows networks, Linux applications have less rigorous field experience. (And please do not counter that Linux is installed more than Windows -- it is Apache that is ostensibly the most popular web server on the net, but a web server Linux box does not a Linux infrastructure make.)
Further, there are certain components of a solution or a data center that are invisible and tend to not be computed in TCO numbers. For example, one of my favorite technology bloggers is Sun's COO, Jonathan Schwartz. He describes the differences in the cost of energy between Sun Opteron x86 boxes and Dell boxes. Schwartz cites a Sun analysis that shows that the University of Buffalo should have chosen Sun Opteron boxes instead of Dells because of electrical and heat efficiencies. The University can only use part of the availability of their Dell cluster because of lack of power supply and cooling ability. Read the analysis; it's a compelling example of the hidden costs of TCO. Whether Schwartz is right or not is open to interpretation but his argument does show that there are costs that are often ignored in not only ROI and TCO calculations but in product selection as well.
All this to say that it is nearly impossible to calculate the TCO of a solution because there are too many variables that have varying degrees of predictable values. It is easy to miss invisible factors like electricity and cooling issues. It is easy to assume constant labor costs. In short, TCO technology calcs are subjec to the same fundamental weakness as ROI technology calcs: questionable and incomplete assumptions. A more appropriate approach would be to caclulate a range of ROI and TCO based on stated assumptions but this then renders ROI and TCO to the realm of probabilities and accountants like dollar figures not probabilities.
The bottom line is that you buy a technology solution because your due diligence process (you do have a structured, defined due diligence process, right?) helps you filter out and select solutions that represent the lowest risk and highest level of functional benefit given what you know about the present and what you believe about the future.
ROI and TCO calculations make accountants, CFOs and boards feel happy but everyone knows that these calculations are almost always bogus. I've seen leaders back numbers into the calculation because they know what the ROI needs to be, so they just put in what gets them the ROI. They make assumptions about costs and benefits that may or may not be pulled out of someone's nether regions and therefore may or may not reflect reality. The end objective is to create a spreadsheet that beats the hurdle rate and gives decision makers warm feelings about signing off on the capital expenditure request.
The bottom line is that people buy what they think will benefit them. ROI and TCO is an exercise in computing the value of tangible assets that is incorrectly applied to the assessment of value for hardware and software solutions. I believe that when it comes to justifying a technology purchase, all you can do is attempt to assess risk, and mitigate it by a structured selection process and a well-managed implementation project. You can calculate ranges of costs and assumed value derived from implementation but in the end, calculating the dollar value of benefits from spending thousands, if not hundreds of thousands, of dollars is tricky.
In an issue of Fortune, there was a section on industrial management and technology. IBM's new chip fabrication facility in East Fishkill, NY is described. It is a high-tech system that involves minimal human labor. This is an advantage because decontaminating humans for the ultra-clean environments of chip fab plants is time-intensive and expensive.
One of the characteristics of the plant is an embedded RFID chip that accompanies the pods (known as FOUPs) that distributes the raw materials from fab machine to fab machine:
The most important piece of hardware in the plant is a small glass vial containing an RFID transponder. One of the devices is embedded in each of the plants 5,000 FOUPs, as well as in similar containers used to ship wafers or transport reticles images of integrated circuits that are projected onto wafers. Each transponder emits a signal that can be read from a few inches away by 60 receivers along the monorail and in each of the plants 1,500 machines. When a FOUP arrives at a processing machine, the computer system tells the machine how to treat the wafers it bears. Whistling while it works, the machine sends progress reports and also alerts the computer when its ready for another load. "A lot of people ask, How do you justify it? How do you [calculate] a return on investment for RFID?" says Perry Hartswick, a senior development manager who led the team that automated the plant. "Its like asking, How did you ROI the wires in the walls? Without RFID, I couldnt have done any of this stuff."
Technology is an assumed component of business. Yet, it is an elusive component, one whose total costs and complete benefits are hard to quantify. Yet, IBM felt RFID was a critical component in spite of its inability to quanitify its value. Implementing technology will always be risky because infrastructures are typically complex. This is because they can only be planned once -- at the beginning -- and then afterwards evolve according to business needs and contemporary technologies. As a result, in an infrastructure of any decent age and size, there is a staggering degree of interdependency and therefore, complexity.
I believe that ROI and TCO are meaningless measures of the value of proposed technology solutions. If business leaders decide that a piece of technology is needed, they should be able to implement it without the absurdity of ROI and TCO calculations. Of course, there needs to be a degree of accountability to the leaders and smart leaders will have designed a consistent selection methodology and may ask the solution provider to share in the risk of implementing the technology but ultimately, the decision to purchase technology cannot be purely a financial one. It is also one of desire and hope.
A consistent process of selection and project management can mitigate these soft factors but I believe that all technology solutions come down to desire and hope. Big desires and dreams can lead to big projects that have large pay-off potential but they can also be dramatic failures. Small play-it-safe desires and hope can bring about stability at the cost of innovation that could have provided competitive advantage.
It's time for businesses to remove the centrality of the accounting department and its justification processes from the decision-making process for technology solutions and place them in the hands of business managers who will have accountability for the results.
In addition, I think the software industry needs to develop a methodology for evaluating the cost, value, risk and benefit of technical solutions. That methodology then needs to be proposed to customers as a basis for making purchase decisions.