Saturday, April 15, 2006

Open Source R&D

One of my esteemed readers made the following comment about open source R&D on one of my OpenOffice comments:
To me, open source SHOULD have a more intuitive interface as well as features. OO clearly is missing that boat... ok missing the dock.... ok it is in the middle of a desert. As I see it, R&D dollars shouldnt be an issue as more open source geeks are the type who would say "this sucks, I am going to modify it" then turn the code over the the originating "owner" Where as I would think non-OS would run into too much red tape before doing something cool.
There are at least three problems when it comes to the open source development of an application suite with consistent functionality and an intuitive, visually enticing GUI:

1. lack of R&D dollars
2. lack of coherent development focus
3. lack of sustained, consistent usability testing

Lack of R&D Dollars:

The use of labor to produce something always has a cost asociated with it, usually money. Open source software development is no exception. OS software development is either approached from a hobbyist perspective, as implied above or it is approached by a for-profit company like Sun, IBM or Novell to be distributed for free to its user base. Since corporations are usually in the business of making money, there is usually a strategy of revenue generation at the back end (e.g. consulting services, hardware) to compensate for the price of Free in the front.

However the software is distributed, what is inescapable is the front-end costs associated with software development. Until we make software that can write other software without human intervention, software will always cost money to make. This is true for the fundamental reason that people have this relentless dependence on food, shelter and clothing and investors have a relentless desire for returns on investment.

Why do OpenOffice and StarOffice have such kludgy GUIs? In part because they are developed from the beginning as something given away for free or very cheaply (Sun charges ~ $35/set for StarOffice; OpenOffice is completely free). Because businesses are essentially oriented around generating revenue and profit, there is a limited amount of R&D dollars that can be spent on software intended to be low-cost because the expenditure is up front and the actual revenue generated through consulting or training services or hardware sales or maintenance contracts can't be entirely predicted. This implies a difficult decision about how much to invest in an open source project so that the final result has some degree of market acceptance but which doesn't require a company to overextend itself on the revenue it assumes will be generated by back-end services.

On the other side of OS development is the hobbyist programmer. Their primary investment is time. Presumably, they don't need money to code open source projects because they make money in some other manner. Yet the hobbyist programmer simply cannot invest the time needed to develop a solid GUI because they flat out lack the time and work capacity to do so. I suspect that OS programmers are motivated more by pride and a belief in the GPL than a desire for money, yet pride and ideology may not be enough to maintain a sufficient pool of adequately talented programmers to see a large-scale open source project through to completion. Similarly, it is difficult for an open source project manager to rely on hobbyist programmers, who will have varying levels of consistency and follow-through. Because the hobbyist programmer does not have their livelihood tied to an open source project, they have little extrinsic motivation to follow through.

In contrast are corporate development projects that are well-funded (at least in the case of Microsoft this is true). While corporate programmers can lack the intrinsic motivation that is probably more prevalent on open source projects, they may be motivated not only by "finish the project or else..." but by the resources available to them that hobbyist programmers would lack.

Lack of Coherent Development Focus:

In a pure hobbyist development model, there is a lack of a coherent development focus that guides decisions regarding functionality, GUI aesthetics and under-the-covers methodology. This is significant because there needs to be some method for concentrating unaffiliated hobbyist programmers so that they develop a product that will function and meet customer needs. It is rare for a person with strong programming skills to also possess an appealing sense of aestehtics and an awarness of what makes GUIs effective. So, yes, the open source model allows a hobbyist programmer to look at something in a product and say "Hey I don't like that; I'm going to change it," but this does not mean that the changes make sense and will lead to a product that has high user acceptance. In fact, if too many hobbyists have that response to what they see and they are not guided by a development focus, then changes become chaotic and any potential quality in the development will fall.

In a coporate environment that devlops OS apps, there is more of a coherent focus but again, the efforts of R&D for software destined to be free will be limited by budgeted cash. Most likely however, a company like Sun Microsystems or IBM, for example, will already have internal methodologies for development that can be applied to open source projects. The corporate-sponsored open source project will be much better positioned to create good software than a loose affiliation of hobbyists because there will be more internal compliance to a development model.

But again, the intent of a corporation is to generate revenue and profit. Spending money on development of an application that will be given away for free is a bet on the ability to generate revenue later on back end services. So for example, Sun doesn't view the code for OpenOffice as a corporate asset that holds measurable economic value. They see it as a magnet that will help induce alternate revenue streams. In contrast, Microsoft views the proprietary code of Office as a corporate asset with measureable economic value not only at the front end but also in its ability to generate revenue by Office's ability to integrate with other technologies like Exchange, SQL and SharePoint, all of which generate up-front revenue.

Microsoft has a robust and vigorous development model. They have a solid development platform (.NET), an application architecture that separates the code into three layers (user interface, business logic and data abstraction) and a lifecycle development methodology. My reader is correct in saying that corporations often inject "red tape," otherwise known as politics into the mix but overall, I would say that a well-managed software project can be designed to circumvent politics and corporate policy.

Lack of Sustained, Consistent Usability Testing:

In both cases of the open source corporation and the hobbyist programmer, there will be a lack of sustained usability testing. Why? Because it is expensive. Hobbyists not only lack the dollars needed but they lack the coordination with other hobbist programmers needed to do so. Corporate open source projects are limited in the amount they can spend on usability testing and user acceptance.

Usability testing requires iterative development cycles, where each cycle of development receives usability feedback which then leads to another cycle of decision-making and development. Microsoft has invested boatloads of money into Office usability and they have made some wise decsions about Office integration with other MS products (e.g. SharePoint). Hobbyist programmers simply do not have the labor and dollar resources to field test their changes with users.

Certainly, as my contributor pointed out, there is a lot of internal friction to overcome within a large corporation to get changes out efficiently. However, there is a tremendous investment in Office. Office and Windows are easily the primary sources of revenue for the company. Their ongoing development of Office indicates that they do not take their dominance lightly. They want to maintain their position, so they pump millions into R&D so that their position does not erode. That erosion is, in my opinion, almost a certainty because I am convinced the browser will replace Windows as a computer OS but nevertheless, Microsoft has a substantial stake in the security of Office's dominance and so they make significant investments in user acceptance.

In my opinon, software development cannot escape from the need for a centralized layer of decision-making; someone must have the authority to make a decision after everyone has voiced their ideas. The development of software that has expectations for broad user acceptance needs a top-level source of cash and decision-making authority. All one has to do is surf for examples of the quality of software most hobbyist programmers churn out and compare it to Office 2003. Then compare hobbyist software to OpenOffice and notice the similarities. OpenOffice has a hobbyist feel to it. Using OO leaves one feeling a bit hollow, with a feeling that something is missing.

That something is exactly what happens when a product has ample R&D dollars; a consistent, coherent development methodology and iterative cycles of usability testing leading to high levels of user acceptability. OpenOffice is what happens when those three factors are lightly-weighted.

Bottom line: Open Source simply cannot touch the sophistication of corporate software development.