Written by Martin Butler
When I came across behavioral studies in finance and economics I realized that I had found something that explained many of the behaviors that can manifest in the IT investment process. These behaviors, or biases as they are generally known, are part of the way we work as humans. It would be a mistake to think that analysis of biases is some form of judgement. Only the deliberate corruption of the investment process is worthy of judgement – biases are just the way human beings behave for emotional and cognitive reasons.
It would have been very difficult to write a paper of this nature without the categories that behavioral studies have created. Such a paper might just have been seen as a loose collection of observations, instead of the well defined behavioral traits that the biases define.
I haven’t seen another study of this kind for the IT investment process, and while this topic certainly isn’t my main research area, I felt strongly that the approach needs to be introduced to the IT community and distributed as widely as possible.
It will hopefully become clear to you as you read this paper that a widespread understanding of behavioral biases will change the way we invest in IT. Unrealistic expectations, the slavish following of IT fashions and much other behavior that is core to the way the IT industry works, will be better understood.
There are around thirty or so biases that are applicable to the IT investment process. This short paper doesn’t cover all of them. We’ll just look at the most relevant. If there is sufficient interest I may produce a fuller paper at a later date.
I hope you find this paper useful and somewhat enlightening. I know that many managers are puzzled by the difficulties that IT investments often present – this analysis should throw some light on the matter.
The IT investment process is pretty much the same as any other investment process. Research is conducted, information gathered, reports written and a decision made. This whole process has been the subject of serious study for at least twenty years and is now encapsulated in behavioral finance and behavioral economics.
The methods that are used are equally applicable to IT investments. Some of them address issues that have plagued the IT investment process for decades, and provide a formal environment for the observation of behavior that many experienced managers will recognize immediately.
IT investment decisions based on careerism (selecting the supplier and technology that provide good career opportunities) are fairly common. If this is present then the analysis of various biases will be a pointless exercise. The same is true if a decision maker and supplier have some form of side agreement – behavioral analysis will add nothing.
If we can be reasonably certain that careerism and side agreements are absent then behavioral analysis will add a great deal. Some of the more dominant biases include:
- Availability bias – using easily available information to make a decision rather than the information that is really needed.
- Overconfidence bias – overconfidence in the outcome of a decision and in the information used to arrive at that decision.
- Framing bias – using a weighted, unrepresentative set of criteria for making a decision.
- Loss aversion bias – pouring good money after bad into a failing project.
These are the main biases outlined in the paper. If an organization could just handle availability bias effectively it would stamp out the slavish following of IT fashions and deliver more relevant solutions to the organization. These are very big issues, and if they are to be dealt with effectively there will have to be sponsorship from the very top of the organization. We will after all be analyzing the preferences, biases and predispositions of powerful figures.
While the IT industry continues to churn out new technology fashions to sustain its growth, most user organizations are sinking under the weight of complex, incompatible technologies. Standish Group reports on the real state of IT in large organizations every year – it isn’t a pretty sight.
The essence of the message in this paper is that successful IT is not particularly dependent on the technology you use – it is very dependent on the biases and behaviors of everyone involved in the IT effort. So this is where the effort must be focused if organizations are to reduce their information costs and improve the value they derive from their information assets.
Is IT Investment Efficient?
There is only one mechanism that would allow the IT investment process to be efficient. If those involved in the investment process had access to all the information relevant to the task at hand, then they would be equipped to make an efficient investment. For a variety of reasons this doesn’t happen. The main sources of information are:
- Suppliers claims for the functionality and performance of their technology.
- Articles in the IT press and business magazines.
- Consultancy advice and studies.
- Analyst reports.
- Peer group comment.
- Case studies and reference sites where a technology has been deployed.
All of these sources are flawed in one way or another. Supplier claims are usually taken with a pinch of salt, and for good reason. It isn’t in the supplier’s interest to talk about the problems that might arise, or admit to product weaknesses.
Articles in the press are generally of an informative nature. They typically address new developments in IT where the benefits and risks associated with a technology are not yet known.
The basic business driver of most consultancy companies can weight consultancy advice. Their bread and butter revenues tend to come from projects that exploit a particular technology or skill set. They will obviously want to bias information toward their own services.
Analyst reports suffer from a similar problem to articles in the press. They address new IT trends where experience is limited, and of necessity cover the relevant issues in a generic manner.
Peer group comment can be a good source of real information, and if not biased it is perhaps the most useful source of information.
Case studies and reference sites are self selective. Suppliers and their customers do not want to expose problems or failures and so the information is weighted.
Despite these reservations the six information sources mentioned above are usually all we have. It should be obvious that the IT investment process cannot be wholly efficient and that risk is involved. A thorough analysis of information available through these sources is a significant contribution to risk reduction. If organizations were more open about their experiences with IT the process would be less problematical. The limited information that is available to those involved in the IT investment process means that considerable risks are present. These are the risks we have to live with. However the situation is aggravated by behavioral issues, and these can be addressed to a large extent. This is where our focus will be for the majority of this paper.
Overconfidence is one of the most widely studied biases and has been found to be a major influence in many types of investment process. One of the best known overconfidence studies looked at the tendency of the CEO to exhibit overconfidence when acquisitions were being considered. It was found to be a major factor in the poor performance of many acquisitions, with CEOs exhibiting overconfidence in their ability to predict the outcome.
Overconfidence not only applies to an individual’s estimation of their own abilities, but also an over estimation of the information they possess. Many of the biases we will discuss are linked, and information overconfidence is closely linked with availability bias, to be discussed later.
The hallmark of overconfidence is confidence intervals that are too narrow. Most outcomes can be defined with an upper and lower confidence level. The deployment of large application suites is a classic example of this. Benefits and costs tend to be precisely defined with no real allowance for the disruption that might occur. Underestimation of downside risk is a classic manifestation of overconfidence. There may be political reasons for specifying unrealistic timescales and budgets for projects. An IT manager may be competing with an external supplier, or senior management may be intolerant of more realistic estimates. For now we’ll simply take note of these issues, and continue with the analysis of overconfidence bias.
This particular bias manifests at many levels during the IT investment process and also during deployment. Senior management can be left feeling overconfident by slick presentations from suppliers. It is of course the job of the salesperson to be overconfident, and if a presentation goes well this can rub off on management. There is often an unwillingness to consider downside risk. Many projects simply would not happen if a realistic estimate of downside risks was included in a proposal – or they might be approached in a manner where downside risk was reduced.
Technicians and technical management are also prone to overconfidence. It is fairly common to see projects approved without a reasonable appreciation of the technical difficulties that might be encountered. This is very much linked with optimism bias – discussed later.
The cure for overconfidence bias is fairly simple, but probably not easy – depending on the cultural climate that exists in your organization. Investment proposals should take full account of possible downside risks and document various possible outcomes. It is more useful to talk about expected timescale and budget, with various scenarios existing either side of these expected values.
Analysis of downside risks sounds like a particularly negative thing to do, particularly when enthusiasm for a proposed project is growing. In reality such an analysis will nearly always add to the outcome – drawing attention to risk reduction and more realistic expectations.
Availability bias sits at the heart of the way the IT industry works. It is quite easily the single most powerful driver in the IT investment process for many organizations. Widespread understanding of this bias would do nothing less than reshape the whole IT industry. This bias is easily understood. It is the tendency to use easily available information for an investment decision, rather than the information that is actually needed.
Such is the nature of the IT industry that information on the latest and greatest applications, technologies and systems is available in great abundance. Everyone is keen to jump onto the next wave of enthusiasm, leaving little room for a more considered analysis. Resisting a technology wave can be commercial suicide for suppliers and commentators alike. This means that uncritical information floods the market.
I’ve had many conversations with managers and technicians who point out that impartial, objective study on the performance of a technology are pretty well impossible to get hold of. Because of this the available information is weighted. Managers looking for information as part of their investment process will be presented with unrealistically optimistic commentary and analysis. What they really need is impartial data on the real performance of the technology – and it usually is simply not available.
This is a self feeding situation. I know from personal experience that users of a technology simply will not share their real experiences. Everyone is happy to talk about the upside – very few will talk about the downside. The information that managers need is usually not available, making IT investment an inherently risky process.
A lack of impartial information is actually very useful information in its own right. Alarm bells should ring if available information is unrealistically optimistic and users are unwilling to talk about their real experiences (particularly downside risk). Investment proposals should highlight the fact that available information is one-sided, and that there is very little, if any analysis of downside risk.
An organization could set out to collect information if a particularly large project is being proposed. This would mean efforts to contact people familiar with a technology in other organizations and an assessment of their willingness to talk about downside risks. Simply using the press, analyst reports, consultants and suppliers as a source of information will not usually result in the information that is needed.
People at all levels of the organization are prone to availability bias. This includes the senior manager who reads an article in the business press, and the technician who comes across a new technology in the IT press. It is unfortunately the case that availability bias creates a strong positive feedback loop and the fashion driven nature of the IT industry. This is a very powerful dynamic and requires uncommon courage if IT investments are to reflect what the organization needs, rather than what available information is saying it needs.
All investments have some frame of reference. In IT this may be nothing more than an abundance of available information on a new hot technology, dissatisfaction with existing systems, a perceived business need or simply the need to keep up with peers. Research has shown that frames are typically too narrow, focusing on a few pressing problems without considering a broader range of issues.
Framing bias occurs when an IT investment is unduly weighted by a limited frame. It may be that investment in a new application suite is considered because sales processing has been problematical. The rest of the organizations would suffer disruption and a poor systems fit, just to solve problems in one part of the organization.
Suppliers are obviously very keen to influence the investment frame. The manner in which a technology is presented has considerable influence on the way managers view a proposal. Suppliers will use peer group pressure, the promise of ‘standardization’ or any number of factors they feel will affect the investment frame. Most notably suppliers will weight the frame on the benefits side of the equation and will obviously avoid downside risks.
The only valid frame is a broadly defined one. It should consider all aspects of the organization that will be affected – costs, skill sets required, perceived benefits, downside risks and any other issues that are particular to the project under consideration. This level of ‘democracy’ in an organization is a fairly rare event. Projects are often sponsored by powerful individuals with very narrowly defined frames, resulting in systems that at best show little benefit in the rest of the organization, or at worst result in damaging disruption.
The history of IT investments in an organization also affects framing biases. If IT has typically been problematical it may weigh the frame unduly toward risk aversion. If the last project was a success it may have the opposite effect.
The investment frame is worthy of documentation and analysis. It should consider the frames proposed by all significant players in an investment proposal – suppliers, senior management, technical management, user management and technicians being the obvious candidates. Each will frame a proposed investment in their own way and a comparison will bring any framing biases to light.
Loss Aversion Bias
The fact that a loss has at least twice the impact that a gain of a similar amount has on investor psychology is well documented. This creates a loss aversion bias that manifests in IT as an unwillingness to pull the plug on ill conceived projects. It is fairly common for managers to spend good money after bad in an attempt to rescue a failing project, instead of assessing whether the investment was a good idea in the first place. A recent example of this is the US jewelry chain Shane Co. This company implemented SAP point of sale and inventory management applications and ended up spending $36 million instead of the $10 million it had budgeted. The company claimed this contributed to its failure. Obviously someone had a severe case of loss aversion – to the point of bringing the company down.
The affect of losses also has another undesirable effect. Since the impact of a loss is so high, it is easy to fall into the large project, fix-it-once-and-for-all mind set. This is something similar to a casino gambler placing everything on a single spin of the roulette wheel after a string of small losses. It usually leads to ruin.
An organization might be better equipped to deal with a loss if it kept a history of previous project outcomes. If projects typically succeeded in delivering what was expected, it might be reasonable to assume that a project that was experiencing problems might also turn out successful. In any case there should be a limit to cost and time defined in a project proposal – although few managers would actually do this. As a result we have the all too common phenomenon of projects running over time and budget by very large factors – possibly by an order of magnitude.
Loss aversion can and does lead to ruin, and IT investments are not different to any other kind of investment. Cases of ruin caused by IT investments that ran out of control are not usually reported, but I have seen this happen on two occasions. The remedy is simple – set a limit to the time, effort and costs you are prepared to bear.
At all levels of the organization, loss aversion will cause individuals to persist with an investment of time, effort or money, when other action might be more appropriate. Senior management will continue to pour money into a large application suite project and at the opposite extreme a programmer might persist with a poorly structured piece of code to achieve some nominal level of functionality. The cure in both cases is to define a level where activity stops and a fundamental rethink takes place.
Anchoring and Adjustment Bias
Anchoring is the process of fixing a rule of thumb measurement in our mind and then evaluating relevant activities by that measure. So it might be that IT investments in your organization must show a particular return before they can be approved. Let’s say that common wisdom suggests a return of 150% of the investment costs. What if a technology or system is capable of returning 1000% – would it be ridiculed or dismissed?
Senior management can have anchoring problems when it comes to IT investments. They might think of returns in the low teens for many activities. When a business intelligence investment is suggested their anchoring may well cause them to dramatically under-estimate the returns that might be generated on the investment (these could well very high). If this is the case a proposal might be rejected in preference to some other proposal that is better understood. I should add that successful business intelligence systems can transform an organization for a relatively modest cost. Jaded by their experience of the poor returns from large projects senior management may not grasp that something quite different is being proposed.
Anchoring is a dangerous habit where IT investment is concerned. It means that opportunities may be lost. It also means that projects are selected which comfortably fall within the anchored view of what an IT investment should return.
While overconfidence bias is an overestimation of an individual’s own capabilities, optimism bias is an unjustified belief that this time the correct system/supplier/methods have been selected. Optimism bias drives much of what happens in the world of IT investment – what is new is generally is believed to be superior to what has gone before. This bias affects people at all levels in the organization. Technicians are generally overoptimistic about new technologies and senior management can become overoptimistic about the outcome of a major IT investment.
Measurement is the key to a more realistic assessment of likely outcomes. If the organization has a history of success then the next project might also be a success. The more likely scenario however is that a history of problematical IT investment is forgotten whenever a new alternative presents itself.
This brief paper is not an exhaustive treatment of behavioral biases and how they affect IT investment. It does however provide an introduction to the type of issues that are addressed. I don’t expect that this will be particularly popular – putting a foot down on the accelerator is usually much more exciting than braking.
In my experience about one in ten organizations will have the will and the means to utilize some of this material. These methods are being used with great success in other areas of investment, and are particularly relevant as the financial services industry tries to climb from a disaster of its own making. We have to remember that global spend on IT is at around three trillion dollars. Despite the never ending claims by the suppliers, IT is a risky business. We need better methods to evaluate investment proposals.