Beyond Benchmarks: Estimate Your Application Maintenance Costs Using Internal Data by Phil Murphy
EXECUTIVE SUMMARY
Applications professionals struggle to explain the cost of their applications' ongoing operations and maintenance to their business counterparts, who commonly perceive the costs as excessive. The business perceives these costs as excessive in part because of the way applications professionals deliver the information — as a large and indecipherable lump-sum budget item. Applications professionals who want to avoid blind mandates to cut costs by "X" percent must break out application costs in a way that business people can analyze and digest. When they do, they give the business an opportunity to make smart, surgical reductions to costs and shift underutilized resources to the work that is most important to the organization, increasing overall agility and innovation while reducing wasteful spending habits.
IN SEARCH OF "INDUSTRY STANDARD" BENCHMARK COMPARISON FIGURES
When business and IT executives push back on the costs of applications, they raise a common set of questions such as, "Are we spending too much?"; "What do other companies spend for the same services?"; and "How do we measure up against 'industry standard' measures and benchmarks?" The erroneous assumption is that "industry standards" exist for virtually every category of IT spending.
The term "industry standard" implies that an industry such as manufacturing or insurance agrees to use a certain body of specifications to ease interoperability. For example, engineers use standard measures (metric versus US measures) to enable interoperability, and PC makers publish their specifications so they can buy parts from many manufacturers and encourage competition among them, lowering costs. In the case of insurance, many firms use standard data formats such as ACORD for exchanging information. In these contexts, the term "industry standard" makes sense. However, when applied to IT spending and resource consumption, the term "industry standard" is a misnomer: It really refers to the availability of external benchmark data.
External Benchmarks Are A Useful Starting Point But Are Not The Final Destination
As CIOs and other executives ask tough questions about spending comparisons, external benchmark data can provide some answers such as hardware and software costs as a percentage of IT spending, IT spending as a percentage of revenue, etc. Comparing your firm's figures to the available benchmark figures can give you some broad insight into whether your figures are lower or higher; however, the figures vary greatly by company size and industry (see Figure 1).
In addition to variations by industry and company size, your firm's expense figures will vary from the benchmark figures based on several factors: business factors such your business drivers, profitability, and location; technology factors such as your application platforms, number and condition of applications, packaged/custom application mix, and programming languages; and people factors such as staff, skills, experience levels, labor costs, and educational levels.
If you can find benchmark data and relate it to your environment, the result is only an indication of whether your expenses are remotely similar to homogenized benchmarks. Is that answer helpful? It may be. However, some firms report that their ongoing operations and support costs exceed 85% to 90%. Should those firms be at 66%? Are they truly overspending by 20% or more?
Figure 1: Percentage Of IT Operating Budget Allocated For Ongoing Operations And Maintenance
________________________________________

DOWNLOAD THIS FIGURE: PowerPoint slides Excel spreadsheet
Whatever The Comparison Outcome, You Still Have To Come Back To Reality
With two-thirds of the IT budget spent on ongoing operations and maintenance, business people are already painfully aware that there aren't enough resources for new initiatives and projects. (see endnote 1) The relationship of your costs to benchmarks may be interesting, but critics aren't likely to be assuaged by assertions that costs are in line — they will continue to hammer away at application funding and productivity levels. So the quest for benchmark data is in some ways a diversion from the task at hand because whatever the comparisons tell you, you still have to explain the inexplicable costs of applications.
Instead of relying on industry benchmarks, focus your efforts on collecting data that will yield actionable answers such as where specifically costs are high, why they are high, and what actions will free resources for higher-priority work.
TAKE THE MYSTERY OUT OF HISTORY TO FORM THE BASIS FOR ACCURATE PROJECTIONS
Projecting future support costs is not an exact science, so remember throughout the process that you aren't striving for perfection; you're looking for a reasonably valid approximation technique to develop a baseline.
Develop A Set Of Baseline Figures For Your Applications
If you don't understand your current cost structure, it's unlikely that you can accurately project future costs. Look to maximize usage of the data you have available and extend it with related data and estimation: Develop a standard full-time equivalent (FTE) cost structure for IT roles. Work with finance to estimate an annual cost figure per FTE for each major IT role. Don't worry about productive versus nonproductive time such as vacation time, holidays, and time spent training; you can't hire and retain people without incurring those costs, so focus only on the annual costs. Look back a year to quantify FTE consumption by business unit and application. Since staff costs are the primary unknown in application support, chart allocation of your resources over the previous 12 months as a baseline. If you use time tracking, look to it for detail that you can aggregate to annual figures. Where you don't have time-tracking data, ask dedicated support staff members to approximate the time they spent during the past year on various applications. If support staff members are not dedicated — if they work on both new development and support simultaneously — approximate the new development/support split first, then approximate support levels across applications. Ask support staff to classify applications as small, medium, or large. To enable application comparison, classify each application by size; for example, in our environment, a small application on average takes between half an FTE and one FTE to support, whereas a large application will require between three and six FTEs. To the extent that it is possible, develop a similar scale for application complexity. Augment the FTE estimates with work-request volume and other indicators. Approximate the volume of support requests for each application area, ideally categorized by the requesting business unit or individual requestor. Request volume will permit you to state that a business unit or application consumed two and a half FTEs to satisfy 300 work requests, as requested by a specific person. Classify the requests as best you can as small, medium, or large to increase your ability to size future requests by comparison to these baselines; for example, historically, a large work request against this application (or one of this size and complexity) requires one and a quarter FTEs. Approximate hardware/software environmental costs using available data. Most operations organizations have good resource consumption data, often in more detail than you will need. In simple cases, divide the environmental costs of the server by the number of applications on the server. Adjust that initial estimate by application size (small, medium, or large), adding storage consumption costs, if possible, for more precision.
Use
The baseline of your existing application costs provides what the benchmark data could not: actual figures that directly relate to the costs to support real applications within your internal constraints. Because they reflect your actual costs, they provide a far more reliable basis for future projections, and they represent the baseline from which you will measure progress in terms of cost reduction, percent of the budget available for innovation, and other internal measures.
Perhaps more importantly, these baselines enable you to make application spending more transparent in a way that business people will understand and allow business executives to compare application spending within and across business units. (see endnote 2)
RECOMMENDATIONS
SPEND THE TIME AND EFFORT IT TAKES TO DEVELOP YOUR SUPPORT BASELINE
As a measure of application support costs, external benchmarks are merely interesting; they are not very actionable. Internal baseline data — even when approximated — provides a better measure of current and future support costs because it reflects all of your internal operating constraints. As you develop your baseline: Start with readily available or easily discernable data. Labor is the big unknown support cost for custom-built applications. What information do you have on hand about the labor? Begin with this information and extend it with indicators of volume, requestor, and the work-request catalyst — whether it was to fix a problem, make an enhancement, address a regulatory mandate, or address another issue. Avoid collecting pointless detail and creating a "big brother is watching" syndrome. Time tracking often fails because management requests overly specific information — to the hourly level. Asking staff to report every minute of the day for every day of the year is pointless tedium that will backfire. Think in terms of FTEs for the year, not individual tasks within a project. Use early results to build momentum for a continuous program. A lack of transparency in application spending breeds distrust; when we fail to supply business people with an understanding of where resources are spent and why, they assume that we have a slush fund of unspent resources or that we are wasteful with resources. Showing them a historical view of resource allocation strips away the mystery and begins to build trust. One of two things will result from transparency: either it will show that you have already allocated resources to the highest-priority work, or it will show that you have discovered some inappropriate spending patterns. In either case, the situation moves from blaming IT for high costs to a business discussion around the most appropriate use of limited resources. Make it somebody's job. Too many firms attack the internal baseline information issue as a one-time event rather than an ongoing management discipline. The need for this information never goes away. Make the collection of this information a routine and continuous event. Use it to align application efforts to business priorities, and make it someone's responsibility.
SUPPLEMENTAL MATERIAL
Methodology
Forrester's Enterprise And SMB Global IT Budgets And Spending Survey, Q2 2009, was fielded to 3,497 IT executives and technology decision-makers located enterprises with 1,000 or more employees in Australia/New Zealand, Brazil, China, India, Japan, Mexico, Russia, Singapore, South Africa, South Korea, and the United Arab Emirates, as well as from companies with two or more employees in Canada, France, Germany, the United Kingdom, and the United States. This survey is part of Forrester's suite of Business Data Services studies. Forrester fielded the survey from February 2009 to May 2009. Ex??vo fielded this survey via telephone on behalf of Forrester. Survey respondent incentives include a summary of the research reports. We have provided exact sample sizes in this report on a question-by-question basis.
Forrester's Business Data Services fields eight business-to-business technology studies in 19 countries each calendar year. For quality control, we carefully screen respondents according to job title and function. Business Data Services ensures that the final survey population contains only those with significant involvement in the planning, funding, and purchasing of IT products and services. Additionally, quotas are set for company size (number of employees) and industry as a means of controlling the data distribution and establishing alignment with IT spend calculated by Forrester analysts.
In addition to sampling error, one should bear in mind that the practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls. Other possible sources of error in polls are probably more serious than theoretical calculations of sampling error. These other potential sources of error include question wording, question ordering, and nonresponse. As with all survey research, it is impossible to quantify the errors that may result from these factors without an experimental control group, so we strongly caution against using the words "margin of error" in reporting any survey data.