A Fresh Approach for Measuring the Value of Technology Investments

There is a famous quote (erroneously attributed to Einstein) that reminds me of the challenges IT executives face: “Not everything that counts can be counted, and not everything that can be counted counts.”

What should CIOs measure? Which measurements actually count and are meaningful to other executives and the board? On top of that, the current economic climate puts further pressure on IT executives to cut costs, optimize the usage of resources, and demonstrate value from technology investments.

IT value demonstration and measurement started with the general IT portfolio management process best detailed in the 2005 book IT Portfolio Management, by Bryan Maizlish and Robert Handler. This book provides tools and techniques for effectively managing an IT portfolio using meaningful KPIs. More recently, strategic portfolio management has been embraced, as covered in Strategic IT Portfolio Management. This 2018 book by Jeffrey Kaplan is about how to create governance for strategic IT portfolio management that focuses on transformation. But with the pressure to cut costs and optimize resource utilization, these traditional approaches to IT portfolio management may no longer be sufficient for measuring the business value of IT investments.

A New IT Value Measurement Framework

A Value Realization Framework (VRF) and governance are needed to drive the evaluation of investment ideas, projects, and initiatives to align them with business outcomes. While this framework uses some familiar KPIs, tools, and techniques, it goes beyond them. The objective of VRF is to measure specific business value when IT investments are considered across their full lifecycle.

In the process of creating your VRF, it is important to work backwards from well-defined business outcomes, to defining the business strategy, objectives, goals, what value is, and how to measure it. This is the foundation for VRF. Without it, organizations cannot agree on what value is and how to measure it.

VRF is about the creation of a mechanism or a methodology that enables the measurement of the business value of products, solutions, or services from inception to consumption by customers or end-users. It is about measuring how these products or services contribute to the organizational goals and objectives over their lifecycles.

This diagram depicts the high-level processes of VRF.

VRF Governance Framework

The VRF model helps guide the development of a governance structure with well-defined roles and responsibilities to answer the questions:

• How will business value be measured and monitored, and by whom?
• To whom will the results be communicated, and on what schedule?

Successful deployment of a VRF will require software tools to enable it. Many such tools are available as cloud-based products that can be deployed quickly.

Once the VRF foundation is built, any trigger for innovative ideas or demand for projects, products, solutions, or services will go through the evaluation funnel and a decision-making process on whether such ideas or projects should be funded, and by how much. This is when the analyses and evaluations for demands are conducted using tools such as business case, cost/benefit analysis, ROI, cost savings, revenue, customer conversion rates, customer engagement, risk mitigation, operational resiliency, compliance, productivity, business agility, and others KPIs and techniques. This is also when the idea is tested on whether it will deliver the desired business outcome through value definition and the KPIs that will be used to measure the desired value.

In addition, the success and failure criteria are defined to drive the post implementation assessment which will determine whether the product or service achieved the desired value or not. It is critical to define the KPI ranges that are considered successes as well as the ranges that indicate failure.

If the demand passes this evaluation gate, it moves on to the implementation and the delivery phase and becomes a project. The project lifecycle kicks in, the delivery and project execution mechanisms unfold, and project delivery KPIs, milestones, and deliverables feed into the overall value realization framework.

Once the project is completed, the decision is made whether to go live with the product, solution, or service. This is the time to measure the project delivery performance and KPIs such as time to market (TTM), budget, and quality, and report them to stakeholders. This helps assess delivery performance, learn from, and improve the project delivery processes and lifecycle.

Once an organization goes live with a product, it becomes an asset, and it moves into the operational phase. The operations of assets include monitoring for performance and value using business and technical KPIs. These KPIs are reviewed on a regular basis to show trends over time. Then, the value of the product or asset is assessed to determine to what degree it meets the business objectives and delivers the desired business outcomes that were defined in the earlier stages, and whether the KPIs fall within what was defined as success or failure.

Advice for Adopting VRF

Some organizations might benefit from a Value Realization Office (VRO) that can help in implementing the above VRF processes and governance. A VRO can help create consistent, repeatable, and reliable ways of measuring business value using data and well-defined KPIs. It can also create an organization wide VRF dashboard to communicate value and progress toward the organizational goals.

To help create VRF governance, Gartner proposes these 7 rules for helping to demonstrate the business value of IT. While all seven rules are useful in shaping a VRF governance, two of the rules stand out.

According to Gartner’s rule number 3, “Build two value narratives, for operating and transforming,” IT leaders should separate how to measure value for ‘run-the-business’ from the value of ‘transforming or growing the business.’ The two value perspectives are different and require a different set of metrics and approaches.

Transforming and growing the business requires measuring, monitoring trends, and communicating KPIs related to ROI along with other KPIs to help refine and improve the asset and to further optimize the ROI. Measuring and reviewing the ROI trends, conversation rates, productivity and other KPIs shed light on the future and features of the product and what steps should be taken to achieve the promised value. If the value is not achieved, then cutting the losses and dropping the product might be the right thing to do.

Gartner’s rule number 6 recommends that you “Communicate IT value in the language of the stakeholder.” The language is different for the different stakeholders especially when they are on the financial or the value stream (business) processes sides.

Developing and deploying VRF governance doesn’t have to be complex. Taking an agile approach to deploying VRF might pay off quickly and efficiently. It can be pragmatic to create a kernel and agile VRF with a small set of KPIs, success criteria, and well-defined business values along with clear roles and responsibilities. This VRF model can be tested on a small number of projects and improved through iteration. This approach will enable organizations to learn and improve the VRF processes and to achieve results quicker.

VRF governance can be critical in helping define what to count and how to measure value from IT investments, and which investments count in meeting the overall organizational outcomes.