Track Adoption as a Critical Project Metric

Over the years I have worked hard to make sure that my teams are working on the right things. I have experimented with executive steering committees, non-executive steering committees, different cost / benefit models, and multiple levels of prioritization (portfolio, program and project). I have tried agile principles and shortened project feedback loops, cynicism, optimism, narrow voting, alignment around a few organizational themes (which assumes the organization can prioritize down to a few themes . . .), and networking with other IT leaders to find out what they do.

In doing all of this, I have learned three important things.

1. The right approach to ensuring that we are working on the right things varies by organization, and with time. Because priorities will change, we shouldn’t overinvest in any single process or the analysis of each project. Close enough is good enough.
   
   
2. Our processes need to be simple. The more complex the process, the less likely it is that we will consistently use it.
   
   
3. While it is important to track the status of each project, it is even more important to track the adoption of the sysem or software application all the way into the marketplace or into operational use.

Missed User Adoption Opportunities

About 18 months ago we overhauled the technology and process used to ingest our customer data. The old technology was fragile, required a lot of manual intervention and was no longer supported by the manufacturer. We worked closely with the team that supported customer data ingestion to make sure that the new system was a significant improvement. The project proceeded as planned, was completed on time and went live around nine months ago. In tracking the project, we marked it as “complete” and went on our merry way.

About six weeks ago in a casual conversation with the project leader, I asked if all was well with the new technology. The project leader told me that he did not know. Why? Because even though we were done with the project, no one was using the new tools.

Here is another example: It used to take too much time and effort to connect to our suppliers via our legacy EDI method. In fact, even someone on my team was dedicated to EDI connectivity full-time, there was a six-month backlog of EDI work.

Knowing that this was not sustainable, we implemented a new EDI gateway that utilized a lot more automation and standardization. We worked closely with our teams in supply chain and purchasing, we ran a number of pilot project with our suppliers to ensure that it all worked, and declared project victory.

About four weeks ago I was curious as to the magnitude of the benefits we were enjoying from our new EDI gateway. But I found out that only about 20% of our suppliers were using the gateway.

Why? Because once the new gateway went live, once our project was done, we stopped working with our suppliers to migrate them to EDI.

Measuring and Rewarding Adoption 

Now, knowing that we had this adoption gap, the next question was what to do about it. Whatever we were going to do had to be simple and transparent.

We added a new column of data to our project status reporting tool: Adoption Status. And for each project, we now set timelines and goals for adoption before the project commences.

We were able to turn our attention back to the client data ingestion project and partner with that team to define new adoption goals: 25% of customers will be using the new system in three months, 75% in six months and 100% in 9 months.

We also worked with our purchasing team to establish the goals for supplier EDI.

When we do our monthly project status update, we now look closely at adoption status (which means that our project status involves not just my IT teams). When we publish our status report it includes the adoption data so that the entire organization can see the goal, and our progress on it.

Even better, we have included adoption status in the bonus calculation for the entire company. If we reach our adoption goals it benefits everyone. And that makes perfect sense as it takes everyone to make sure that what we build gets fully used.