Measuring R&D performance with Innovation Accounting
If you read Lean Startup by Eric Ries, one topic you’ll find difficult to comprehend is Innovation Accounting. The reason for the incomprehensibility and importance of the term is explained by Eric himself in one of his answers on Quora:
I don’t find this especially surprising. People love the parts of Lean Startup that can fit on a bumper sticker or t-shirt. Slogans like “get out of the building” or “minimum viable product” are way more fun than the dry drudgery of accounting. And yet, there’s absolutely no way for us to make the massive changes as a movement that we are striving for if we don’t make reforms to accounting and the accountability systems it drives in most corporations.
The fundamental problem is: during the long, flat part of the hockey stick, how do we know if we are making progress? How do we keep from deluding ourselves? And how to convince key allies – like our funders – that we are moving in the right direction?
Simply put, innovation accounting helps in:
- Finding and assigning metrics of growth
- Measuring progress without using revenue as KPI
- Making sure you are on the right track
At that time, I wondered if an Enterprise can implement this concept. And my curiosity led me to an article by Dan. His article explained how an organization can use the concept. You can read his take by clicking here, which I’ve concisely compiled below.
As per Dan, one mistake most organizations make is that they don’t separate the innovation department’s KPI from their core business’s KPIs. Rather than the regular KPIs,, which are detrimental to individual motivation and the overall innovation goal of an R&D team, organizations can follow the three KPIs below-
- Reporting KPIs
- Governance KPIs
- Global KPIs
Let’s discuss these KPIs in detail.
It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.
— Mark Twain
Reporting KPIs are focused on checking the progress made from an ideation stage to prototype and finally to the product-market fit. Under these KPIs, metrics like the number of hypotheses validated/invalidated, the number of experiments conducted, the number of prototypes built, the number of minimum viable products (MVPs), etc., are considered.
Dan further suggests that some of these KPIs could be vanity metrics and may not provide any actionable insights. Hence, he offers using actionable KPIs like Cost Per Learning, Time-Cost Per learning, and Validation Velocity under Reporting KPIs. Below is a brief overview of why to use these KPIs:
Cost Per Learning
Cost per learning is a better KPI than the number of MVPs or prototypes built. Because focusing on cost per learning will inspire the R&D team to conduct more experiments. For example, if you allocate a budget of $100k to a team and they build one prototype which fails, their cost per learning would be $100k. However, under the number of prototypes built/failed, a team may look good as they failed at only one prototype.
The number of MVPs/Prototypes KPI discourages a team from creating multiple prototypes as their performance sheet looks terrible when there are ten failed products. On the other hand, if the cost per learning were a KPI, a team would be more inclined to build more prototypes. And with each prototype, say if 10 got to build on the same budget of $100k, the cost per learning would be less, and a subsequent prototype would be better than the last one.
Time-Cost Per Learning
This KPI focuses on how fast a learning cycle is completed and is directly related to the KPI above. If a Time-Cost per Learning is small, moving from ideation to the final product stage will be quicker.
It is the KPI that measures the speed with which assumptions and hypotheses were validated/invalidated while moving from ideation to prototype and finally to product-market fit.
Governance KPIs are instrumental in selecting new ideas for funding or continuing fund supplies to already funded ideas. These KPIs also are conducive to the idea of moving from the ideation to the market-fit stage than revenue. Depending upon different stages, the below KPIs could be used:
This is the stage where an organization makes funding decisions when a team displays that they have located a problemand a solution withs a considerable market.
Product Market Fit
Under this KPI, a team has to prove it has built a prototype with the right product-market fit and has a sustainable business model. If a team proves so, the management makes further funding decisions.
Knowledge to Assumption Ratio
Knowledge to Assumption ratio is a governance KPI that gives intel on how close a team the company is about to fund is to having a product market fit. The closer a team’s knowledge-to-assumption ratio is to 1, the higher its chances of having a product market fit.
Barebones Net Present Value (NPV) is a concept given by Prof. Rita McGrath which Dan describes as an art of pulling the plug. This KPI gives you a few parameters on the basis of which you can make a funding decision. You can explore the Barebones NPV in detail from these links: Link1, Link2.
Reporting and governance KPIs alone are not enough for Corporate C-Suite to evaluate the performance of the innovation department, writes Dan. Global KPIs with a focus on ROI help C-suite to find whether their innovation department is successful or not.
These KPIs stem from the idea that the overall investment made in the innovation dept. should lead to an increase in the wealth of an organization. The following metrics fall under the global KPIs:
This KPI measures the profit percentage contribution of the products launched by the innovation team in the overall company revenue.
Under this KPI, the innovation dept of an organization should be performing better than the cohort from the previous quarter.
This KPI measures the number of old customers adopting new products. These new products could or couldn’t be a replacement for an old product by the same organization.
Companies should realize that when innovating, they are involved in searching rather than working on an acquainted business model. Hence the KPIs will be different here. Teams must pay attention to the most consequential assumptions they’ve made about their project. The KPIs under innovation accounting can help them with it. These KPIs will enable the exact comparisons between two or more enterprises to evaluate which is most worthy of continuing investment. Innovation accounting provides a way to see an innovation project as a monetary instrument with precise value and reflects a range of future costs and financial outcomes over time.