OKRs, the future of product roadmaps

How and why we replaced product roadmaps with OKRs

Locaweb is Brazil’s leader in web hosting and SaaS applications like email marketing and online store. Locaweb hosts approximately 23% of all Brazilian domains. I work there since 2005 and I’m responsible for managing product strategy, roadmaps, product life cycle and product development.

Since 2012 we review product roadmaps every quarter. At the beginning of each quarter we look back on what was done in the previous quarter, which items were delivered, which weren’t and what were the reasons for success or failure. Then we plan what to do in the following quarter.

In the middle of 2014 I wrote an article on writing roadmaps including motivation behind each item in the roadmap and metrics that showed that the motivation was being fulfilled. It was the result of several conversations we had at Locaweb about having the roadmap as a list of items to make every quarter but not always having clear why we are doing each those planned items. Since that time we started to make our roadmaps with each item composed of three sub-items, what to do, why it had to be done and what metric we expected move when we do that item. However, while we try to make clear the motivation and the metric of each roadmap item, the discussions ended up mainly around the “what to do”.

In the first half of 2015 we heard about a framework called OKR, which means Objectives and Key Results. This framework is used by Google since its early days and was brought there by John Doerr, an Intel employee, where this framework was created. John Doerr, after leaving Intel, became an investor in technology businesses such as Google, Amazon, Intuit and Zynga and brought this framework to these companies. Several technology companies today use OKRs. A few more examples are LinkedIn, GoPro, Flipboard, Yahoo !, Amazon, Adobe, Baidu, Dropbox, Facebook, Netflix and Spotify.

OKR is a framework derived from a management technique called Management by Objectives, a term coined by Peter Drucker in his book The Practice of Management, 1954. Management by Objectives is a process that requires the identification and accurate description of goals to achieve and deadlines for completion and monitoring. This process requires that those involved agree with what you want to achieve and that all play their roles for the achievement of the objectives.

How does OKRs work?

There are several articles and videos that explain in detail how OKRs work, so I will make my explanation very brief. OKRs are composed of two parts, a goal (objective) and 2 to 5 key outcomes (key results) indicating that the goal was achieved. For example:

Objective: To have satisfied customers point to recommend our services to your friends

Key Result 1: Maintain 80% of satisfaction survey notes above 8 on a scale from 0 to 10.

Key Result 2: At least 50% of new sales should come from existing customers recommendations.

The goal does not necessarily need to have numbers. However, key results must always have numbers, i.e., must be some metric and must say where you are and where you want to be with regards to the metric, the goal we want to achieve with that metric.

It is recommended to have at least two key results. When there is only one key result we can suffer what is called “perverse effect”. For example, suppose the goal is to raise the productivity of telephone service team. Let’s say we define only one key result, the AHT (average handle time) which is now in 8 minutes and we want it to drop to 2 minutes. One way to achieve this key result is the attendant hang up the phone when close to 2 minutes. Of course this would be bad for the quality of service, but the key result and the goal would be achieved. In this case, to balance the “perverse effect” it is recommended another key result that ensures customer satisfaction to be maintained.

Implementing OKRs at Locaweb

After studying OKR for some time, we realized that it was very similar to what we always wanted to do, focusing more on the motivations and metrics than the “what to do” itself. The big difference is that in OKRs the “what to do” simply does not exist. It can be discussed when defining each goal and their key results, but “what to do” is not documented and, therefore, it is not committed and can be changed. What matters is the goal and key results that indicate that the goal has been reached.

To help us in this change, we brought Felipe Castro from Lean Performance, a company specialized in OKR implementation. We brought him in June 2015 and started implementation in the 3rd quarter of 2015 with a series of internal training on OKR, goal setting, metrics and objectives. In August we made our first training planning session to set OKRs for the month of September. It was a test for us to understand a little better the mechanics of OKR setting process. In late September we made our first planning session of a full quarter, where we defined the product development and marketing OKRs for the 4th quarter of 2015. In parallel, we continued with the quarterly product planning roadmap based on a list of items to do.

Each team updated their OKRs weekly. Furthermore, we followed the evolution of the roadmap monthly. We have seen throughout these follow ups that the roadmap items were the tasks that enabled the achievement of the objectives and key results, i.e., there were monitoring tools — OKRs and Roadmaps — for the same job. Throughout this monitoring sessions we had an epiphany: what if we abandon the traditional roadmap, the list of tasks to do, and we focus only on defining and tracking OKRs?

From ToDoers to needle managers

That’s what we did in the 1st quarter of 2016. The planning was done completely based on goals and metrics that we wanted to move, i.e., the needles that wanted to manage. We cease to be mere ToDoers, mere executors of tasks, to become needle managers. Given a goal and a metric that indicates that this objective is being achieved, we decide what to do. In the OKRs review meeting for the 1st quarter of 2016 and planning of the 2nd quarter, no one missed the old roadmap list of tasks to do. Obviously during the retrospective each team commented a bit about what they did to move the needles, but the “what to do” was just a means to achieve a goal, and not the goal itself. And of course in each OKR planning session the teams already have a sense of “what they will do” to achieve their goals, but they have autonomy to decide “what to do” as they please.

Lessons learned

We have learned some valuable lessons during this almost one year in which we are working with OKRs. We frequently review these lessons learned so that we can continuously improve our OKRs:

• More frequent deliveries and measurements help achieve goals. If deliveries and measurements occur monthly, the team will have only two chances during the quarter to test their idea of how to move the metric needle. If there are weekly deliveries and measurements, there are 12 opportunities to evaluate and change course if necessary.
• Technical objectives can and should enter if necessary. Some product development teams may have chosen to put on their roadmaps only tasks that deliver value to the customer and prefer not put technical stories as “upgrade version of the programming language” or “review architecture to make it more scalable. “ This was the case Locaweb. In making the transition roadmap to OKR, came to doubt whether it would make sense to define technical objectives, as technical tasks not entered the roadmaps. After some discussion, all agreed that yes, we can and must put more technical objectives in our OKRs. For example, having more robust and scalable infrastructure is a technical objective that can have metrics as MTBF, MTTR, SLA, queues, etc.
• Objectives and KRs can be modified or canceled as long as it is agreed with everybody. Sometimes, when a team starts working on some new metric that the team had never worked before, one can find that moving the metric needle is harder or easier than initially imagined. It is possible, in this case, to change the target for that key result as long as everybody agrees. It can also happen that a certain key result or even a certain objective becomes less important than something that comes after the OKRs planning session. It is also no problem removing the OKR to put a new one, provided that everybody involved in the OKR is in agreement.
• Beware of interdependencies. Tasks that can influence KRs and depend on other teams who have other priorities or even external suppliers have great risk of delay. The ideal is to do the task at the beginning of the quarter, or even in the previous quarter. In case of interdependence where a team needs to do A so that the other team can start to do B, you can put A in the first team OKRs in a quarter and the other team includes B as OKR in the following quarter.
• Beware of too generic metrics. This is one of our latest learnings. Metrics such as NPS and AHT (average handle time) are too generic. They can be impacted by a multitude of factors. For example, the AHT can be influenced by reasons as diverse as the slow performance of the customer care system, the product with a usability that leaves the customer confused, very high turn-over service employees and so on. In these cases it is recommended to turn the metric into an objective, for instance, “want to lower the AHT in 3 minutes” and setting specific KRs such as “decrease system load time from 30 seconds to 3 seconds”.
• Beware of excess of optimism! When you start working with OKRs and begin to set targets for the metrics, the team normally underestimate the effort required to move the needle. There is a natural tendency to set targets too optimistic and, in hindsight, realize that moving the needle is more difficult than initially imagined. In such cases, during the next OKRs planning session, it may be the opposite, that is, after the exaggerated optimism, the team may be extremely cautious and will end up setting too shy targets.

In conclusion

As I said above, in our OKRs review and planning sessions at Locaweb, no one missed the list of tasks to do done that we used to discuss in our old roadmap quarterly review and planning sessions. During the review each team says a little bit about what they did to try to move the metric needle, but the “what to do” are only a means to achieve a goal, and not the goal itself. And of course in each OKR planning session the teams already have a good sense of “what they will do” to achieve their goals, but they have autonomy to decide and change the “what to do” as they wish, provided they achieve their goals.

For this reason, it is increasingly clear to me that the OKRs are the future of the roadmaps. It is a tool that puts the goals and metrics in the center of the discussion, leaving the decision on “what to do” more fluid and flexible, giving teams more autonomy in deciding what to do.

Product Management: Delight your customers with your software

In 2015 I wrote a book on Software Product Management in Portuguese. In the beginning of 2016, Paulo Caroli talked to me about how he enjoyed the book and how this book could be useful to people in the software industry not only in Brazil but anywhere in the world. For this reason, we decided to create an English version of my book.

The book is organized in 5 sections:

• Definitions and requirements
• Life cycle of a software product
• Relationship with other areas
• Product portfolio management
• Where to use software product management

This book is suitable for anyone working with software. Even companies that do not have software as its core business use software in their day to day and often have developed some software that interfaces with its customers such as a website or a mobile application. It is important for these companies to understand the software product management role and responsibilities, so they can better manage this software and increase its chances of success.

We are working on the translation but as we progress we are already releasing the content. If you want to see the work in progress, please visit the book page at LeanPub. Still in beta but already with valuable content. Feedback is always welcome!