Measuring the Product Manager

I was speaking with a CTO the other day and he asked me what metrics I use to track the performance of Product Managers. I responded with the quantitative measures that I use, as well as some qualitative metrics. The discussion made me realize that this is not the easiest thing to articulate, and I feel like he was looking for more. But it's important to understand that measuring the performance of a Product Manager isn't easy, it's not black and white, and it's not the same as measuring the performance of a product.

Measuring the performance of a Product Manager isn't as simple as measuring a sales person. When it comes to a sales person, you know their sales volumes for the present period vs. past periods, their sales targets, how their performance compares to sales peers, etc. The performance of a sales person is so easily measured that they have actual numbers (targets) that they need to hit, tied directly to their compensation. But how do you measure the performance of a Product Manager whose entire job is based on creating a strategy, and the soft skills, such as influence, required to implement it?

An easy, and incorrect assumption, is to use the same metrics that are used to evaluate the performance of a product. Metrics such as MAU, CAC, MRR, LTV, ARPA/ARPU, churn, etc. are metrics that are of interest to, and should be closely monitored by the Product Manager. But the Product Manager is not responsible for writing code, creating marketing material, selling the product, responding to support requests, etc. These are all factors that affect the aforementioned product metrics, but they are not under the direct control of the Product Manager. Certainly, Product Managers are driven by, and fixate on these metrics, but there are many variables that can affect these metrics beyond the actions of the individual Product Manager. How then can their performance be measured by activities in which they have little to no control? The answer is simple, they can't be. The metrics used to assess the performance of a Product Manager must be based on activities that they (individually) are responsible for and have direct influence over. So what metrics does that leave us?

Qualitative Measures

There are certainly many qualitative metrics that can be used to evaluate a Product Manager. Some of these metrics include:

• Communication
• Leadership
• Attention to Detail
• Influence/Persuasion
• Customer Focused
• Clear/concise/logical

These qualitative measures are useful and form part of the equation in evaluating the performance of a Product Manager. This information should come from observation as well as soliciting input from other areas of the organization that interact with the Product Manager. These areas could be the Engineering team i.e., "How does the Product Manager perform during Sprint reviews? Are they regarded as an asset to the review or a liability?". From the Support team i.e., "Does the Product Manager answer questions in a timely and accurate manner? Does he/she have a reputation of being helpful to the Support team and customers?" Most departments in your organization will have some level of interaction with the Product Manager and are therefore good sources of qualitative information.

While the above metrics should play a role in determining the performance of a Product Manager, and me being a "data guy", I need quantifiable metrics.

Quantifiable Metrics

As mentioned previously, Product Managers are driven by product performance metrics, but Product Managers are only one variable influencing product performance outcome.

However, there are activities that help drive product performance which are 100% under the control of the Product Manager.

• Competitive intelligence & updates. (Used to inform the Product Roadmap. Updates to competitive matrices, strategy canvas updates, etc.)
• Customer visits & calls. (Important to ensure that products meet the needs of the customer.)
• SWOT analysis & updates.
• Regular updates to the product roadmap and communication to internal and external parties.
• Performing user experience analysis. (Conduct usability testing, user surveys, etc.)

These are all activities undertaken exclusively by the Product Manager with deliverables that can be easily measured.

Another set of quantifiable metrics focuses on requirements. Not only should you be measuring the delivery and quality of the requirements, but also measuring changes made to them, and to Sprint processes. Examples of this are:

• Requirements added, or changes made after the Sprint has been locked down.
• How many requirements are sent back to the Product Manager because they are incorrect or incomplete.
• Requirements added that are outside the scope of the deliverable.

A Product Manager has deadlines to deliver, and they've probably made commitments to do so. Timeliness is yet another metric that should be used to measure performance. If a Product Manager commits to a deadline, then he/she should be measured on their ability to meet that commitment.

In Closing

While I stated previously that a Product Manager should not be measured solely on product performance metrics, partially tying a Product Managers performance to metrics such as churn, MAU, CAC, LTV, etc. is a legitimate approach and should be included in their overall assessment. I know this sounds contradictory to my earlier statement, but after all, if the product isn't hitting corporate goals, then the Product Manager should share some of that responsibility, and this should be reflected in a review of their performance. But again, these should never be the only measurements used to evaluate a Product Managers performance.

Each quarter, I set my Product teams OKRs and I am always sure to include some qualitative, some quantitative, and some product performance metrics to their objectives. I attempt to balance across these three types of metrics because if you focus solely on numbers, then your Product Managers will tend to do the same and forego other important tasks such as planning, spending time with customers, communicating, collaborating, etc. This is especially true if the Product Managers compensation is tied to his/her ability to hit these numbers.

One final note on OKRs. I always get the buy in of the Product Manager to his/her OKRs prior to locking them in for the quarter. Having this conversation with the Product Manager ensures that they fully understand the OKR, allows him/her to raise any concerns that they have in their ability to meet the objectives, and most importantly, commit to meeting them.

I'm always interested in learning more about the processes implemented by other Product Managers and Product teams. Feel free to add any additional metrics that you measure your Product team against to the comments section below.

As always, thanks for reading.

All the best

Mike

LTV = (Customer) Lifetime Value

CAC = Customer Acquisition Cost

ARPA/ARPU = Average Revenue Per Account / Average Revenue Per User

MAU = Monthly Active Users

MRR = Monthly Recurring Revenue

OKR = Objectives and Key Results