Product Analytics

Before you start reading, I would advise you to read about Data Fallacies to avoid (1 minute infographic read).

[Wikipedia] List of cognitive biases: Avoid these to make informed decisions.

Types of Data Analytics

  1. Descriptive: What has happened

  2. Predictive: What might happen?

  3. Prescriptive: What should we do?

  1. Which metric to use and what data to capture, solely depends upon to goal to be accomplished

  2. The Guide to Product Analytics

  3. RICE (Reach-Impact-Confidence-Effort): Simple prioritization for product managers (download a sample from here)

  4. Top 10 Metrics Every Product Manager Should Know & Track

  5. 15 Key Product Management Metrics and KPIs

  6. How do you monitor performance and success?

  7. Prioritization and Stakeholder Management

  8. 16 Startup Metrics

  9. RICE score: A prioritization framework for estimating the value of ideas

A professional product manager will have a strong set of key performance indicators (KPIs) that they will monitor in order to understand their position, growth, progress and success. There are four key categories to listen for, including business metrics, product usage metrics, product development metrics, and product quality metrics. Listen for a solid selection of the following KPIs:

  1. Revenues or bookings: Top-line dollars that have committed to in conjunction with the sales team

  2. Funnel: Sales in process

  3. Retention, attrition, churn, customer lifetime value: Track the movement or flow of customers

  4. Customer counts: Current customer base

  5. Velocity, time to revenue, on-boarding times: Looking to accelerate revenue recognition and customer adoption

  6. Margins, gross margins, costs of goods sold (COGS), and operational costs of goods sold (OCOGS): Understanding the cost basis on which to calculate profitability

  7. Net promoter score (NPS) or customer satisfaction (CSAT): Subjective analysis of customer feedback

  8. Number of users per feature or transaction volumes: Can track feature importance for prioritizing sprints, and can highlight value for marketing or competitive positioning

  9. Time to execute: Records of time that functions take to perform, possibly indicating infrastructure or complex calculations which will result in customer complaints for poor performance

  10. On-time delivery: Managing the roadmap and creating credibility—the integrity of the team will depend on delivering as expected, on-time

  11. Team velocity: Monitoring team performance against sprint calculations using story points

  12. Resource availability: Monitoring critical resource availability and planning appropriately for coverage

  13. Support tickets and escalations: Monitoring the quality of the released product

  14. Testing or QA: Monitoring the quality of the code going into test

As a follow up, ask them what they did when they found a KPI that was not moving in the right direction. Listen for an action plan that would include common sense root cause analysis, and some creative thinking to solve for an unexpected KPI.

Product managers should also be using KPIs to plan for growth, perhaps in their NetOps environments or of on-boarding resources to meet more demand. Likewise, if the support and maintenance metrics are reducing, look for an adjustment of engineering resources as an appropriate data-driven management decision.

Reporting frameworks

  1. HEART

  2. PULSE

  3. AARRR