Product Analytics
Before you start reading, I would advise you to read about Data Fallacies to avoid (1 minute infographic read).
Types of Data Analytics
Descriptive: What has happened
Predictive: What might happen?
Prescriptive: What should we do?
Which metric to use and what data to capture, solely depends upon to goal to be accomplished
RICE (Reach-Impact-Confidence-Effort): Simple prioritization for product managers (download a sample from here)
How do you monitor performance and success?
RICE score: A prioritization framework for estimating the value of ideas
A professional product manager will have a strong set of key performance indicators (KPIs) that they will monitor in order to understand their position, growth, progress and success. There are four key categories to listen for, including business metrics, product usage metrics, product development metrics, and product quality metrics. Listen for a solid selection of the following KPIs:
Revenues or bookings: Top-line dollars that have committed to in conjunction with the sales team
Funnel: Sales in process
Retention, attrition, churn, customer lifetime value: Track the movement or flow of customers
Customer counts: Current customer base
Velocity, time to revenue, on-boarding times: Looking to accelerate revenue recognition and customer adoption
Margins, gross margins, costs of goods sold (COGS), and operational costs of goods sold (OCOGS): Understanding the cost basis on which to calculate profitability
Net promoter score (NPS) or customer satisfaction (CSAT): Subjective analysis of customer feedback
Number of users per feature or transaction volumes: Can track feature importance for prioritizing sprints, and can highlight value for marketing or competitive positioning
Time to execute: Records of time that functions take to perform, possibly indicating infrastructure or complex calculations which will result in customer complaints for poor performance
On-time delivery: Managing the roadmap and creating credibility—the integrity of the team will depend on delivering as expected, on-time
Team velocity: Monitoring team performance against sprint calculations using story points
Resource availability: Monitoring critical resource availability and planning appropriately for coverage
Support tickets and escalations: Monitoring the quality of the released product
Testing or QA: Monitoring the quality of the code going into test
As a follow up, ask them what they did when they found a KPI that was not moving in the right direction. Listen for an action plan that would include common sense root cause analysis, and some creative thinking to solve for an unexpected KPI.
Product managers should also be using KPIs to plan for growth, perhaps in their NetOps environments or of on-boarding resources to meet more demand. Likewise, if the support and maintenance metrics are reducing, look for an adjustment of engineering resources as an appropriate data-driven management decision.
Reporting frameworks