“DAU declined” can refer to users who logged in, opened an app, or completed a valuable action. If definitions differ, teams can use the same label while discussing different facts.
A metric system is more than a metric tree. A durable system also needs a dictionary, ownership, query versions, quality rules, and a change process. This guide provides a template you can copy into a document or data catalog.
1. Start with a Decision, Not a Dashboard
Every metric should support a decision:
| Decision | Primary metric | Diagnostics | Guardrails |
|---|---|---|---|
| Expand onboarding rollout? | Activation | Step conversion, time to value | Complaints, uninstall, support contact |
| Continue a channel? | Incremental payers or revenue | Arrival, signup, first purchase | Fraud, refund, acquisition cost |
| Change content supply? | Users receiving valuable consumption | Impression, click, completion | Negative feedback, creator concentration |
If no action changes when a metric changes, it may be context rather than a core dashboard metric.
2. Copyable Metric Dictionary
Record at least:
| Field | What to define | Example |
|---|---|---|
| metric_id | Stable machine-readable identifier | activation_rate_v1 |
| Display name | Human label, not unique key | New-user activation |
| Business question | Decision this supports | Did a new user receive first value? |
| Definition | One-sentence population and behavior | Core action within seven days of signup |
| Numerator | Event, filters, and deduplication | Unique new users completing core action |
| Denominator | Eligible population and exclusions | Valid completed registrations |
| Window | Calendar, rolling, or cohort | Seven days from registration |
| Time zone | Business reporting zone | Asia/Shanghai |
| Identity | User, device, order, or task key | canonical_user_id |
| Source | Table, event, and query version | mart_user_activation / query v3 |
| Freshness | Availability and delay | Daily 09:00, T+1 |
| Dimensions | Approved breakdowns | Channel, platform, signup version |
| Owners | Business and data owners | Growth lead / analyst |
| Quality rules | Freshness, uniqueness, ranges | Denominator nonnegative |
| Limitations | What it cannot answer | Anonymous cross-device activity not merged |
| Version | Effective date and change history | v1.2, 2026-08-13 |
Blank template
metric_id: display_name: business_question: definition: numerator: denominator: exclusions: window_and_timezone: identity_key: source_and_query_version: refresh_and_availability: approved_dimensions: business_owner: data_owner: quality_rules: alert_and_response: known_limitations: version_and_effective_date: change_log:
3. Connect Goals and Drivers with a Metric Tree
Write a computable relationship, then identify outcomes, drivers, and guardrails:
Paid orders = arriving users × arrival-to-order conversion × order-to-payment conversion
When decomposing arriving users, keep branches non-overlapping:
Arriving users = attributed organic users + attributed paid users + attributed other users
If one person can enter through multiple channels, direct addition duplicates them. Define attribution first or express the formula as contribution allocated by a named attribution model.
Do not mix these metric types
- Outcome: the user or business result, such as completed tasks, valid orders, or retention.
- Process: a step that drives the outcome, such as arrival, attempt, and completion.
- Guardrail: what cannot be damaged, such as refunds, complaints, privacy, or cost.
DAU, GMV, and click-through rate are not inherently north stars. Suitability depends on user value, product stage, and actionability.
4. Six Definitions That Commonly Conflict
Active user
App open and meaningful action answer different questions. Keep app_open_dau and core_action_dau distinct instead of sharing an ambiguous dau.
New user
First install, first registration, first verification, or first purchase? Define reinstall, cross-device, and merged-account behavior.
Conversion
Numerator and denominator must belong to one traceable population. Payers divided by all site visitors is not checkout payment success.
Retention
Define cohort start, return behavior, observation day, and window. Calendar next-day, 24-hour, and rolling seven-day retention are not directly interchangeable.
Revenue and GMV
Paid amount, fulfilled amount, net of refund, and recognized revenue answer different questions. Define currency, tax, discount, and cancellation handling.
ROI
State whether revenue is total or incremental and whether cost includes discount, channel rebate, labor, and fixed cost. Without a control or causal design, do not attribute all period growth to the campaign.
5. Worked Definition: Seven-Day Activation
This is instructional, not a real business definition.
metric_id: new_user_activation_7d_v1 definition: Valid registrants completing a first core task within 7×24 hours numerator: Unique denominator users with core_task_completed denominator: Completed registration, passed anti-fraud, not internal window: 168 hours from registered_at timezone: UTC storage, Asia/Shanghai display identity: merged canonical_user_id dimensions: signup channel, platform, signup version refresh: daily T+1; recent cohorts marked provisional guardrails: task cancellation, support contact, fraud block limitation: anonymous cross-device behavior is not merged
Why immature cohorts need a label
Users who register today have not had a full seven-day observation window. Including them directly depresses the metric. Hide immature cohorts or label them “observation incomplete”; do not compare them with mature cohorts.
6. Data-Quality Rules
Add automatic or manual checks for:
- Freshness: data arrives before its promised time.
- Completeness: partitions, events, and required fields are present.
- Uniqueness: primary keys and business events are not duplicated.
- Validity: amounts, timestamps, and enums stay in valid ranges.
- Consistency: detail, aggregate, ledger, or independent sources reconcile.
- Continuity: schema, instrumentation version, and filters are tracked.
- Traceability: a dashboard points to query, table, and owner.
Calibrate thresholds with historical replay and false-positive cost instead of copying “alert at 10%.” When an alert fires, use the operations anomaly diagnosis guide to validate definition and pipeline before business attribution.
7. A Metric Change Is Not Just a SQL Edit
For any change affecting trend comparability:
- Submit reason, affected pages, and consuming teams.
- Obtain business-owner and data-owner review.
- Backfill old and new definitions over a comparable period.
- Mark the effective date and discontinuity on dashboards.
- Notify downstream reports, experiments, alerts, and goals.
- Retain the old query and change record.
- Retire the old metric instead of keeping two same-name versions forever.
Create a new metric_id for a new business concept. For a query bug fix, record affected dates and backfill.
8. Dashboard Contract
A dashboard module should show:
- Metric name with a definition link.
- Latest refresh and data health.
- Current value, comparable baseline, and change.
- Numerator, denominator, and sample size when needed.
- Allowed dimensions and small-sample policy.
- Incident owner and response guide.
- Version-change annotations.
Do not circulate screenshots without definitions. Preserve metric version, filters, date range, and time zone in an export.
9. Governance Cadence
Weekly
- Resolve data-quality alerts and definition disputes.
- Check that critical dashboards refreshed on time.
- Put new agreements in the dictionary instead of leaving them in chat.
Monthly
- Review whether each metric still supports a decision.
- Find unused, duplicate, or unexplained metrics.
- Review alert precision and response time.
Quarterly or after major business change
- Revisit metric tree and product goals.
- Retire old metrics and permissions.
- Verify lineage, ownership, and incident response.
10. Release Checklist
- Every metric has a stable ID.
- Numerator, denominator, exclusions, identity, and window are executable.
- Outcome, process, and guardrail metrics are distinct.
- Source, query version, and freshness are traceable.
- Immature cohorts and small samples are labeled.
- Business and data owners approve the definition.
- Changes have versions, backfill comparison, and notification.
- Every dashboard alert maps to an action.
Continue with the SQL practice dataset, data operations role guide, SQL interview questions, or the data operations topic.