A subscription business sees more cancellations and considers offering every customer a discount. Before committing to the cost, it needs to establish what changed. More cancellations could reflect a larger customer base, a problem concentrated in one plan, different renewal timing, or a genuine decline in retention. Each explanation points towards different work.
A data analysis strategy connects that decision with the evidence needed to assess it. It establishes the population, measures, sources, methods, and limits of interpretation before a set of charts becomes a recommendation. The process can remain exploratory where the question is uncertain, while keeping a clear distinction between discovering a pattern and demonstrating that a proposed intervention will help.
Specify the Decision and the Competing Explanations
The initial question might be whether to offer a discount, revise onboarding, investigate service failures, or gather more information. Identify who can make that decision and what evidence would alter their choice. An analysis with no available action can still support research, but it should not promise an immediate operational result.
Write down plausible explanations before selecting variables. For cancellations, these might include pricing, product reliability, customer mix, billing errors, and contract changes. This makes it easier to notice missing information and reduces the temptation to build a story around whichever relationship happens to look strongest.
Define the outcome precisely. A cancellation request, a subscription ending, a failed payment, and a customer reducing their plan are different events. Decide whether the question concerns accounts, subscriptions, recurring revenue, or another unit. A customer with several subscriptions can produce different results under each definition.
Record the comparison period and the purpose of the measure. A monthly cancellation rate based on customers active at the start of the month can be useful, but it needs rules for new customers, reactivations, and multiple cancellations. State those rules rather than relying on an apparently familiar label such as “churn.”
Check the Denominator Before Explaining the Trend
Consider an illustrative measure counting cancellations among accounts active at the beginning of each month. In one month, five of 100 eligible accounts cancel. In the next, eight of 200 cancel. The cancellation count rises from five to eight, while the defined rate falls from 5% to 4%.
Neither result is contradictory. The two measures answer different questions. The count can inform support workload; the rate describes the proportion of the defined population cancelling. A presentation that shows only rising counts could lead the business towards a retention response unsupported by that rate.
| Period | Eligible opening accounts | Cancellations from that population | Defined cancellation rate |
|---|---|---|---|
| First month | 100 | 5 | 5% |
| Second month | 200 | 8 | 4% |
Check whether changes in the population explain the pattern. Newer customers may behave differently from established customers, and different plans may renew on different schedules. Aggregate comparisons can mix those effects. Breakdowns should follow the decision and available evidence rather than generate an unmanageable number of charts.
This is also where classification quality matters. If plan names or customer segments changed between periods, the groups may not be comparable. The business-classification article explains why category definitions and effective periods belong alongside the assigned labels.
Establish What Each Row and Join Represents
Identify the grain of every source. An account table may contain one row per account, an invoice table one row per invoice, and a support table one row per interaction. Joining invoices directly to support interactions can multiply rows and inflate totals when both have several records for one account.
Prepare sources at the required grain before combining them, and verify join coverage and cardinality. Reconcile counts and monetary totals before and after important joins. Investigate unmatched records rather than silently dropping them, particularly if the missing population could differ from the matched one.
Inspect time and meaning as well as format. A cancellation timestamp may record when a request was received or when access ended. Currency, time zone, billing period, and account status can affect interpretation. Standardising text case alone will not resolve these semantic differences.
Keep the original evidence and make transformations explicit. A zero can be a legitimate value, an unknown value represented incorrectly, or the result of an earlier calculation. Replacing every zero with a mean would hide that distinction. Cleaning should follow documented assumptions, with sensitivity checks where several interpretations are plausible.
Choose Methods for the Question Being Asked
Descriptive analysis establishes what happened using counts, rates, distributions, and comparisons. Diagnostic work examines possible explanations. Predictive analysis estimates future or otherwise unknown outcomes. Prescriptive work evaluates actions under stated objectives and constraints. These are useful purposes, rather than mandatory stages every project must complete.
For the subscription business, cohort analysis can compare customers with similar starting events at comparable ages. Only compare six-month retention where the cohorts have had sufficient follow-up, or use a method that handles incomplete observation appropriately. Treating a recent cohort as though every account had six months to cancel creates an unfair comparison.
Regression can describe associations or support prediction while accounting for variables included in the model. Its interpretation still depends on design, assumptions, measurement, and relevant omitted factors. A coefficient associated with support contacts does not by itself show that reducing support access will improve retention.
Clustering can identify groups similar under selected features and a chosen method. It does not reveal a uniquely correct set of customer types. Review whether the groups are stable enough and meaningful for an available action. For forecasting questions, respect information timing and temporal validation as described in the predictive analytics guide.
Separate Prediction From the Effect of an Intervention
Customers with more support contacts may cancel more often because they experienced product failures. They might also be more engaged, have larger accounts, or have different contracts. The association can help identify a group to investigate without demonstrating that changing contact frequency will change cancellations.
The decision to offer a discount asks a causal question: what would happen with the offer compared with a suitable alternative? A churn model identifies likely cancellations; it does not automatically identify who will respond to a discount. Some customers predicted to leave may be unaffected by the proposed offer.
A well-designed randomised evaluation can strengthen evidence about an intervention when feasible and appropriate. Define assignment, outcome, follow-up, and relevant safeguards before examining results. If customers are assigned an invitation, comparing only those who accept it with those who do not can lose the protection provided by random assignment.
Observational analysis can also inform decisions, but the causal claim needs a defensible design and explicit assumptions. A before-and-after change may coincide with seasonality, a different customer mix, or other operational changes. Where those explanations cannot be separated, say what remains uncertain and which further evidence would help.
Report Magnitude and Uncertainty Together
Present the size of the effect in units relevant to the business. Moving a rate from 4% to 5% is an increase of one percentage point and a relative increase of 25%. Both descriptions are mathematically valid, but they create different impressions if their basis is not stated.
Statistical significance does not establish commercial importance, and a result that fails a threshold does not prove that the effect is zero. The American Statistical Association’s statement on p-values emphasises that a p-value does not measure the probability that a hypothesis is true or the importance of a result. Avoid reducing the decision to whether a value falls just above or below a familiar cutoff.
Use appropriate intervals or other uncertainty summaries, with their assumptions explained at the level the audience needs. Small groups and rare outcomes can support only limited conclusions. Searching many segments or alternative specifications can also produce striking patterns by chance; label exploratory findings and evaluate them with further evidence where necessary.
Assess practical sensitivity. Does the recommendation change if uncertain cancellation dates move between months, unmatched accounts are included under another plausible assumption, or unusually large customers are considered separately? A finding that survives relevant checks is easier to assess than a single preferred model output with no account of its dependencies.
Make the Analysis Reproducible and Useful
Preserve the source snapshot or extraction boundary, transformation code, definitions, exclusions, and analysis version needed to reproduce the result. The W3C’s PROV data model provides a formal vocabulary for relating entities, activities, and responsible agents. A team need not implement that standard in full to maintain an understandable record of how a finding was produced.
Reproducibility also has privacy and retention limits. Keeping an unrestricted customer export indefinitely is not the only way to document an analysis. Decide which source records may be retained, who can access them, and how long the evidence is needed. Versioned extraction rules and aggregate checks can support an audit trail, but be explicit when expired or deleted source data means exact reproduction is no longer possible.
Communicate the question, main finding, evidence, limitations, and proposed next action. For the subscription business, the result might be that increased cancellations reflect growth, while one mature customer group merits further investigation. That supports a narrower response than a company-wide discount without pretending the analysis established every cause.
The production data-strategy article addresses the service needed when this analysis becomes recurring. Definitions, freshness, corrections, and ownership then require continuing maintenance. A useful analysis strategy makes those requirements visible early and leaves the decision maker with evidence they can challenge, apply, and revisit.
