Does an NPS Increase of 2 Points Mean Improvement? First Consider Category Counts, Sample Composition, and Error

A software company received 1,000 responses in each of two quarters, with NPS rising from 9.6 to 11.0. The dashboard showed a green upward arrow, and the team was ready to declare improved customer loyalty; however, the approximate 95% interval for the difference between the two periods was about -6.0 to 8.8, allowing for either a decline or a more noticeable rise. More critically, the share of new customers increased from 30% to 50%, meaning the customer structure being compared had already changed.

NPS is best used as a thermometer of recommendation intent under a fixed definition, not as automatic proof of growth, retention, or customer experience improvement. A credible interpretation should at minimum present the raw counts of promoters, passives, and detractors; clarify who was invited, who responded, and at what point in time; provide uncertainty for the score and its change; and use specific experience and behavioral evidence to guide action. Displaying only a single integer leaves the team unable to even discern how it was formed.

How NPS Is Calculated: 7 Is Not Zero, and Passives Do Not Disappear from the Denominator

Bain's definition of the Net Promoter Score is: on a 0-to-10 likelihood-to-recommend question, 9–10 are promoters, 7–8 are passives, and 0–6 are detractors; NPS equals the percentage of promoters minus the percentage of detractors. Thus, the result ranges from -100 to 100, not from 0 to 10.

Suppose there are 400 promoters, 300 passives, and 300 detractors:

NPS = (400 ÷ 1000 - 300 ÷ 1000) × 100 = 10

Passives contribute zero to the subtraction but remain in the denominator of 1,000. If one incorrectly calculates using only the 700 promoters and detractors, the result would be 14.3, which is no longer the standard NPS. Nor can one average the raw 0–10 scores and then convert that average to an NPS; two batches of customers with the same average can have completely different distributions across the three categories.

When reporting, it is recommended to retain the counts and proportions of the three categories, the NPS, and the number of non-responses. The integer NPS can be rounded for presentation, but analysis data should retain sufficient precision. Net Promoter, NPS, and related terms are registered trademarks of their respective owners; this article discusses the research design and statistical interpretation of this metric.

The Object of Recommendation and the Boundaries of Experience Must Be Answerable by Respondents

“How likely are you to recommend us?” may seem uniform, but across different products it may not carry the same meaning. Do enterprise administrators recommend the purchase decision, daily usage, or the vendor relationship? In sensitive contexts such as healthcare, finance, or job seeking, satisfied users may still be unwilling to recommend to friends. In multi-buyer B2B software, the user may not have selection authority yet is asked to speak for the organization.

Before launching a survey, four boundaries should be fixed:

• Object of evaluation: The company, product, a specific service, or a particular interaction, and it must not switch between quarters;
• Object of recommendation: Friends, colleagues, or people facing the same task, and it should align with the real-world referral context;
• Experience qualification: Respondents should have sufficient and recent exposure; newly registered users who have not completed any task generally cannot judge;
• Measurement timing: Relational tracking and post-interaction feedback after ticket closure or transaction completion should be labeled and analyzed separately.

Once the definition is established, do not quietly change anchors, ordering, or trigger channels to make the question “friendlier.” Necessary modifications should retain versions, use old and new wording randomly during a bridging period, and assess whether changes arise from customers or the measurement instrument.

Why a 1.4-Point Rise Still Does Not Allow Conclusions

Below is a fictional customer relationship survey; all numbers are for calculation demonstration only, not from any questionnaire platform or industry benchmark. Both periods yielded 1,000 valid responses:

NPS is the “promoter proportion minus detractor proportion” from the same three-category data. Promoter and detractor proportions are not two independent binomial proportions. For a simple random, unweighted, large sample, an approximate variance under the multinomial distribution can be used:

Var(NPS proportion) ≈ [p_promoter + p_detractor - (p_promoter - p_detractor)²] ÷ n

Substituting 0.412, 0.316, and 1,000 for Q1 gives a standard error of about 2.68 NPS points, or an interval of roughly 9.6 ± 5.3; for Q2, the standard error is about 2.64 points. When the two periods are independent samples, the 1.4-point change has a standard error of about 3.77 points, with an approximate 95% interval of -6.0 to 8.8. Since the interval crosses zero, one cannot claim an observed clear improvement; nor can one prove “no effect at all,” because the interval still includes positive or negative changes of business significance.

A 2026 study on the coverage of NPS confidence intervals emphasizes that under small samples or extreme three-category distributions, simple normal approximations may have poor coverage. The calculation here is only to illustrate the magnitude of error. For very small samples, clustering by account, paired tracking, complex sampling, or weighting, resampling, adapted multinomial intervals, or survey statistical methods should be used according to the actual design, with assumptions fully stated.

Total Score Barely Moved, Yet Both Customer Segments Improved

Breaking the same case down by customer stage reveals a sample composition effect:

New customers improved by 8 points and mature customers by 6 points, but because the lower-scoring new customers rose from 30% to 50% of the sample, the overall figure increased by only 1.4 points. For a comparable trend, standardizing Q2 to a fixed composition of 30% new and 70% mature customers, as in Q1, yields 0.3×(-2)+0.7×24=16.2, an improvement of 6.6 points relative to Q1's 9.6.

Standardization does not replace the observed 11.0 with 16.2. The former answers “what total score would result from segment-level changes if the customer-stage composition were unchanged,” while the latter describes the actual mix of respondents. Reports should present both, the weights used, and why composition changed. If the composition shift is the business reality—for example, a large influx of new customers—the actual total remains important; if it is merely due to invitation channels or response tendencies, the sampling process should be fixed first. For further checks, refer to the nonresponse bias guide.

“Why Did You Give This Score?” Is Only the Entry Point for Root-Cause Investigation

Bain's system itself does not ask only one question; it also includes the main reason, improvement suggestions, fast feedback, and closed-loop follow-up. To keep all scores comparable, one can first ask every respondent the same neutral open-ended question: “What is the main reason for the score you gave? Please describe one specific experience.” Then branch according to actual experiences:

1. Confirm the most recent task or touchpoint rather than letting researchers guess functions from vague adjectives;
2. Ask whether the goal was achieved, whether the problem was resolved, and what consequences resulted;
3. Distinguish product issues, service handling, pricing policy, the customer's own processes, and temporary incidents;
4. Code themes back to the original text, denominators, and customer stages, while retaining counterexamples;
5. Choose controllable themes, assign owners, and set behavioral verification metrics for the next round.

For example, if detractors frequently mention “collaboration hassles,” one should not immediately launch a project to rebuild the collaboration module. If the original texts refer separately to missing invitation emails, slow permission approvals, and no one maintaining the organization internally, the owners and verification methods for these three issues are entirely different. Coding of open-ended responses should first have definitions and mutually exclusive/multi-select rules; see the survey codebook guide.

NPS Alone Cannot Predict Retention, Revenue, or Growth

Recommendation intent is an attitude, not actual referral behavior; actual referral is not equivalent to renewal. Contract duration, switching costs, price, alternative supply, and purchasing authority all influence behavior. Keiningham et al., using two-year longitudinal data from more than 8,000 customers across three industries, compared multiple satisfaction and loyalty metrics and concluded that a single recommendation-intent metric is insufficient as the sole predictor of future loyalty behavior; multi-metric models performed better for both referral and retention.

This does not mean NPS is useless, but that its use is limited: it can serve as a stable relational signal and a discussion entry point, but it cannot be assigned causal or financial meaning merely due to correlation. To verify an experience improvement, one should simultaneously observe corresponding task success, issue rates, service records, actual retention or adoption, and, where possible, use a controlled or phased design. For differences from overall satisfaction, see the product satisfaction survey guide.

Industry Rankings and Cross-Country Comparisons Require Particular Restraint

There is no universal answer to “what is a good score” that is independent of methodology. Customer type, relationship length, product category, presence of a genuine referral context, invitation channel, response rate, and market structure all alter the score. Different countries or languages may also show different tendencies in extreme-value and midpoint usage. Cross-national scale research shows that response-scale usage across countries may affect comparative conclusions; therefore, one should check whether results are stable under different reasonable treatments.

External benchmarks provide background only when the question, target population, measurement timing, channel, weighting, and market are sufficiently similar; they cannot be turned directly into performance rankings. For most teams, a more reliable benchmark is their own historical distribution under a stable methodology and a pre-defined threshold for substantive change.

From Survey to Action: Maintain Two Distinct Closed Loops

The individual loop is for resolving specific issues when respondents explicitly consent to be contacted; anonymous research responses cannot automatically become sales leads, and high scores do not equal consent to public endorsement. The systemic loop is for aggregating root causes, judging affected populations, modifying products or processes, and then verifying with pre-specified metrics. The two paths should maintain separate records of permissions, owners, handling status, and outcomes.

Survey platforms can configure the 0–10 question, reason probes, and experience branches, and can provide assisted summarization of open responses; researchers still need to control the sampling frame, contact frequency, customer attribute permissions, raw category counts, and statistical interpretation. AI-generated themes must be traceable to the original text and should not automatically turn “frequently mentioned” into “most impactful.” Before formal launch, according to the survey analysis plan, pre-specify the primary population, segments, comparison thresholds, and missing-data handling.

Ultimately, one can say to management: “This quarter's observed NPS is 11.0, which is 1.4 points above last quarter, but the change interval is wide; the customer-stage composition shifted notably, and fixed-composition results suggest improvement in both stages. We will treat this as a signal that needs continued tracking and verify with onboarding task success and continued-use behavior.” One cannot say: “The NPS increase proves the growth strategy succeeded.” The former places the number back into the evidence chain; the latter turns a recommendation-intent metric into a fact it does not measure.