Probability Chart of Results of Test Mailings
Statistical Probability and Marketing Test Reliability
When you undertake a marketing activity that produces a result, you naturally want to know: if I repeat this activity, will I get similar results? If I double the investment, will I double the returns?
The reality is that there’s no absolute certainty. However, the larger your test sample size, the more confidently you can predict the results of additional marketing activities based on the probability of the result. This is a fundamental principle of statistical testing used in marketing and direct mail campaigns.
Probability Table: 95% Confidence Intervals
The table below shows how confident you can be about future results at 95% confidence level. If your test mailing achieved a specific return rate (1-5%), this table shows the expected range of results if you mailed the same message to your entire prospect list.
How to read this table:
- Find your test mailing size (100 or 250 recipients)
- Find your observed return rate (1%, 2%, 3%, 4%, or 5%)
- Look at the resulting range in the “between” column
- This is the 95% confidence interval for your full-list mailing
| Test Sample Size | Return Rate on Test | 95% Confidence Range |
|---|---|---|
| 100 | 1% | 0% to 2.99% |
| 100 | 2% | 0% to 4.80% |
| 100 | 3% | 0% to 6.41% |
| 100 | 4% | 0.08% to 7.92% |
| 100 | 5% | 0.64% to 9.36% |
| 250 | 1% | 0% to 2.26% |
| 250 | 2% | 0.23% to 3.77% |
| 250 | 3% | 0.84% to 5.16% |
| 250 | 4% | 1.52% to 6.48% |
| 250 | 5% | 2.24% to 7.76% |
Understanding the Ranges
Larger test size = tighter confidence range. Notice that testing with 250 recipients produces narrower ranges than testing with 100. This is because statistically, larger samples give us more confidence in predicting future results.
Lower return rates = wider ranges. A 1% return produces wider confidence intervals than a 5% return, because lower response rates have more statistical variability.
Practical Application
Use this data when:
- Planning a direct mail or email campaign test
- Deciding whether to roll out based on test results
- Forecasting expected ROI from a full campaign
- Determining whether your test size was large enough
- Setting realistic expectations for campaign performance
Important note: These are statistical probabilities based on sample size and observed return rate. Your actual results may fall outside the predicted range, but at 95% confidence, they should fall within this range 95% of the time.
General Information Disclaimer
This guide provides statistical guidance based on standard probability theory used in direct marketing and test campaign planning. The probability ranges shown are based on binomial distribution statistical methodology and represent 95% confidence intervals.
This guidance is educational and does not guarantee specific results. Actual campaign performance will vary based on message, audience, timing, competitive factors, and many other variables beyond statistical prediction.
How HA & CO Can Support Your Business
At HA & CO, we help UK small businesses plan effective marketing campaigns, forecast results, and make data-driven decisions about campaign investment and scaling. Whether you’re testing a new marketing channel or rolling out a proven concept, our team can help you analyze results and plan your next steps.
Contact us to discuss your marketing planning and campaign forecasting needs.