In a pilot, 100 valid entered screens yielded 12 qualified respondents, giving an observed eligibility rate of 12%. To obtain 300 valid completes, assuming an 80% completion rate among eligible respondents and a 95% post-completion quality pass rate, approximately 3,290 screening entries would be needed based on the point estimate. However, 12/100 itself is highly unstable: the Wilson 95% interval is roughly 7.0% to 19.8%, corresponding to a screening requirement ranging from about 1,994 to 5,640 entries.
A low eligibility rate raises costs not simply because “rare populations command higher unit prices,” but because before each valid response, more people spend time screening, more invitations and channel capacity are consumed, and rare quotas fill later. A workable costing model must separate reach, entry, eligibility determination, quota availability, main completion, and quality pass-through; reporting a single eligibility rate conflates coverage, survey burden, and quality issues into one number.
Standardize the Denominator First: Eligibility Rate Is Not Market Penetration
The observed eligibility rate in a project can typically be written as:
Eligibility rate = Number confirmed eligible ÷ Number with valid eligibility determination completed
The denominator should include those confirmed eligible and those confirmed ineligible, but should not count individuals who exited before key eligibility questions, or whose invitations were undelivered or never opened, as ineligible. These individuals have unknown eligibility. Invalid entries such as duplicates, bots, and test data should also be tracked separately, rather than being used to lower or inflate the eligibility rate.
12% only describes the observed incidence under that invitation source, screening specification, collection window, and entry mechanism; it does not mean that “12% of the market are target users.” A sample panel itself may already be enriched with certain category users, while open ads may attract self-selected clicks by those interested in the topic; conversely, using only a brand's customer list will miss competitors' customers and those who have lapsed. Change the source, and the eligibility rate changes.
A Disposition Table Explains More Than a Single Percentage
| Final Status | Eligibility Known | Included in Eligibility Denominator | Operational Meaning |
|---|---|---|---|
| Confirmed eligible and quota open | Eligible | Yes | Proceed to main survey |
| Confirmed eligible but quota full | Eligible | Yes, flag as quota-full | Reflects true eligibility, not "ineligible" |
| Confirmed ineligible | Ineligible | Yes | Record by first failed criterion |
| Terminated before eligibility determination | Unknown | No | May indicate burden, technical issues, or eligibility differences |
| Invitation not opened or unreachable | Unknown | No | Belongs to reach and participation funnel |
| Duplicate, bot, or test | Invalid | No | Record verification failures separately |
AAPOR's standard definitions require recording complete dispositions for sample units and distinguishing among eligible, ineligible, unknown eligibility, response, refusal, and non-contact. The U.S. Census Bureau's corresponding requirements also classify samples into eligible, ineligible, and unknown eligibility; the proportion of unknown cases that might be eligible can only be estimated when a defensible basis exists.
This distinction matters equally for online projects. If the system terminates immediately after an age-quota full condition is detected, and purchase behavior is never asked, that person is only “quota-full on age and subsequent eligibility unknown”—not “ineligible based on behavior.” Otherwise, different sequences of quota closings would mechanically alter a project's eligibility rate.
From Delivered Invitations to Valid Responses: At Least Six Gates
When planning, write the funnel as a multiplication:
Valid responses = Delivered invitations × Entry rate × Determinale rate × Eligibility rate × Completion rate among eligible × Quality pass rate
- Delivered invitations: After removing bounces, invalid numbers, and technical failures.
- Entry rate: Actually opened and started the screener once the invitation was seen; not the same as response rate.
- Determinable rate: Reached a point where eligibility can be confirmed or rejected.
- Eligibility rate: Among those determinable, met the target definition.
- Completion rate: Eligible participants who started and completed the main task.
- Quality pass rate: Retained as analysis sample per pre-set rules.
If only screening entries are recorded, calculate from entry volume; if purchasing lists or placing invitations, you must further include delivery and entry. For instance, if the point estimate requires 3,290 screening entries and the expected post-delivery entry rate is 30%, about 10,967 delivered invitations are needed. Calling both “invitations” can cause schedule misunderstandings by a factor of three or more.
Pilot 12/100: Point Estimate 3,290 Entries, Risk Scenario ~5,640
The following is illustrative fictional data, not representative of WJPro panel, any supplier, or industry benchmarks. Target is 300 valid completes. The pilot completed eligibility determination for 100 screening entries with duplicates and invalids removed, and 12 qualified. Expected completion rate among eligible is 80%, and quality pass rate 95%.
Under the point estimate, the probability that each screening entry becomes a valid response is 0.12 × 0.80 × 0.95 = 0.0912. The required entries are:
300 ÷ 0.0912 = 3289.5, rounded up to 3,290.
But a small pilot cannot support that the eligibility rate is exactly 12%. Using the Wilson interval for a binomial proportion, the 95% interval for 12/100 is roughly 7.0% to 19.8%. Holding completion and quality assumptions constant:
| Eligibility Scenario | Probability per Entry to Valid Sample | Screening Entries Required for 300 |
|---|---|---|
| Optimistic 19.8% | 19.8% × 80% × 95% = 15.05% | ~1,994 |
| Point Estimate 12.0% | 12.0% × 80% × 95% = 9.12% | ~3,290 |
| Conservative 7.0% | 7.0% × 80% × 95% = 5.32% | ~5,640 |
The interval is not a guarantee that future eligibility rates will fall within it; changes in source, screener, or respondent mix over time can invalidate it. Its purpose is to shift scheduling from a single-point commitment to a capacity range: secure resources for the point estimate first, then define triggers for adding sources, extending deadlines, or trimming minor quotas.
Multiple Eligibility Criteria Cannot Be Simply Multiplied Together
If a team knows that 30% have purchased the category, 40% purchased within the last three months, and 50% have decision involvement, they might estimate the joint eligibility rate as 30% × 40% × 50% = 6%. This only holds if the three criteria are mutually independent and definitions and sources are consistent—which is rarely true in practice. Recent purchasers may be more likely to have decision involvement, and category buyers' role distribution differs from the full sample.
Priority should be given to using a joint distribution from the same source or running a formal logic pilot, and recording stepwise criterion attrition. The screener order also affects observation: asking a sensitive or complex criterion early may cause exits, leaving subsequent eligibility unknown. Without revealing correct answers, place high-screen-out, low-burden criteria early, and put sensitive or time-consuming verification exactly where necessary.
When Eligibility Rate Suddenly Drops, Diagnose Which Gate Changed First
| Symptom | Possible Cause | Evidence to Check First |
|---|---|---|
| Entry rate down | Invitation subject, timing, channel delivery, or incentive attractiveness | Delivery, open rates, device, and source |
| Determinable rate down | Screener too long, sensitive, technical errors, or incompatibility | Drop-off by question and error logs |
| Eligibility rate down | Source mix, time window, quota closures, or true population shift | Screen-out by criterion, source, and collection day |
| Main completion down | Survey length, task device, content relevance | Break-offs after eligibility and duration |
| Quality pass rate down | Fraud, incentivized screening, rule false positives, or source anomalies | Specific rules, manual review, and source variance |
Looking only at “rising effective cost” may lead to incorrectly relaxing eligibility. If the real issue is a mobile upload failure, changing the target population both harms the research and fails to fix the experience; if an age quota is full and causing many eligible respondents to be blocked, adjust the targeting, not declare the population rarer.
Raising Within-Source Eligibility Does Not Necessarily Improve Overall Representativeness
Using historical behavior tags to pre-screen, targeting professional communities, or drawing from client records can enrich entrants and reduce screening loss. But doing so changes within-source incidence and may miss people without tags, less active, or not in those communities. If the research aims to estimate population proportions, selection probabilities, coverage, and weighting must be reported; if merely recruiting qualified respondents for concept testing, retain sources for sensitivity checks.
A sensible order for reducing cost is:
- Check whether each eligibility criterion genuinely changes a decision and remove decorative tags.
- Pre-screen using framework information consistent with the target population to reduce obvious ineligible contact.
- Shorten and pre-test the screener so genuinely eligible respondents do not exit before determination.
- Estimate the funnel separately for each source, device, and quota; do not apply a single average.
- Use multiple sources and deduplication for low-incidence populations, and record coverage limits per source.
- Core eligibility must not become broad interest just to meet pricing; adjust sample size or scope of decisions when necessary.
Screened-Out Individuals Also Gave Time—Cost Is Not Only for Successful Respondents
The UK Government Analysis Function's respondent burden guidance counts screener time used to establish eligibility as part of overall participation cost. Commercial research should likewise calculate time for all entrants, not just the 300 completions. If 3,290 people spent on average two minutes each on eligibility determination, the screener alone consumed roughly 109.7 person-hours; a lengthy screener not only raises platform and incentive costs but also repeatedly consumes goodwill in rare populations.
When comparing quotes, ask all suppliers to fill the same cost ledger:
- Whether the quoted unit is delivered invitation, screening entry, eligible start, main completion, or quality-passed.
- Whether screen-outs, unknown eligibility, quota-full, technical exits, and quality failures are billed.
- Whether participant incentives, hard quota premiums, source expansions, project management, and taxes are included.
- Definition of quality replacement, cap, review mechanism, and delivery time.
- How survey length, open-ended questions, audio/video, file uploads, and device requirements are accounted for.
- What rules re-estimate price, timeline, and scope when actual eligibility deviates from the interval.
A “cheaper per complete” price that excludes screening loss and quality replacement may cost more than an all-inclusive valid sample unit price. Conversely, a supplier promising a fixed low price does not prove sufficient audience; evidence must be shown under the same definitions, source, and disposition criteria.
Update Forecasts During Fieldwork by Source and Quota—But Do Not Chasing Results to Change Eligibility
After soft launch, build a daily funnel: deliveries, entries, determinables, eligible, quota-full, completes, quality passes, and cumulative speed by source. Update eligibility rates using intervals rather than single points, and forecast completion dates for the hardest quota separately. If switching sources, view old and new data side by side to avoid confounding source changes with collection time.
AAPOR's 2025 report on unknown eligibility stresses that estimating the proportion eligible among unknown cases should rely on the best available scientific information, not a number that flatters the rate. Likewise, do not classify all undetermined cases as ineligible to emphasize “scarcity,” nor delete them all to show a high eligibility rate to the client. When method notes provide disposition counts, readers can recalculate.
WJPro can configure screening, quotas, and multi-channel collection, and record statuses from entry to valid response. Project teams still need to define which statuses are unknown eligibility or quota-full, ensuring supplier exports and internal dashboards share the same definitions; quality judgments should follow predetermined rules and manual review, not turn pricing pressure into post-hoc looser cleaning.
A Conclusion Suitable for Scheduling and Procurement Approval
This case can be written as: “Of 100 valid eligibility determinations, 12 qualified; point estimate 12%, Wilson 95% interval approximately 7.0% to 19.8%. Assuming 80% completion among eligible and 95% quality pass, 300 valid responses require approximately 3,290 screening entries; across the interval ends, capacity scenarios are approximately 1,994 to 5,640, excluding invitation entry rate. The project starts with the point estimate and updates the interval after 300 cumulative determinations; if a source falls below the preset lower bound, first check dispositions and screener break-offs, then expand sources per the set rules.”
It must not be written as: “The market eligibility rate is 12%, so 3,290 entrants are necessarily needed.” Only by spelling out the denominator, interval, sources, and trigger rules can the eligibility rate truly inform cost and timeline decisions.
Further planning can be combined with Target Sample Population Definition, Sample Quota Strategy, Survey Length and Sample Cost, and Survey Soft Launch Monitoring.
Eligibility Disposition, Respondent Burden, and Sample Quality References
- AAPOR — Standard Definitions, 10th edition
- AAPOR — Estimating the Eligibility Status of Cases with Unknown Eligibility
- U.S. Census Bureau — Requirements for Calculating and Reporting Response Rates
- UK Government Analysis Function — Monitoring and reducing respondent burden
- ESOMAR/GRBN — Guideline on Online Sample Quality
- NIST/SEMATECH — Confidence intervals for a binomial proportion