AB Test
Compare two independent conversion rates with effect sizes, a 95% interval and a two-sided z/Fisher test.
About this tool
Enter users and users with a conversion for independent groups A and B. Each user contributes one binary outcome. Whole digit counts only: users 1–1,000,000,000; conversions 0–users. Counts are not rounded. The comparison updates automatically. Only the four raw counts are kept through the shared local browser adapter; results and the expanded statistics panel are not stored. No upload.
Effects use B − A. Absolute difference is in percentage points (pp); relative lift is (rate B − rate A)/rate A and is undefined at rate A=0. No automatic winner or benefit is inferred from the sign.
If every pooled expected success/failure cell is at least 5, use the two-sided pooled two-proportion z-test without continuity correction. Otherwise use Fisher’s two-sided conditional test, summing tables no more probable than the observed table. The full sparse support is bounded by the input limits. If nobody or everybody converts, Fisher gives p=1; this is not evidence that the underlying rates are equal.
The 95% interval for B − A is Newcombe’s combination of separate Wilson score intervals, without continuity correction. It is not the inversion of the Fisher or pooled z-test, so interval and p-threshold decisions can differ. Small p-values use scientific notation; floating-point underflow is shown as <1e-300, never as an exact zero.
Assumptions: independent Bernoulli observations and a planned fixed sample/horizon. Randomization, representative traffic and a meaningful conversion definition are not checked. Repeated peeking, multiple comparisons, sequential stopping, paired/repeated users, clustering, Bayesian probabilities, power and sample-size planning are not handled. A p-value is not the probability that B is better or that the null hypothesis is true.