ratesci

ratesci is an R package to compute confidence intervals (‘CIs’) and tests for:

• a single binomial proportion, or Poisson rate (‘p’)
• a difference between independent binomial proportions or Poisson rates (risk difference or rate difference, ‘RD’)
• a ratio of proportions or rates (relative risk or rate ratio, ‘RR’)
• a binomial odds ratio (‘OR’)
• stratified calculations for any of the above
• a difference (RD) or ratio (RR) for paired binomial proportions
• odds ratio for paired proportions using the conditional model
• a binomial proportion from clustered data

A number of different methods are offered, but in each case, the recommended default is based on asymptotic score methodology (from (Wilson 1927), (Miettinen and Nurminen 1985) and (Tango 1999)), but including skewness corrections following the principles of (Gart and Nam 1988). The resulting family of skewness-corrected asymptotic score (SCAS) methods (Laud 2017), (and Laud 2026, under review) ensures equal-tailed coverage (also referred to as central location), in other words for a nominal 95% confidence interval, the one-sided non-coverage probability is (on average) close to 2.5% on each side. The equal-tailed property is particularly important for non-inferiority testing (as it ensures correct type I error rates for one-sided tests), but should be considered generally desirable for two-sided confidence intervals. Unfortunately this aspect of performance is often overlooked in the literature, in favour of interval width.

Most of the above list is covered by scoreci(), with the exception of clustered proportions (which are handled by clusterpci()) and paired contrasts (scorepairci()). Stratified calculations are also catered for (e.g. for analysis of a clinical trial with stratified randomisation, or for a meta-analysis, including random effects). Options are included for omitting the skewness correction to obtain legacy methods such as stratified or unstratified Miettinen-Nurminen and Wilson intervals, Tango intervals for paired data, or chi-squared, CMH, Farrington-Manning or McNemar tests.

Simplified functions producing a selection of different confidence interval methods for each contrast (with or without continuity adjustment) are provided by rdci(), rrci() and orci() for independent binomial or Poisson rates, and rdpairci(), rrpairci() and orpairci() for paired binomial proportions.

• See the vignettes for further details and examples of the SCAS and other intervals, including plots illustrating the impact of the skewness correction, for the single rate, basic contrasts, stratified contrasts, and paired contrasts.

In each case, the asymptotic score methods provide a matching 2-sided test for association, and also a 1-sided hypothesis test against any specified null parameter value for a non-inferiority test. Both tests are guaranteed to be coherent with the interval, so that a 100(1-$\alpha$)% interval excludes the null hypothesis value if and only if the hypothesis test is significant at the $\alpha$ 2-sided or $\alpha$/2 1-sided significance level. The tests for association are variants of (and in many cases directly equivalent to) a chi-squared test or CMH test, and the non-inferiority test is analogous to a Farrington-Manning test, all with improved control of type I error achieved by the bias and skewness corrections. For paired proportions, the test for association is a variant of the McNemar test, incorporating an ‘N-1’ adjustment which (empirically) avoids any violations of the nominal significance level.

• See the hypothesis tests vignette for further details of the relationships between SCAS tests and conventional chi-squared and CMH tests.

Another family of methods offered by the package, with reasonable performance for large (single-stratum) sample sizes (but without a matching hypothesis test), uses the Method of Variance Estimates Recovery (MOVER), also known as Square-and-Add (Newcombe 2012, chap. 7). These methods combine intervals calculated separately for each proportion. The recommended default gives the MOVER-J method, using Jeffreys equal-tailed intervals instead of the Wilson method preferred by Newcombe. This improves on traditional approximate methods with respect to one-sided and two-sided coverage, particularly for the RR contrast, but does not match the performance of the SCAS method. As the Jeffreys interval is based on a Bayesian conjugate prior, the MOVER approach allows the option to incorporate prior beliefs about the rates in each group - by default, the non-informative Jeffreys $Beta(0.5, 0.5)$ priors are used (or corresponding Gamma priors for Poisson rates). MOVER intervals are available in moverci() for all contrasts of independent binomial and Poisson rates, and in moverpairci() for paired binomial RD and RR.

For those wishing to achieve strictly conservative coverage, continuity adjustments are provided as approximations to “exact” methods, with the option to adjust the strength of the adjustment (as the Yates correction is widely recognised to be an over-conservative adjustment). The performance of these adjustments has not been extensively evaluated, but they appear to be more successful for SCAS than for MOVER, in terms of achieving conservative coverage.

An online calculator based on this package is available here.

Installation

The current official (i.e. CRAN) release can be installed within R with:

install.packages("ratesci")

The latest development version of the package can be installed with:

# install.packages("pak")
# pak::pak("petelaud/ratesci")
install.packages("ratesci", repos = c('https://petelaud.r-universe.dev', 'https://cloud.r-project.org'))

This builds the package from source based on the current version on GitHub

A SAS macro implementation of scoreci() and scorepairci() is also available at https://github.com/petelaud/ratesci-sas

Example

Below is a basic example which shows you how to request a confidence interval for the difference between proportions 5/56 - 0/29. The $call output element shows that the default settings give an interval for the risk difference (contrast = "RD"), for binomial proportions (distrib = "bin"), at a 95% confidence level. Variance bias correction (bcf) and skewness correction (skew) are applied, continuity adjustment (cc) is not. This is the skewness-corrected asymptotic score (“SCAS”) confidence interval. (For Miettinen-Nurminen, use skew = FALSE, for Gart-Nam, use bcf = FALSE.)

library(ratesci)
scoreci(x1 = 5, n1 = 56, x2 = 0, n2 = 29)
#> $estimates
#>         lower     est  upper level x1 n1 x2 n2   p1hat p2hat   p1mle p2mle
#> [1,] -0.01862 0.09169 0.1867  0.95  5 56  0 29 0.08929     0 0.09169     0
#> 
#> $pval
#>      chisq pval2sided theta0 scorenull pval_left pval_right
#> [1,] 3.025      0.082      0     1.739     0.959      0.041
#> 
#> $call
#>  distrib contrast    level      bcf     skew       cc 
#>    "bin"     "RD"   "0.95"   "TRUE"   "TRUE"  "FALSE"

An example of a paired analysis follows, using the data from Table II of (Fagerland et al. 2014). Here the bias and skewness corrections are again applied by default. Omitting both would produce the Tango asymptotic score interval for contrast = "RD", or the Tang method if contrast = "RR".

scorepairci(x = c(1, 1, 7, 12))
#> $data
#>          Test_2
#> Test_1    Success Failure
#>   Success       1       1
#>   Failure       7      12
#> 
#> $estimates
#>        lower     est    upper level   p1hat p2hat  p1mle  p2mle phi_hat phi_c
#> [1,] -0.5281 -0.2859 -0.01842  0.95 0.09524 0.381 0.0952 0.3811 0.07954     0
#>      psi_hat
#> [1,]   1.714
#> 
#> $pval
#>      chisq pval2sided theta0 scorenull pval_left pval_right
#> [1,] 4.286    0.03843      0     -2.07   0.01922     0.9808
#> 
#> $call
#>   contrast      level        bcf       skew         cc closedform 
#>       "RD"     "0.95"     "TRUE"     "TRUE"    "FALSE"    "FALSE"

References