20.15 — Meta-analysis
Introduction
Meta-analysis is a type of systematic review (Greenhalgh 1997, Akobeng 2005), a statistical method used to combine the results from multiple independent studies that addressed the same topic. By combining studies, the goal is to calculate a pooled estimate as a measure of overall effectiveness of the treatment. The underlying logic is that multiple studies draw subjects from the same reference population, thus, combining data provides a more powerful test of the hypothesis.
Meta-analysis can increase the overall sample size, which improves the study’s ability to detect an effect if one exists. So, how to combine results from different studies ostensibly addressing the same hypothesis? Inevitably, some subjectivity will come into decisions about including or excluding studies — the goal is to identify quality studies, which may have conflicting conclusions and test whether a single conclusion of effectiveness is warranted based on a summary of evidence.
Meta-analysis is now a routine type of study, conducted in multiple disciplines, including biomedical field (eg, Cochrane Reviews, more than 15K since 1993, Anderson et al 2024), and environmental studies (eg, Resende et al 2021), and sports science (eg, Miyamoto-Mikami et al 2018).
In general, meta-analysis studies return the combined effect size –the single, pooled effect across all included studies — of an intervention and the heterogeneity among the studies — how consistent measured differences. Models may be fixed effects — that is, any observed differences among studies are due to chance or sampling error — or random effects — the true effect size varies from study to study and that these effects are drawn from a distribution.
The purpose of this page is to provide a basic overview and an introduction to the meta-analysis approach. We previously introduced the Cochran’s Q test aka heterogeneity
test in Chapter 9.4. The chi-square test test asks whether results from combined studies are consistent. After discussing inclusion criteria, we provide plotting techniques and additional statistical tests for heterogeneity of studies.
In Chapter 12.4, we introduced a simple meta-analysis combining ten separate one sample t-tests of the hypothesis that inbred mice and genetically diverse mouse strains do not differ for average lifespan. P-values were combined and we used Fisher’s method to evaluate the hypothesis.
Note 1: The most robust and recommended method for meta-analysis is to use effect sizes (eg, standardized mean differences, odds ratios, risk ratios) and their corresponding standard errors or confidence intervals (Nakagawa and Cuthill 2007). This approach avoids the limitations of arbitrary p-value thresholds and provides a more complete picture of the effect’s magnitude and precision.
Selection criteria
Any project, whether an observational study or an experiment, starts with a question that is modified into one or more testable hypothesis. We don’t collect the data then speculate about causes for perceived patterns that may appear. Rather, pattern recognition process can be used to generate hypotheses subject to future testing. Our example was about lifespan and genetic background and genetic diversity: Longevity refers to the capacity to live a long life whereas lifespan is the maximum amount of time an individual can live. We ask if genetic diversity increases longevity across a population; specific genetic background is linked to an individual’s predisposition to long life.
Given the importance such work has in clinical research, as you can imagine, there is plenty of advice and suggested guidelines for how to proceed. Of note, review systematic review Population, Intervention, Comparison, Outcome (PICO) guidelines (Schardt et al 2007). Additionally, see the Cochrane Handbook (2024).
With a clear hypothesis in hand, for both systematic review and meta-analysis, the study begins with a search of the published literature. Search results from the query “inbred mice strain lifespan,” in Pubmed returned about 1134 publications; replacing “longevity” for “lifespan,” returned 654 publications. We want primary literature, not reviews, so we add NOT (Review[Publication Type]) to the query: 1080 and 617, respectively. Further restricting the pool by restricting range to last ten years returned a manageable 197 and 116 results, respectively. We can begin selecting our data from the studies. We now have to decide whether all or only some meet our standards for consideration. Of course, this is not the same thing as looking at the results and selecting only the studies that confirm our thinking! Ideally, reporting of both positive (large effect size, small p-value) and negative (small effect size, large p-value) studies can — and should — be included. Our goal, combine the results of multiple independent studies on the same topic to reach a more powerful and reliable conclusion than a single study could provide.
Note 2: Publication bias concerns likely overestimates of effect size for meta-analysis, just as the tendency for reporting only “statistically significant” p-values (Ioannidis 2008).
Inclusion criteria are the requirements a study must meet to be considered for the review; exclusion criteria are the specific reasons a study is disqualified. Decisions about which studies to include consider demographics, intervention type, and study design. For example, the review may focus only on RCT studies. Likely, the review is focused on experimental work and so, excluding previous systematic reviews would be an example of an exclusion criteria. At the same time we need to be transparent about our approach; that includes defining a clear research question, establishing strict inclusion/exclusion criteria, conducting a comprehensive literature search across multiple databases, and performing a rigorous data extraction and quality assessment of included studies (Meline 2006, Paldam 2015, Cochrane Handbook). For our lifespan and genetic diversity example I found two studies that met my criteria: the studies needed multiple inbred strains or outbred strains and the raw data needed to be available. Two studies met this need: Yuan et al (2009) and Mullis et al (2025).
An imperfect example. To practice applying selection and exclusion criteria for a meta-analysis, I ran a PubMed search using the keywords botox migraine. I limited the results to clinical trials published within the past five years, which gave me 14 papers. I read the abstracts and decided to include studies only if they had a control group, focused on patients with chronic migraine, and reported an exact p-value instead of just saying “P < 0.05.” Table 1 shows what I found, including the PubMed link for each paper, whether it was included, the sample size, the p-value, and my comments on why any studies were excluded.
Table 1. Example data collection for meta-analysis, topic botox and treatment of chronic migraines.
| paper | Include | N | p-value | Why exclude? |
|---|---|---|---|---|
| https://pubmed.ncbi.nlm.nih.gov/37499085/ | No | review | ||
| https://pubmed.ncbi.nlm.nih.gov/38982666/ | Yes | 1384 | 0.01 | |
| https://pubmed.ncbi.nlm.nih.gov/37994890/ | No | 209 | 0.155 | no control group |
| https://pubmed.ncbi.nlm.nih.gov/41091731/ | No | 775 | > 0.05 | no CI, no exact p-value |
| https://pubmed.ncbi.nlm.nih.gov/34404257/ | No | 32 | 0.03 | within subjects design, no control group |
| https://pubmed.ncbi.nlm.nih.gov/37315247/ | Yes | 209 | 0.365 | |
| https://pubmed.ncbi.nlm.nih.gov/33722518/ | No | 60 | < 0.001 | within subjects design, no control group, no exact p-value |
| https://pubmed.ncbi.nlm.nih.gov/36189948/ | No | 0.092 | no sample size, no control | |
| https://pubmed.ncbi.nlm.nih.gov/36189948/ | No | 0.174 | no sample size, no control | |
| https://pubmed.ncbi.nlm.nih.gov/35166150/ | No | method paper | ||
| https://pubmed.ncbi.nlm.nih.gov/33106278/ | Yes | 15 | 0.038 | |
| https://pubmed.ncbi.nlm.nih.gov/37235358/ | No | 139 | < 0.0001 | no CI, no exact p-value |
| https://pubmed.ncbi.nlm.nih.gov/35064733/ | No | overuse, not migraine | ||
| https://pubmed.ncbi.nlm.nih.gov/33241323/ | No | PTH, not migraine | ||
| https://pubmed.ncbi.nlm.nih.gov/32873093/ | No | no control group |
That leaves only three papers, not enough for a meta-analysis. By changing years from five to ten, 40 papers were available to search. By removing all time restrictions, 109 papers were returned since year 2000. Medical use of botox dates to the 1970s (Scott 2023).
R packages
metafor package: provides extensive functions for calculating various effect sizes, fitting different statistical models (e.g., fixed- and random-effects, mixed-effects), conducting meta-regression, and generating numerous meta-analytical plots.
See also metaforGUI package which provides a Graphical User Interface (GUI) for the R metafor Package.
meta package: provides user-friendly and comprehensive collection of functions for conducting standard meta-analyses.
Much has been written about how to conduct meta-analysis and caveat emptor — readers should be on notice that I present here just a fraction of the “how to” and pitfalls of meta-analysis. Interested readers should see Doing meta-analysis with R: A hands-on guide by M. Harrer et al (2021).
I2 or I-squared test
In a meta-analysis, the I-squared test
quantifies the percentage of total variation in a set of study results that is due to differences between studies rather than random sampling error (chance). An
of 0% means all variation is due to chance, while a higher percentage indicates that differences between studies are a significant source of variation. As a reminder, the
test returns a p-value that can be used to interpret whether that heterogeneity is statistically significant. We run our life span of inbred and genetically diverse outbred mouse strains example in Chapter 12.4; the p-value was small — we used Fisher’s method p-value combination approach to test the main hypothesis.
Here, we calculate
; we first need the effect size; Cohen’s d (Chapter 11.4) will do.
Table 1. Updated Table 1 from Chapter 12.4.
| Strain | n | mean | sd | cohen’s d | V(d) |
| 129S1/SvImJ | 32 | 787.4 | 159.16 | 0.045 | 0.191 |
| A/J | 32 | 630.7 | 130.20 | 1.26 | 0.04 |
| BALB/cByJ | 32 | 734.4 | 154.43 | 0.389 | 0.0368 |
| BUB/BnJ | 24 | 611.3 | 218.34 | 0.839 | 0.0485 |
| C3H/HeJ | 29 | 724.1 | 131.48 | 0.536 | 0.0403 |
| C57BL/6J | 29 | 855.7 | 185.34 | 0.330 | 0.0399 |
| CBA/J | 30 | 622.9 | 181.95 | 0.943 | 0.0405 |
| FVB/NJ | 26 | 750.3 | 230.11 | 0.192 | 0.0438 |
| P/J | 32 | 676.0 | 178.82 | 0.663 | 0.0374 |
| SWR/J | 31 | 831.9 | 181.31 | 0.206 | 0.0376 |
The effect size ranged from practically no difference (Cohen’s d of 4.5%) to substantial effect on average lifespan of inbred vs outbred strains (Cohen’s d of 126%). The simple average of the Cohen’s d values was 0.541; a weighted average accounting for differences in variance needs to be applied. Regardless, the statistic we want is the coefficient from meta-regression, which indicates how a change in that predictor relates to the effect size.
Note 3: The greatest effect size — difference in average lifespan — was reported for the A/J strain — this strain is primarily used in cancer research because of its susceptibility to cancers. The smallest effect size was for the 129S1/SvImJ strain, which is used for creating genetically modified mice.
With effect size calculated, we proceed to get
. We use rma() from metafor package. This function performs several tests: a fixed-effects model and therefore we test a model where we assume all studies found the exact same effect, a random-effects model where the assumption is that effects are similar but vary across studies, plus a mixed-effects model uses a combination of both, and finally, a meta-regression. The meta-regression uses the data from those studies to try and explain why the results differ.
R code
library(metafor) strains <- 1:10 my_effsize <- c(0.0446092, 1.258065, 0.3891731, 0.8390583, 0.5354427, 0.3302039, 0.9431162, 0.192082, 0.6626776, 0.2062765) my_sample <- c(32, 32, 32, 24, 29, 29, 30, 26, 32, 31) my_varC <- (my_effsize^2)/(2*my_sample) myData <- data.frame(strains, my_effsize, my_sample, my_varC) rma(yi = my_effsize, vi = my_varC, data = myData)
R output
Random-Effects Model (k = 10; tau^2 estimator: REML) tau^2 (estimated amount of total heterogeneity): 0.1299 (SE = 0.0643) tau (square root of estimated tau^2 value): 0.3604 I^2 (total heterogeneity / total variability): 99.15% H^2 (total variability / sampling variability): 117.11 Test for Heterogeneity: Q(df = 9) = 390.9415, p-val < .0001 Model Results: estimate se zval pval ci.lb ci.ub 0.5208 0.1169 4.4558 <.0001 0.2917 0.7499 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Interpretation.
With an
of 99%, which suggests a very high degree of heterogeneity among the comparison between the DO strain and the ten inbred strains for life span. As before, the heterogeneity
or Q test was highly statistically significant, suggests that the observed variability is larger than what would be expected due to chance alone. Thus, all of the variability in the observed effects comes from between-strain differences, not random chance — it also justifies use of a random effects model, not a fixed model. Strain A/J, with the greatest difference in lifespan compared to the DO strain, is recognized for its susceptibility to carcinogenic tumors
From the meta regression, the coefficient was 0.52 — inbred strain membership was associated with change in the size of the effect. We conclude that in general, the genetically inbred strains differ substantially for average lifespan compared to the genetically variable DO strain: The true average effect was somewhere between 0.2917 and 0.7499.
Our results conform to our expectations — longevity is likely affected by a combination of effects from many genes along with an individual’s genetic background and interaction with the environment, cf discussion in Mullis et al (2025).
Forest plot of effect size.
A forest plot visually summarizes results from multiple studies, showing each study’s effect size (eg, odds ratio, mean difference, Cohen’s d) and variance or confidence interval as squares and lines, plus the overall pooled estimate (diamond). It helps quickly assess study consistency (heterogeneity), see individual study contributions, and determine if the combined evidence points to a significant overall effect.
For example, our inbred strains vs the outbred DO strain for lifespan shows a clear, overall effect (Fig 1). The diverse genetic strain lifespan is significantly greater than the majority of inbred strains, and the combined effect is about 0.5.
R code
library(metafor)
myData <- data.frame(
Strains = paste("Strains", c("129S1/SvImJ", "A/J", "BALB/cByJ", "BUB/BnJ",
"C3H/HeJ", "C57BL/6J", "CBA/J", "FVB/NJ", "P/J", "SWR/J")),
cohenD = c(0.045, 1.26, 0.389, 0.839, 0.536, 0.33, 0.943, 0.192, 0.663, 0.206),
cohenV = c(0.1910, 0.0400, 0.0368, 0.0485, 0.0403, 0.0399, 0.0405, 0.0438, 0.0374, 0.0376)
)
res2 <- rma(cohenD, cohenV, data = myData, method="REML")
# str(res2)
forest(res2,
slab = myData$Strains,
xlab = "Cohen's d")
Figure 1. Forest plot, Cohen’s effect size lifespan differences among inbred strains of mice compared to outbred strain.
Questions
1. What’s the difference between a literature review and a systematic review?
2. For which kinds of research goals would a meta-analysis be more appropriate than a systematic review?
Quiz Chapter 20.15
Meta-analysis
References and suggested readings
Akobeng, A. K. (2005). Understanding systematic reviews and meta-analysis. Arch Dis Child, 90:845-848.
Andersen, M. Z., Zeinert, P., Rosenberg, J., & Fonnes, S. (2024). Comparative analysis of Cochrane and non-Cochrane reviews over three decades. Systematic Reviews, 13, 120.
Cochrane reviews, Cochrane Library. (2025). https://www.cochranelibrary.com/
Greenhalgh, T. (1997). How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). BMJ, 315(7109), 672–675.
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555(7695), 175–182.
Hansen, C., Steinmetz, H., & Block, J. (2022). How to conduct a meta-analysis in eight steps: A practical guide. Management Review Quarterly, 72(1), 1–19.
Harrer, M., Cuijpers, P., Furukawa, T.A., & Ebert, D.D. (2021). Doing Meta-Analysis with R: A Hands-On Guide. Boca Raton, FL and London: Chapman & Hall/CRC Press. Link to website.
Ioannidis, J. P. A. (2008). Why Most Discovered True Associations Are Inflated. Epidemiology, 19(5), 640.
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5 (updated August 2024). Cochrane, 2024. Available from www.cochrane.org/handbook.
Meline, T. (2006). Selecting studies for systemic review: Inclusion and exclusion criteria. Contemporary Issues in Communication Science and Disorders, 33(Spring), 21–27.
Miyamoto-Mikami, E., Zempo, H., Fuku, N., Kikuchi, N., Miyachi, M., & Murakami, H. (2018). Heritability estimates of endurance-related phenotypes: A systematic review and meta-analysis. Scandinavian Journal of Medicine & Science in Sports, 28(3), 834–845.
Mullis, M. N., Wright, K. M., Raj, A., Gatti, D. M., Reifsnyder, P. C., Flurkey, K., Archer, J. R., Robinson, L., Di Francesco, A., Svenson, K. L., Korstanje, R., Harrison, D. E., Ruby, J. G., & Churchill, G. A. (2025). Analysis of lifespan across diversity outbred mouse studies identifies multiple longevity-associated loci. Genetics, 230(4), iyaf081.
Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews, 82(4), 591–605.
Paldam, M. (2015). Meta-analysis in a nutshell: Techniques and general findings. Economics, 9(1), 20150011.
Resende, P. S., Viana-Junior, A. B., Young, R. J., & Azevedo, C. S. (2021). What is better for animal conservation translocation programmes: Soft- or hard-release? A phylogenetic meta-analytical approach. Journal of Applied Ecology, 58(6), 1122–1132.
Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P. (2007). Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Medical Informatics and Decision Making, 7, 16.
Scott, A. B., Honeychurch, D., & Brin, M. F. (2023). Early development history of Botox (onabotulinumtoxinA). Medicine, 102(Suppl), e32371.
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting Meta-Analyses Based on p Values. Perspectives on Psychological Science, 11(5), 713–729.
Yuan, R., Tsaih, S.-W., Petkova, S. B., de Evsikova, C. M., Xing, S., Marion, M. A., Bogue, M. A., Mills, K. D., Peters, L. L., Bult, C. J., Rosen, C. J., Sundberg, J. P., Harrison, D. E., Churchill, G. A., & Paigen, B. (2009). Aging in inbred strains of mice: Study design and interim report on median lifespans and circulating IGF1 levels. Aging Cell, 8(3), 277–287.
Chapter 20 contents
- Additional topics
- Area under the curve
- Peak detection
- Baseline correction
- Surveys
- Time series
- Dimensional analysis
- Estimating population size
- Diversity indexes
- Survival analysis
- Growth equations and dose response calculations
- Plot a Newick tree
- Phylogenetically independent contrasts
- How to get the distances from a distance tree
- Binary classification
- Meta-analysis
12.4 – ANOVA from “sufficient statistics”
Introduction
By now you should be able to run a one-way ANOVA using R (and R Commander) with ease. As a reminder, You should also be aware that, if you need to, you could use spreadsheet software like Microsoft Excel or LibreOffice Calc to run one way ANOVA on a small data set. Still, there are times when you may need to run a one-way ANOVA on a small data set, and doing so by hand calculator may be just as convenient. What are your available options?
Following the formulas I have given would be one way to calculate ANOVA by hand, but it would be tedious and subject to error. Instead of working with the standard formulas, calculator shortcuts can be derived with a little algebra and this is where I want to draw your attention now. This technique will come in handy in lab classes or other scenarios where you collect some data among a set number of groups and calculate means and standard deviations. The purpose of this posting is to show you how to obtain the necessary statistics to calculate a one-way ANOVA from the available descriptive statistics: means, standard deviations, and sample sizes. In other words, these are the sufficient statistics for one-way ANOVA.
Note 1: In Chapter 11.5, we introduced use of summary statistics, ie, “sufficient statistics,” to calculate the independent sample t-test.
As you recall, a one-way ANOVA yields a single F test of the null hypothesis that all group means are equal. To calculate the F test, you need
- Mean Square Between Groups,

- Mean Squares Within Groups or Error,

F is then calculated as
![]()
with degrees of freedom k – 1 for the numerator and N – 1 for the denominator.
can also appear as ![]()
We can calculate
as
![]()
where k is the number of i th groups, ni is the sample size of the ith group,
refers to the overall mean for all of the
sample means.
Next, for the Error Mean Square,
, all we need is the average of the sample variances (the square of the sample standard deviation, s).
![]()
ANOVA from sufficient statistics
Consider an example data set (Table 1) for which only summary statistics are available (mean and standard deviation, sd). The data set is for life span(days) for several inbred strains of laboratory mice from a study (. Sample size for each group was seven mice.
Table 1. Descriptive statistics female life span (days) for ten different inbred strains of mice (Mus domesticus).
| Strains | n | mean | sd |
|---|---|---|---|
| 129S1/SvImJ | 32 | 787.4 | 159.16 |
| A/J | 32 | 630.7 | 130.20 |
| BALB/cByJ | 32 | 734.4 | 154.43 |
| BUB/BnJ | 24 | 611.3 | 218.34 |
| C3H/HeJ | 29 | 724.1 | 131.48 |
| C57BL/6J | 29 | 855.7 | 185.34 |
| CBA/J | 30 | 622.9 | 181.95 |
| FVB/NJ | 26 | 750.3 | 230.11 |
| P/J | 32 | 676.0 | 178.82 |
| SWR/J | 31 | 831.9 | 181.31 |
Data from Yuan et al (2021; https://phenome.jax.org).
Spreadsheet calculations.
You have several options at this point, ranging from using your calculator and the formulas above (don’t forget to square the standard deviation to get the variances!), or you could use Microsoft Excel or LibreOffice Calc and enter the necessary formulas by hand (Table 2). You’ll also find many online calculators for one-way ANOVA by sufficient statistics (eg, https://www.danielsoper.com/statcalc/calculator.aspx?id=43).
Table 2. Spreadsheet with formulas for calculating one-way ANOVA from means and standard deviations from statistics presented in Table 1.
| A | B | C | D | E | F | G | H | I | |
| 1 | Strain | n | Mean | sd | squared | variance | grand mean |
=AVERAGE(C:C) | |
| 2 | 129S1/SvImJ | 32 | 787.4 | 159.16 | =B2*(C2-$I$1)^2 | =D2^2 | dfB | =COUNT(B:B)-1 | |
| 3 | A/J | 32 | 630.7 | 130.2 | =B2*(C3-$I$1)^2 | =D3^2 | dfE | =SUM(B:B)-I2 | |
| 4 | BALB/cByJ | 32 | 734.4 | 154.43 | =B2*(C4-$I$1)^2 | =D4^2 | Msb | =SUM(E:E)/(COUNT(E:E)-1) | |
| 5 | BUB/BnJ | 24 | 611.3 | 218.34 | =B2*(C5-$I$1)^2 | =D5^2 | Mse | =SUM(F:F)/COUNT(F:F) | |
| 6 | C3H/HeJ | 29 | 724.1 | 131.48 | =B2*(C6-$I$1)^2 | =D6^2 | F | =I4/I5 | |
| 7 | C57BL/6J | 29 | 855.7 | 185.34 | =B2*(C7-$I$1)^2 | =D7^2 | P-value | =FDIST(I6,I2,I3) | |
| 8 | CBA/J | 30 | 622.9 | 181.95 | =B2*(C8-$I$1)^2 | =D8^2 | |||
| 9 | FVB/NJ | 26 | 750.3 | 230.11 | =B2*(C9-$I$1)^2 | =D9^2 | |||
| 10 | P/J | 32 | 676.0 | 178.82 | =B2*(C10-$I$1)^2 | =D10^2 | |||
| 11 | SWR/J | 31 | 831.9 | 181.31 | =B2*(C11-$I$1)^2 | =D11^2 |
For this example, you should get the following
MSB = 219899.5 MSE = 31634.1 F = 6.951 P-value = 4.45E-09
Note 2: The number of figures reported for the P-value implies a precision that the data simply do not support. For a report, recommend writing the P-value < 0.001. But note — truncating the p-value diminishes the value to others who may wish to conduct a meta analysis using your work (Chapter 20.15)!
But, R can do it better.
Here’s how. Install the HH package (or RcmdrPlugin.HH for use in Rcmdr) and call the ?aovSufficient function.
Step 1. Install the HH package from a CRAN mirror, eg, cloud.r-project.org, in the usual way.
chooseCRANmirror()
install.packages("HH")
library(HH)
Step 2. Enter the data. Do this in the usual way (eg, from a text file), or enter directly using the read.table command as follows.
MouseData <- read.table(header=TRUE, sep = ",", text= " Strain, n.size, average, stdev 129S1, 32, 787.4, 159.16 A.J, 32, 630.7, 130.20 BALB.c, 32, 734.4, 154.43 BUB.BnJ, 24, 611.3, 218.34 C3H.HeJ, 29, 724.1, 131.48 C57BL.6J, 29, 855.7, 185.34 CBA.J, 30, 622.9, 181.95 FVB.NJ, 26, 750.3, 230.11 P.J, 32, 676.0, 178.82 SWR.J, 31, 831.9, 181.31") #Check import head(MouseData)
End of R input
I know, a little hard to read, but from the MouseData to the end bracket ") before the comment line #Check import, that’s all one command.
Of course, you could copy the data and import the data from your computer’s clipboard in Rcmdr: Data → Import data → from text file, clipboard, or URL…
Note 3: Hint: for field separator, try comma; if that fails, try Tabs
Once the data set is loaded, proceed to Step 3.
Step 3. In our example, sample size was included for each group. Skip to step 4. If, however, the table lacked the sample size information, you can always add a new variable. For example, if we needed to add sample size to the data frame, we would use the repeat element function, rep().
MouseData$n <- rep(7, 10)
If you check the View data set button in Rcmdr, you will see that the command in Step 3 has added a new variable “n” for each of the eleven rows. The function rep() stands for “replicate elements in vectors” and what it did here was enter a value of 7 for each of the ten rows in the data set. Again, this step is not necessary for this example because sample size is already part of the data frame. Proceed to step 4.
Step 4. Run the one way ANOVA using the sufficient statistics and the HH function aovSufficient
MouseData.aov <- aovSufficient(average ~ Strain, data=MouseData)
Step 5. Get the ANOVA table
summary(MouseData.aov)
Here’s the R output
Df Sum Sq Mean Sq F value Pr(>F) Strain 9 1977732 219748 7.123 2.54e-09 *** Residuals 287 8853743 30849 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
End R output
To explore other features of the package, type ?aovSufficient at the R prompt (like all R functions, extensive help is generally available for each function in a package).
Meta-analysis and sufficient statistics.
Meta-analysis (see Chapter 20.15 — Meta-analysis), can be performed using sufficient statistics from different tests. Combining sufficient statistics allows for a more precise overall estimate by effectively merging the information from multiple studies, which can be especially useful when individual studies have small sample sizes or lack a unified outcome measure. Common methods include creating summary effect sizes like the standardized mean difference or using p-value combination methods, while ensuring the underlying statistical assumptions are met through goodness-of-fit tests.
Fisher’s method combines the p-values from multiple independent studies into a single overall p-value to test a joint null hypothesis. The method’s test statistic is calculated as the sum of the negative log-transformed p-values. P-values from
number of tests are combined; each p-value,
, contributes to a variable,
, with
degrees of freedom. The chi-square distribution is used to evaluate the significance of the Fisher’s method test statistic.
Fisher’s method is available in several R packages (eg,poolr package); here, we’ll just create a simple R code.
Note 4: Not to be confused with Fisher’s exact test, Chapter 9.5.
Limitations: Methods that only use p-values or test statistics may not adequately address the effects of heterogeneity and potential publication bias among studies. Because many older studies do not report effect sizes and confidence intervals, using test statistics or p-values is often the only available method to combine their results.
Example.
One hypothesis about longevity is that the greater the genetic diversity can lead to longer life. The mice in Table 1 were all from inbred mouse strains. By definition, genetic diversity is restricted in inbred mice, compared to outbred strains. Jackson Labs produced a “Diverse Outbred” line of mice, and a few individuals have lived nearly five years, well past the typical 1 – 2 year lifespan of mice (Cohen 2017, Mullis et al 2025). Thus, a natural comparison is to ask: is there a difference in average lifespan between the DO and inbred mouse strains? We’ll run a series of one-sample t-tests against the mean lifespan for the diverse outbred stock of mice, save the p-values, and apply Fisher’s method. While not strictly a meta-analysis, which would draw from multiple studies, this example outlines the basic steps to take given a series of hypothesis of the same kind.
Note 4: Used another R package, BSDA, for a series of one-sample t-tests. DO mean of 794.5 + 262.85 days (n = 192 females), calculated from supplement data in Mullis et al 2025). R code example:
do_avg= 794.5 # dataset MouseData tsum.test(average[1], stdev[1], n.size[1], mu=do_avg)
Abbreviated R results
t = -0.25235, df = 31, p-value = 0.8024
Repeat for all ten strains from Table 1; p-values were saved to Table 2.
Obviously, a for loop would be appropriate here! The R code extracts p-values from each of the ten tests and saves to p_values object.
p_values <- numeric(length(average))
for (i in 1:length(average)) {
suppressWarnings({
test_result <- tsum.test(
mean.x = average[i],
s.x = stdev[i],
n.x = n.size[i],
mu = do_avg
)
p_values[i] <- test_result$p.value
})
}
print(p_values)
I’ve wrapped the tsum.test in a suppressWarnings({}) line to prevent multiple, but harmless warnings the code causes. Use of tidyverse would help clean this code up.
The p-values from all ten tests are sorted by strain in Table 2.
Table 2. Results of one sample t-tests inbred strains (Table 1) compared to average lifespan of the J:DO diversity outbred mouse strain from Jackson labs
| Strains | p-value | Strains | p-value | |
| 129S1/SvImJ | 0.8204 | C57BL/6J | 0.08623 | |
| A/J | 5.367e-08 | CBA/J | 1.601e-05 | |
| BALB/cByJ | 0.03527 | FVB/NJ | 0.3368 | |
| BUB/BnJ | 0.0004274 | P/J | 0.0007307 | |
| C3H/HeJ | 0.007479 | SWR/J | 0.2598 |
R-code for Fisher’s method
my_pValues <- c(0.08623, 1.601e-05, 0.3368, 0.0007307, 0.2598)
X_squared <- -2 * sum(log(my_pValues),base = exp(1))
myDf <- 2 * length(my_pValues)
combinedP <- pchisq(X_squared, myDf, lower.tail = FALSE)
result_table <- as.data.frame(rbind(X_squared, myDf, combinedP))
rownames(result_table) <- c("chi_sqr", "df", "P-value")
colnames(result_table) <- c("Value")
print(result_table)
and the R output
Value
chi_sqr 44.7017431931
df 16.0000000000
P-value 0.0001541831
With the small p-value, we would reject the null hypothesis of no difference in lifespan among the inbred strains compared to the genetically diverse DO strain of mice.
Limitations of ANOVA from sufficient statistics.
This was pretty easy, so it is worth asking — Why go through the bother of analyzing the raw data, why not just go to the summary statistics and run the calculator formula? First, the chief reason against the calculator formula and use of only sufficient statistics loses information about the individual values and therefore you have no access to the residuals. The residual of an observation is the difference between the original observation and the model prediction. The residuals are important for determining whether the model fits the data well and are, therefore, part of the toolkit that statisticians need to do proper data analysis. We will spend considerable time looking at residual patterns and it is an important aspect of doing statistics correctly.
Secondly, while it is possible to extend this approach to more complicated ANOVA problems like the two-way ANOVA (Cohen 2002), the statistical significance of the interaction term(s) calculated in this way are only approximate (the main effects are OK to interpret). Thus, ANOVA from sufficient statistics has its place when all you have is access to descriptive statistics, but its use is limited and not at all the preferred option for data analysis when the original, raw observations are in hand.
Questions
1. Calculate the one-way ANOVA for body weight of 47 female (F) and 97 male (M) cats (kilograms, dataset cats in MASS R package) from the following summary statistics.
| n | Mean | sd | |
| F | 47 | 2.36 | 0.274 |
| M | 97 | 2.9 | 0.468 |
2. Bonus: Load the cats data set (package MASS, loaded with Rcmdr) and run a one-way ANOVA using the aov() function via Rcmdr. Are the ANOVA from sufficient statistics the same as results from the direct ANOVA calculation? If not, why not.
Quiz Chapter 12.4
ANOVA from “sufficient statistics”
