Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology

© 2021 The Authors. Global Change Biology published by John Wiley & Sons Ltd.

Bibliographische Detailangaben
Veröffentlicht in:Global change biology. - 1999. - 28(2022), 3 vom: 03. Feb., Seite 969-989
1. Verfasser: Yang, Yefeng (VerfasserIn)
Weitere Verfasser: Hillebrand, Helmut, Lagisz, Malgorzata, Cleasby, Ian, Nakagawa, Shinichi
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2022
Zugriff auf das übergeordnete Werk:Global change biology
Schlagworte:Journal Article climate change exaggerated effect size experimentation meta-research meta-science reproducibility second-order meta-analysis selective reporting bias small-study effect transparency
LEADER 01000caa a22002652 4500
001 NLM332743888
003 DE-627
005 20240829231911.0
007 cr uuu---uuuuu
008 231225s2022 xx |||||o 00| ||eng c
024 7 |a 10.1111/gcb.15972  |2 doi 
028 5 2 |a pubmed24n1516.xml 
035 |a (DE-627)NLM332743888 
035 |a (NLM)34736291 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Yang, Yefeng  |e verfasserin  |4 aut 
245 1 0 |a Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology 
264 1 |c 2022 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 23.02.2022 
500 |a Date Revised 29.08.2024 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a © 2021 The Authors. Global Change Biology published by John Wiley & Sons Ltd. 
520 |a Field studies are essential to reliably quantify ecological responses to global change because they are exposed to realistic climate manipulations. Yet such studies are limited in replicates, resulting in less power and, therefore, potentially unreliable effect estimates. Furthermore, while manipulative field experiments are assumed to be more powerful than non-manipulative observations, it has rarely been scrutinized using extensive data. Here, using 3847 field experiments that were designed to estimate the effect of environmental stressors on ecosystems, we systematically quantified their statistical power and magnitude (Type M) and sign (Type S) errors. Our investigations focused upon the reliability of field experiments to assess the effect of stressors on both ecosystem's response magnitude and variability. When controlling for publication bias, single experiments were underpowered to detect response magnitude (median power: 18%-38% depending on effect sizes). Single experiments also had much lower power to detect response variability (6%-12% depending on effect sizes) than response magnitude. Such underpowered studies could exaggerate estimates of response magnitude by 2-3 times (Type M errors) and variability by 4-10 times. Type S errors were comparatively rare. These observations indicate that low power, coupled with publication bias, inflates the estimates of anthropogenic impacts. Importantly, we found that meta-analyses largely mitigated the issues of low power and exaggerated effect size estimates. Rather surprisingly, manipulative experiments and non-manipulative observations had very similar results in terms of their power, Type M and S errors. Therefore, the previous assumption about the superiority of manipulative experiments in terms of power is overstated. These results call for highly powered field studies to reliably inform theory building and policymaking, via more collaboration and team science, and large-scale ecosystem facilities. Future studies also require transparent reporting and open science practices to approach reproducible and reliable empirical work and evidence synthesis 
650 4 |a Journal Article 
650 4 |a climate change 
650 4 |a exaggerated effect size 
650 4 |a experimentation 
650 4 |a meta-research 
650 4 |a meta-science 
650 4 |a reproducibility 
650 4 |a second-order meta-analysis 
650 4 |a selective reporting bias 
650 4 |a small-study effect 
650 4 |a transparency 
700 1 |a Hillebrand, Helmut  |e verfasserin  |4 aut 
700 1 |a Lagisz, Malgorzata  |e verfasserin  |4 aut 
700 1 |a Cleasby, Ian  |e verfasserin  |4 aut 
700 1 |a Nakagawa, Shinichi  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t Global change biology  |d 1999  |g 28(2022), 3 vom: 03. Feb., Seite 969-989  |w (DE-627)NLM098239996  |x 1365-2486  |7 nnns 
773 1 8 |g volume:28  |g year:2022  |g number:3  |g day:03  |g month:02  |g pages:969-989 
856 4 0 |u http://dx.doi.org/10.1111/gcb.15972  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 28  |j 2022  |e 3  |b 03  |c 02  |h 969-989