View More View Less
  • 1 Institute of Behavioural Sciences, Semmelweis University, Hungary
  • 2 National Institute of Clinical Neuroscience, Hungary
Open access

The scientific discussion of sleep spindles now routinely mentions the association between these oscillations and cognitive ability. Numerous studies have been published on the topic with various methodologies and highly divergent results. In this meta-analysis of the relevant literature (total k = 22 articles, total N = 953 subjects), it is concluded that there is evidence for a modest positive association between cognitive ability and slow (r = .113) and fast (r = .183) spindle amplitudes and also some evidence for an association between cognitive ability and slow spindle duration (r = .087) but none for associations with other parameters. Evidence for publication bias was found in case of fast spindle amplitude and density, but the correlation between cognitive ability and amplitude was robust to this effect. Studies with more females reported weaker associations with slow spindle duration, but this effect was driven by a single study with an all-male sample, and no other effect size was significantly moderated by age or sex. Most studies were conducted in small data sets and did not exhaustively report all measured associations. It is recommended that future studies having access to both sleep spindle and intelligence measures report their associations, regardless of their nature, that data sets be pooled across research groups for more statistical power, and that at least a basic agreement of spindle detection and classification criteria be reached in the research community.

Abstract

The scientific discussion of sleep spindles now routinely mentions the association between these oscillations and cognitive ability. Numerous studies have been published on the topic with various methodologies and highly divergent results. In this meta-analysis of the relevant literature (total k = 22 articles, total N = 953 subjects), it is concluded that there is evidence for a modest positive association between cognitive ability and slow (r = .113) and fast (r = .183) spindle amplitudes and also some evidence for an association between cognitive ability and slow spindle duration (r = .087) but none for associations with other parameters. Evidence for publication bias was found in case of fast spindle amplitude and density, but the correlation between cognitive ability and amplitude was robust to this effect. Studies with more females reported weaker associations with slow spindle duration, but this effect was driven by a single study with an all-male sample, and no other effect size was significantly moderated by age or sex. Most studies were conducted in small data sets and did not exhaustively report all measured associations. It is recommended that future studies having access to both sleep spindle and intelligence measures report their associations, regardless of their nature, that data sets be pooled across research groups for more statistical power, and that at least a basic agreement of spindle detection and classification criteria be reached in the research community.

Highlights

  1. Numerous single studies reported associations between IQ and sleep spindle parameters.
  2. The literature is heterogeneous and a meta-analysis was performed.
  3. Some heterogeneity, publication bias, and moderation by age and sex were found.
  4. Slow and fast spindle amplitude is significantly associated with intelligence.
  5. Results highlight the significance of thalamocortical white matter for IQ.

Introduction

It has been 17 years since the positive association between sleep spindles and general cognitive ability has first been reported (Nader and Smith, 2001). Numerous studies investigating this relationship have since been published, so that many contemporary studies concerning sleep spindles state that their significance is in part due to their association with cognitive ability (Clawson, Durkin, and Aton, 2016; Merikanto et al., 2017; Ray et al., 2015), even if they do not directly address this issue.

However, caution is warranted before accepting an omnipresent association between sleep spindles and cognitive ability at face value, for at least two reasons. The first issue is that although biological correlates of intelligence have been sought after for over a century (Deary, Penke, and Johnson, 2010; Haier, 2016), they are elusive, non-specific, and often poorly replicable. Well-replicated biological correlates of intelligence include a positive correlation with head and brain size (Pietschnig, Penke, Wicherts, Zeiler, and Voracek, 2015), neural efficiency (Neubauer and Fink, 2009), chronometric variables, specifically a smaller reaction time increment as a function of task complexity (Jensen, 2006), and gray and white matter volume in certain areas most frequently located in the frontal or parietal lobes (Jung and Haier, 2007). These findings fail to implicate any single well-delineated physiological system as a substrate of general cognitive ability. While the discovery of any well-replicated neural correlate can pave the way toward a better understanding of how higher intelligence is supported by the central nervous system, the establishment of a neural mechanism as a correlate of intelligence requires substantial evidence.

The second issue is that both psychological and biomedical literatures are fraught with false-positive findings, which do not hold up under closer scrutiny or simply fail independent replication (Ioannidis, 2005a, 2005b; Open Science Collaboration, 2015). Journal editors, funding agencies, university boards, and to a certain degree researchers themselves are highly averse to both replications and negative findings, resulting in the proliferation of positive findings in publication, especially among small studies (Giner-Sorolla, 2012). Furthermore, small pilot studies reporting large effects often keep getting cited after larger studies with more modest or negative findings are published (Ferguson and Heene, 2012; Ioannidis, 2005a), while small studies with negative results usually never see the light of day (Sterne, Gavaghan, and Egger, 2000). The study of polysomnographic measures as correlates of psychological traits is a field that is plagued by many characteristics, which are conductive for poor reproducibility: both sleep EEG and psychological traits are relatively noisy measurements with typical maximal test–retest reliability of ≈0.8 (Tan, Campbell, and Feinberg, 2001), expected effect sizes are small, sleep EEG is costly and difficult to measure, usually involving at least two consecutive full nights of cooperation by each subject and a qualified research assistant, and funding is limited due to the expectedly indirect translational effects of any results.

As a result, most polysomnography studies about the possible association between sleep EEG measures and cognitive ability were very small – the median sample size in the studies involved in this meta-analysis is 24. The smallest correlation coefficient that can be detected with N = 24 with 80% power is 0.55 [Power calculations in the paper are based on the following formula: N = [(Zα + Zβ)/C]2 + 3, where Zα and Zβ are the standard normal deviates of the critical p value (.05) and 1-power, respectively, and C = 0.5 × ln[(1 + r)/(1 − r)], where r is the correlation coefficient. These calculations were performed using the online calculator available at http://www.sample-size.net/correlation-sample-size/]. Assuming a reliability of 0.8 for both sleep EEG measures and intelligence tests, the maximum possible correlation strength (if intelligence depended on nothing but something directly reflected by sleep EEG measures) would be 0.82 = 0.64. Therefore, unless sleep EEG measures are the single best biological correlates of intelligence, accounting for almost all observable variance in test scores, most previous studies were underpowered to reliably detect their association.

Methodological problems of the polysomnography research of psychological traits are exacerbated by how many “researcher degrees of freedom” (Wicherts et al., 2016) are available for experts of polysomnography: due to the complexity of such research, subjects are usually extensively phenotyped, they participate in numerous tests, experiments, and other procedures, while EEG data can also be analyzed in an almost infinite different ways, especially if many channels are available. As a result, it is almost always possible to find a combination of polysomnography features, which correlate with a psychological trait in a given sample.

Underpowered studies, however, are not useless: subject to a meta-analysis, their findings aggregate and approximate the true underlying effect size of the investigated association. Therefore, in this study, a meta-analysis of all available studies addressing the correlation between sleep spindle measures and intelligence was performed in order to establish whether a true effect is present and whether its effect size is subject to the moderating effects of age and sex.

Methods

Study selection

A non-systematic, but extensive revision of previous scientific papers about the association between sleep spindles and cognitive ability is available in the study of Ujma et al. (2014). In this study, the list of studies [k = 13 including Ujma et al. (2014) itself] presented therein as an initial catalog of relevant literature was used, which also included effect sizes not reported in the original studies but later procured by contacting the authors. This list was expanded by searching PubMed using the key words “sleep spindle” and “intelligence,” as well as Google Scholar using the same key words. PubMed yielded 31 results, all of which were considered for inclusion based on the title and abstract. Google Scholar yielded 12,400 results, of which the first 100 were considered for inclusion. Furthermore, all papers citing the ones previously identified were manually reviewed using PubMed. One paper (Fang, Ray, Owen, and Fogel, 2017) was previously found by the author on Bioarxiv, whereas another one based on the Finnish GLAKU birth cohort (Halonen et al., in preparation) is part of a collaboration the author is currently involved in. For some papers, no effect size was reported and the authors were unavailable for providing this data, including an unpublished MA thesis (McCreesh, 2016). Therefore, missing effect sizes from these papers were not included in the meta-analysis. Study selection is illustrated in Fig. 1.

Fig. 1.
Fig. 1.

PRISMA chart of study identification and selection. Note that k in this chart indicates papers, while in the rest of this meta-analysis, it refers to the number of studies, as several papers contain more than one study

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

Several of the identified studies did not report effect sizes. The authors of these studies were contacted by e-mail, and in case of a positive answer, their effect sizes were included in the meta-analysis.

All identified papers with the reported effect sizes (Bódizs et al., 2005; Bódizs, Gombos, Ujma, and Kovács, 2014; Chatburn et al., 2013; Fang, Ray, et al., 2017; Fang, Sergeeva, et al., 2017; Fogel and Smith, 2006; Fogel, Nader, Cote, and Smith, 2007; Geiger et al., 2011; Gruber et al., 2013; Hoedlmoser et al., 2014; Lustenberger, Maric, Durr, Achermann, and Huber, 2012; Nader and Smith, 2015; Peters, Smith, and Smith, 2007; Peters, Ray, Smith, and Smith, 2008; Schabus et al., 2006; Tessier et al., 2015; Tucker and Fishbein, 2009; Ujma et al., 2014; Ujma, Bódizs, et al., 2015; Ujma, Sandor, Szakadat, Gombos, and Bodizs, 2016; Ward, Peters, and Smith, 2013) and the studies therein are summarized in Table 1. One paper with a known effect size (Clemens, Fabo, and Halasz, 2006) was excluded, because its sample completely overlapped with another (Bódizs et al., 2005).

Table 1.

List of studies involved in meta-analytic calculations, with cognitive tests, spindle detection methods, and sample size (N)

StudyIQ testSpindle detectorMean ageAge rangeRecruitmentNFemale (%)AmplitudeDensityDurationFrequencyTypeElectrode
Bódizs et al. (2005)aRPMTIAM37.3227–67No post-secondary education1926−0.117SlowC3+C4
Bódizs et al. (2005)aRPMTIAM37.3227–67No post-secondary education19260.577FastC3+C4
Schabus et al. (2006)APMSIESTA2420–30University48500.390.060.09SlowC3
Schabus et al. (2006)APMSIESTA2420–30University48500.35−0.010.34FastC3
Tucker and Fishbein (2009)bMAB-II FSIQPSD21Not reportedUniversity2450−0.176Not specifiedC3+C4
Geiger et al. (2011)bWISC-IV FSIQPSD10.59–13Eight normal and six gifted children14430.67−0.56Not specifiedC3
Fogel and Smith (2006)cMAB-II PIQVisual/PSD22.520–25University12100−0.176Not specifiedC3+C4
Fogel et al. (2007)b,c: Study 1MAB-II FSIQVisual/PSD23.518–29University101000.67Not specifiedC3+C4
Fogel et al. (2007)b,c: Study 2MAB-II PIQVisual/PSD22.520–25University121000.71Not specifiedC3+C4
Fogel et al. (2007)b,c: Study 3MAB-II PIQVisual/PSD2018–26University35820.760.76Not specifiedCz
Peters et al. (2007)MAB-II FSIQVisual21Not reportedUniversity2450−0.11Not specifiedC3+C4
Peters et al. (2008): Study 1MAB-II FSIQVisual2017–24University1450−0.25Not specifiedC3+C4
Peters et al. (2008): Study 2MAB-II FSIQVisual7062–79Local community1450−0.15Not specifiedC3+C4
Lustenberger et al. (2012)bZVTPSD21.419–23Not reported1500.550Not specifiedC4
Chatburn et al. (2013)SBIS FSIQWaveform-based84–13National average SES2748−0.0150.038−0.025SlowC3
Chatburn et al. (2013)SBIS FSIQWaveform-based84–13National average SES2748−0.1290.0250.011FastC3
Hoedlmoser et al. (2014)WISC-IV FSIQSIESTA9.568–11Public elementary schools54460.2370.3630.285−0.25SlowCz
Hoedlmoser et al. (2014)WISC-IV FSIQSIESTA9.568–11Public elementary schools54460.167−0.2040.001−0.068FastCz
Gruber et al. (2013)dWISC-IV WM+PRFixF97–11Middle-class SES2948−0.561Not specifiedSeveral
Ward et al. (2013)bMAB-II FSIQPRANA2118–29University30700.220Not specifiedC3
Ujma et al. (2014)aAPM and CFTIAM29.717–69Mixed160450.0930.1980.1030.031SlowCz
Ujma et al. (2014)aAPM and CFTIAM29.717–69Mixed160450.094−0.0780.005−0.014FastCz
Bódizs et al. (2014)aAPMIAM1815–22Convenience23500.175−0.093−0.1700.182SlowCz
Bódizs et al. (2014)aAPMIAM1815–22Convenience23520.3880.1630.2010.163FastCz
Ujma, Bódizs, et al. (2015)eCFTIAM23.2918–30University790−0.0410.2400.297−0.059SlowC3+C4
Ujma, Bódizs, et al. (2015)eCFTIAM23.2918–30University7900.1170.0560.123−0.178FastC3+C4
Ujma et al. (2016)aCPMIAM6.214–8Mixed, middle class28540.389−0.192−0.036−0.073SlowCz
Ujma et al. (2016)aCPMIAM6.214–8Mixed, middle class28540.2820.079−0.02−0.055FastCz
Nader and Smith (2015)WISC-IV/WAIS-III FSIQPRANA15.3612–19Convenience3253−0.027SlowC3+C4
Nader and Smith (2015)WISC-IV/WAIS-III FSIQPRANA15.3612–19Convenience3253−0.303FastC3+C4
Tessier et al. (2015)bWISC-III FSIQPSD9.786–13Control subjects1800.27SlowC3
Tessier et al. (2015)bWISC-III FSIQPSD9.786–13Control subjects1800.33FastC3
Fang, Sergeeva, et al. (2017fCBS Trials and reasoningCD&IN2619–40Not reported27660.069−0.0470.002SlowFz
Fang, Sergeeva, et al. (2017fCBS Trials and reasoningCD&IN2619–40Not reported27660.3500.374−0.118FastPz
Fang, Ray, et al. (2017)CBS Trials and reasoningCD&IN2420–35Not reported2959−0.028−0.04−0.248SlowFz
Fang, Ray, et al. (2017)CBS Trials and reasoningCD&IN2420–35Not reported29590.019−0.347−0.01FastPz
Halonen et al. (in preparation)WAIS-III FSIQIAM17No varianceBirth cohort176570.048−0.012−0.0190.029SlowC3+C4
Halonen et al. (in preparation)WAIS-III FSIQIAM17No varianceBirth cohort176570.199−0.0540.0070.033FastC3+C4

Note. Multiple electrode names indicate the Fisher’s Z-transformed, averaged and back-transformed value of the correlation coefficients obtained from the electrodes. Recruitment refers to how or based on what criteria participants were recruited. RPMT: Raven’s Standard Progressive Matrices; APM: Advanced Progressive Matrices; MAB: Multidimensional Aptitude Battery; WISC: Wechsler Intelligence Scale for Children; ZVT: Zahlen-Verbindungs-Test (number connection test); SBIS: Stanford-Binet Intelligence Scale; CFT: Culture Fair Test; CPM: Colored Progressive Matrices; CD&IN: complex demodulation and individualized normalization; IAM: Individual Adjustment Method; PRANA: PRANA: Polygraphic Recording Analyzer; WAIS: Wechsler Adult Intelligence Scale; CBS: Cambridge Brain Sciences; FSIQ: Full Scale IQ; PIQ: performance IQ; PSD: power spectral density; FixF: spindle detection based on fixed frequencies and a distribution percentage cutoff. All spindle detection methods are described in the relevant papers.

aAge-corrected partial correlation. bSigma power instead of spindle amplitude, also indicated by “PSD” under spindle detector. cNumber of spindles instead of spindle density. dThis correlation is the average between working memory/perceptual reasoning and spindle frequency, calculated from the coefficients of determination given in Fig. 3 of the original study. eAfternoon nap sleep. fMean correlation (using Fisher’s transformation) of N2 and slow-wave sleep values.

Outcome measures

Various measures of general cognitive ability and many different sleep spindle detection methods were used in the identified studies. Cognitive ability tests mostly measure the underlying g factor (Jensen, 1998), which is especially true for the non-verbal Raven tests implemented by some studies. In order to ensure the highest possible degree of compatibility between the studies, Full-Scale IQ (FSIQ) was used in this study as a measure in the studies where multiple cognitive ability subtest scores were available and FSIQ was reported. Thus, this meta-analysis specifically concerns the association between sleep spindles and g and not verbal abilities, working memory, or other lower level abilities. For the lack of an agreed-upon standard of sleep spindle analysis criteria, no study selection was performed for the various spindle detection methods implemented in different studies. Instead, a potential effect of sleep spindle detectors was expected to be reflected by between-study heterogeneity. Effect sizes of interest were the simple, uncorrected Pearson’s correlations between sleep spindle measures (density, duration, amplitude, and frequency) and IQ, except for studies with a wide age range among subjects (specified in Table 1) where age-corrected partial correlations were used. Sleep spindle measures were preferably obtained from the electrode Cz, but due to different study designs some deviations were made, which are noted in Table 1. Some studies only measured a subset of possible sleep spindle measures, which in this case were treated as missing data in the analyses concerning the others. Studies only reporting sigma power were included in the analyses of sleep spindle amplitude (see an explanation for this later in this section). If a study distinguished between slow and fast spindles, the effect sizes were entered accordingly, otherwise they were listed as “non-specified” and excluded from analyses concerning only slow or fast spindles. All comparisons were run for both strictly defined slow and fast spindles and for the latter in combination with non-specified spindles. A sex-pooled reanalysis of the data sets previously used in the author’s publications (Bódizs et al., 2014; Ujma, Bódizs, et al., 2015; Ujma et al., 2014, 2016) was performed in order to be compatible with the other studies the meta-analysis.

Some empirical justifications must be made of the smoothing over of methodological differences reported above. These will be based on the reanalysis of a large data set (N = 159) of all-night polysomnography recordings, from which full-scalp spindle detections and power spectrum calculations are available.

First, spindle measures on various scalp electrodes (P3, P4, Fz, C3, C4, and Cz) are strongly correlated, with typical intersite correlations well over 0.9. The distribution of all correlations by spindle measure is given in Supplementary Fig. S1. Therefore, interstudy differences in the specific electrode selected for spindle analyses are unlikely to significantly bias the meta-analysis.

Second, both the raw sigma power (r = .76) and the 10-base logarithm of sigma power (r = .82) on the representative electrode C4 are very strongly correlated with fast spindle amplitude on the same site and much modestly with other spindle measures (rmax = .4481 with fast spindle density). Furthermore, this correlation is still very strong with sleep spindle activity (Duration × Amplitude), yielding r = .61 between log sigma power and slow spindle activity and r = .86 between log sigma power and fast spindle activity. These correlations are much more modest with relative power due to the removal of baseline interindividual differences in voltage (rfast_density = .30; rfast_amplitude = .38, all other correlations <0.15 and non-significant). The full set of correlations is reported in Supplementary Table S1. Thus, studies using only sigma power or spindle activity can be presumed to have mostly approximated fast spindle amplitude and will be included in the meta-analysis as such. Due to this dominance of fast spindles in sigma range activity, non-specified spindle density and frequency were also meta-analyzed together with fast spindles, but a separate analysis of strictly defined fast spindles has always been provided.

Statistics

Meta-analytic effect sizes are based on the weighted mean of individual effect sizes reported by single studies. Weights can be assigned based on the assumption of fixed and random effects. In short, the difference of these approaches is whether there is meaningful methodological heterogeneity in the studies. A fixed-effects model assigns great weight to the largest studies, and it is more appropriate if all studies measure exactly or almost exactly the same outcome. In contrast, a random-effects model treats studies with more equal weight, and it is more appropriate when an imperfect overlap of the reported effect sizes is expected due to methodological heterogeneity. Meta-analytic effect sizes in this study were calculated using the weighted, Fisher’s Z-transformed average of effect sizes (correlation coefficients). Both fixed-effects and random-effects models were calculated and reported.

Between-study heterogeneity – the proportion of effect size variance not explained by sampling error, but a difference in the underlying effect resulting from a true heterogeneity of the study design, such as different signal detection methods or a different mean age of subjects in the sample – was estimated using the Q and I2 statistics. Q is the weighted sum of squared differences between individual study effects and the pooled effect across studies, which follows a χ2 distribution and a significance level can be directly assigned to it. I2 is a function of Q, with I2 = 100 × (Q − df)/Q, expressed as a percentage and as a result with a lower bound of 0. Low I2 and Q statistics indicate that the methodological differences of the meta-analyzed studies did not substantially contribute to the differences of the reported effect sizes, which can therefore be assumed to be approximations of the same underlying effect, and their differences are due to sampling error.

The presence of publication bias – defined as a systematic absence of some performed studies from the literature, such as when small studies are only published when they find significant effects by chance – was assessed using two similar methods: Begg and Mazumdar’s (1994) correlation and Egger’s regression (Egger, Davey Smith, Schneider, and Minder, 1997). The two methods have a similar philosophy: smaller studies are subject to more sampling error, but this should not be systematically positive or negative – there should be about an equal number of overestimated and underestimated true effect sizes in smaller studies. Formally, this would mean that effect sizes and their standard errors (which are a function of sample size) are uncorrelated. A significant correlation between effect sizes and standard errors would mean that results were not systematically published. Instead, only those small studies were published, which found a significant association; therefore, the larger the standard error, the larger the effect size necessary to get a study published. Begg and Mazumdar’s correlation estimates this by computing the rank correlation (Kendall’s τ) between standardized effect sizes and their variances, whereas Egger’s method performs a weighted least-square regression (weighted by the inverse of the variance) with the standardized effect sizes as the dependent variable and precision (1/standard error) as a covariate. A non-significant Begg and Mazumdar’s correlation or Egger’s regression coefficient suggests that there is no evidence for publication bias, although a low number of available studies may result in insufficient statistical power for these procedures. Publication bias is often illustrated by funnel plots, which are scatterplots of standardized effect sizes against standard errors, the latter usually plotted on inverted axes, so that studies with smaller standard errors – i.e., larger samples – appear at the top of the plot. Effect sizes are expected to be spread out in a funnel-like shape with increased scattering as a function of standard error. However, this “funnel” is supposed to be symmetrical; if it is not the case, then the likely explanation is that small studies with null findings or significant findings in the opposite direction of what would be expected do not appear in the literature.

Finally, the effects of age (defined as the mean age of subjects in a study) and sex ratio (defined as the percentage of females among the subjects in a study) were estimated using metaregression. Metaregression regresses the effect sizes against the values of moderator variables, finding out whether the reported associations are stronger as a function of a moderator variable, for instance, whether stronger associations between two variables are systematically found in older subjects. In all cases, three models were built: Model 1 incorporated an intercept and mean age as a covariate; Model 2 incorporated an intercept and sex ratio as a covariate; and Model 3 incorporated an intercept and both age and sex ratio as covariates. The summary tables in this study always report the significance of predictors from Model 3. All reported metaregression models were run using random effects.

All statistical analyses were implemented in Comprehensive Meta-analysis V3 (Biostat Inc., Englewood, NJ, USA).

Results

Amplitude

Data about spindle amplitude were available from 10 studies (N = 642) with strictly defined slow and fast spindles. Another six studies reported data about sigma power, which – as described in the “Methods” section – was meta-analyzed together with fast spindle amplitude (k = 16, N = 752).

Both explicitly identified slow (r = .113, p = .005) and fast (r = .183, p < .001) spindle amplitudes had a significant meta-analytic correlation with cognitive ability, with no substantial between-study heterogeneity, and no evidence for publication bias. Including the seven studies with sigma power data, between-study heterogeneity (I2 = 39.12) and publication bias became significant. An inspection of the funnel plot (Fig. 2d) revealed that both between-study heterogeneity and publication bias were driven by four small studies, two of them published in the same article (Nmax = 15) reporting very high effect sizes (r = .55 – .76). Note that these studies had both the lowest sample sizes and the strongest reported effect sizes, and the rank correlation of sample size and effect size was unity even within these four studies, while other small studies (Gruber et al., 2013; Peters et al., 2007, 2008) including an unpublished MA thesis (McCreesh, 2016) either found no significant association and did not report an effect size or did not assess spindle amplitude, excluding these studies from the meta-analysis and resulting in funnel plot asymmetry.

Fig. 2.
Fig. 2.

Forest plots for the Z-transformed correlations with 95% confidence intervals between slow (a) and fast and non-specified (c) spindle amplitudes and IQ. Funnel plots panels [slow spindles (b) and fast and non-specified spindles (d)] illustrate the association between effect size and standard error, which should be zero under the assumption of no publication bias. The diameter of markers on the forest plots is proportional to the sample size. Average effect sizes on forest plots are indicated with diamond shapes and they are based on a fixed-effects model. On the funnel plot, selected high-leverage studies are marked with a reference label. Funnel plot boundaries are based on the standard error

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

Excluding these four studies from the meta-analysis resulted in a still large sample (k = 12, N = 701), the absence of both heterogeneity (I2 = 0%) and publication bias (p > .05 for both Egger’s and Begg and Mazumdar’s methods), and a somewhat reduced but still significant positive meta-analytic correlation (r = .166, p < .001). The moderating effects of age and sex ratio were not significant for either comparison.

All meta-analytic effect sizes reported in-text refer to fixed-effects models, but random-effects models reported very similar findings. Detailed statistics for all models (including the different inclusion criteria for fast spindle amplitude) are reported in Table 2. The findings are illustrated in Fig. 2.

Table 2.

Meta-analytic correlations between sleep spindle amplitude and cognitive ability with heterogeneity statistics, publication bias statistics, and moderator effects

AmplitudeSlowFastFast and non-specifiedFast and non-specified-trimmed
Basic resultsk10101612
N642642752701
r (fixed effects).113.183.199.166
r (random effects).123.183.234.166
p (fixed effects).005<.001<.001<.001
p (random effects).008<.001<.001<.001
HeterogeneityQ10.7296.3224.649.5
p(Q).295.707.055.576
I2(%)16.12039.120
Publication biasEgger’s regression p.18.1.01.57
Begg and Mazumdar’s p.18.08.004.41
Moderatorsp (linear effect of age).24.5.46.31
p (linear effect of % female).55.83.5.96

Note. Significant effects are represented in bold. For publication bias and moderator effects, only the p values of the relevant statistical measures are reported; see Supplementary Material for detailed analysis results. Slow and fast refer to strictly defined slow and fast spindles, respectively. Fast and non-specified refers to a model in which strictly defined fast spindles and non-specified spindles were meta-analyzed together. Fast and non-specified-trimmed refers to a model in which four small studies with large effect sizes were removed (see text for details).

In sum, both slow and fast spindle amplitudes had a positive meta-analytic correlation with cognitive ability. The apparent between-study heterogeneity and publication bias in case of fast spindle amplitude were caused by the fact that small studies failed to report effect sizes unless they were significant, but both issues were resolved after excluding the smallest studies.

Density

Data about explicitly defined slow and fast spindle densities were available from 12 studies (N = 700). Another eight studies reported results about the relationship between either the total count (Fogel and Smith, 2006; Fogel et al., 2007) or the density (Peters et al., 2007, 2008; Ward et al., 2013) of generic “spindles” (without the identification of slow or fast subtypes) and cognitive ability, which were meta-analyzed together with fast spindles in a different, less selective model (k = 20, N = 853), rendering it the spindle measure with the most available data.

Slow spindle density was significantly associated with cognitive ability using the fixed-effects (r = .087, p = .024) but not with the random-effects model (r = .078, p = .11). Between-study substantial was not substantial (I2 = 25.73%), and there was no evidence for publication bias and no moderating effect of age or sex ratio. There was no significant meta-analytic correlation between fast spindle density and cognitive ability, but in this case, between-study heterogeneity was substantial (I2 = 47.43%). This effect was even stronger when the less restrictive model, including non-specified spindles, was used (I2 = 61.12%), and this method also induced publication bias.

An inspection of the funnel plot (Fig. 3d) shows that publication bias is mainly caused by three small studies from two articles (Fogel and Smith, 2006; Fogel et al., 2007) reporting very high effect sizes. As these studies did not specify the type of spindles, they do not appear in the analyses exclusively focusing on strictly defined fast spindles, where there is consequently no evidence for either publication bias. Detailed statistics are provided in Table 3, whereas findings are illustrated in Fig. 3.

Fig. 3.
Fig. 3.

Forest plots for the Z-transformed correlations with 95% confidence intervals between slow (a) and fast and non-specified (c) spindle densities and IQ. Funnel plots panels [slow spindles (b) and fast and non-specified spindles (d)] illustrate the association between effect size and standard error, which should be zero under the assumption of no publication bias. The diameter of markers on the forest plots is proportional to the sample size. Average effect sizes on forest plots are indicated with diamond shapes and they are based on a fixed-effects model. On the funnel plot, selected high-leverage studies are marked with a reference label. Funnel plot boundaries are based on the standard error

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

Table 3.

Meta-analytic correlations between sleep spindle density and cognitive ability with heterogeneity statistics, publication bias statistics, and moderator effects

DensitySlowFastFast and non-specified
Basic resultsk121220
N700700853
r (fixed effects).087−.037.001
r (random effects).078−.02.065
p (fixed effects).024.337.981
p (random effects).11.741.313
HeterogeneityQ14.8120.92448.862
p(Q).191.034<.001
I2(%)25.7347.4361.12
Publication biasEgger’s regression p.21.36.03
Begg and Mazumdar’s p.06.14.02
Moderatorsp (linear effect of age).95.14.98
p (linear effect of % female).1.68.19

Note. Significant effects are represented in bold. For publication bias and moderator effects, only the p values of the relevant statistical measures are reported; see Supplementary Material for detailed analysis results. Slow and fast refer to strictly defined slow and fast spindles, respectively. Fast and non-specified refers to a model in which strictly defined fast spindles and non-specified spindles were meta-analyzed together.

In sum, on average, there is limited evidence for a modest association between slow spindle density and cognitive ability. There is no meta-analytic evidence that fast sleep spindle density is associated with cognitive ability.

Duration

Data about spindle duration were available from 10 studies (N = 651), all of which explicitly identified slow and fast spindle duration.

Across-study heterogeneity was low, with non-significant Q statistics for both slow and fast spindles. There was no evidence for publication bias. Neither slow nor fast spindle duration had a significant meta-analytic correlation with cognitive measures. Age and sex ratio had no significant moderating effect on the correlations with fast spindle duration, but a significant negative association was found between the proportion of female subjects and effect sizes. This relationship, however, was mainly driven by a single study (Ujma, Bódizs, et al., 2015) with only male subjects and a relatively high correlation coefficient reported in nap sleep (Fig. 4).

Fig. 4.
Fig. 4.

Scatterplot of the Fisher’s Z-transformed correlation coefficients between slow sleep spindle duration and IQ as a function of the percentage of female subjects in the study. A reference label marks the single study driving this association

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

Findings about spindle duration are presented in Table 4 and Fig. 5.

Table 4.

Meta-analytic correlations between sleep spindle duration and cognitive ability with heterogeneity statistics, publication bias statistics, and moderator effects

DurationSlowFast
Basic resultsk1010
N651651
r (fixed effects).073.046
r (random effects).072.046
p (fixed effects).069.255
p (random effects).166.255
HeterogeneityQ12.6436.717
p(Q).179.667
I2(%)28.820
Publication biasEgger’s regression p.55.6
Begg and Mazumdar’s p.28.47
Moderatorsp (linear effect of age).77.92
p (linear effect of % female).007.41

Note. Bold value represents the significant effects. For publication bias and moderator effects, only the p values of the relevant statistical measures are reported; see Supplementary Material for detailed analysis results. Slow and fast refer to strictly defined slow and fast spindles, respectively.

Fig. 5.
Fig. 5.

Forest plots for the Z-transformed correlations with 95% confidence intervals between slow (a) and fast and non-specified (c) spindle durations and IQ. Funnel plots panels [slow spindles (b) and fast and non-specified spindles (d)] illustrate the association between effect size and standard error, which should be zero under the assumption of no publication bias. The diameter of markers on the forest plots is proportional to the sample size. Average effect sizes on forest plots are indicated with diamond shapes and they are based on a fixed-effects model. On the funnel plot, selected high-leverage studies are marked with a reference label. Funnel plot boundaries are based on the standard error

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

In sum, results from all studies meaningfully converge on the absence of an association between sleep spindle duration and cognitive ability. In the absence of more studies with a low, but non-zero proportion of female subjects and nap sleep studies, it is unclear whether the negative correlation between the effect sizes and the proportion of female subjects is spurious or driven by the difference between nap sleep and night sleep.

Frequency

Data about strictly defined slow and fast spindle frequencies were available from seven studies (N = 547). Two further studies reported data about spindles without a specified subtype, which were meta-analyzed together with fast spindles, resulting in a slightly larger sample (k = 9, N = 590).

Neither strictly defined slow or fast spindle frequency was significantly associated with cognitive ability. Heterogeneity was low, there was no evidence for publication bias and the effect of moderator variables was not significant. A combined meta-analysis of fast and non-specified spindles similarly resulted in non-significant meta-analytic correlations, but higher between-study heterogeneity (I2 = 50.69%), although there was still no evidence for either publication bias or the moderating effects of sex ratio and age.

Detailed statistics are reported in Table 5. The findings are illustrated in Fig. 6.

Table 5.

Meta-analytic correlations between sleep spindle frequency and cognitive ability with heterogeneity statistics, publication bias statistics, and moderator effects

FrequencySlowFastFast and non-specified
Basic resultsk779
N547547590
r (fixed effects)−.012−.022−.062
r (random effects)−.012−.022−.102
p (fixed effects).781.617.141
p (random effects).781.617.133
HeterogeneityQ4.6383.27616.225
p(Q).591.773.039
I2(%)0050.69
Publication biasEgger’s regression p.58.93.16
Begg and Mazumdar’s p.88.65.53
Moderatorsp (linear effect of age).21.78.23
p (linear effect of % female).35.11.29

Note. Significant effects are represented in bold. For publication bias and moderator effects, only the p values of the relevant statistical measures are reported; see Supplementary Material for detailed analysis results. Slow and fast refer to strictly defined slow and fast spindles, respectively. Fast and non-specified refers to a model in which strictly defined fast spindles and non-specified spindles were meta-analyzed together.

Fig. 6.
Fig. 6.

Forest plots for the Z-transformed correlations with 95% confidence intervals between slow (a) and fast and non-specified (c) spindle frequencies and IQ. Funnel plots panels [slow spindles (b) and fast and non-specified spindles (d)] illustrate the association between effect size and standard error, which should be zero under the assumption of no publication bias. The diameter of markers on the forest plots is proportional to the sample size. Average effect sizes on forest plots are indicated with diamond shapes and they are based on a fixed-effects model. On the funnel plot, selected high-leverage studies are marked with a reference label. Funnel plot boundaries are based on the standard error

Citation: Sleep Spindles & Cortical Up States Sleep Spindles & Cortical Up States 2021; 10.1556/2053.2.2018.01

In sum, no evidence was found for a significant association between spindle frequency and cognitive ability. The reliability of this finding was supported by the lack of substantial between-study heterogeneity, publication bias, and moderator effects.

Discussion

Spindle amplitude alone is unambiguously associated with cognitive ability

This meta-analysis of the extant literature was designed to clarify the obvious discrepancies between many of the previously reported single studies. A robust association was found between sleep spindle amplitude and general cognitive ability: its magnitude, however, is more modest than previously estimated by single studies, approximating r = .15. An even more modest correlation was found between slow spindle density and cognitive ability, which was, however, significant only assuming fixed effects, and not with random effects, and unlike amplitude, it was not robust to the exclusion of all studies (Table 6). Therefore, further study and larger meta-analyses are necessary to establish a true modest positive association between slow spindle density and cognitive ability. Studies about all spindle measures significantly associated with intelligence are summarized in Table 6. These tables contain the estimated meta-analytic effect sizes with the exclusion of each individual study in order to demonstrate the robustness of the findings, which is substantial for spindle amplitude but depends on single studies in case of slow spindle density.

Table 6.

One-study-excluded statistics for the spindle parameters associated with intelligence

MeasurePoint estimate without study95% CIZ scorep
Amplitude (fast and non-specified)
Schabus et al. (2006)Fast0.1880.1140.2614.894<.001
Tucker and Fishbein (2009)Non-specified0.2100.1380.2815.580<.001
Geiger et al. (2011)Non-specified0.1900.1170.265.059<.001
Fogel et al. (2007): Study 1Non-specified0.1910.1190.2625.118<.001
Fogel et al. (2007): Study 2Non-specified0.1910.1190.2625.100<.001
Fogel et al. (2007): Study 3Non-specified0.2060.1330.2775.426<.001
Lustenberg et al. (2012)Non-specified0.1920.1200.2635.119<.001
Hoedlmoser et al. (2014)Fast0.2020.1270.2745.224<.001
Ujma et al. (2014)Fast0.2290.1480.3065.442<.001
Bódizs et al. (2014)Fast0.1930.120.2645.120<.001
Ujma, Bódizs, et al. (2015)Fast0.2090.1330.2825.314<.001
Ujma et al. (2016)Fast0.1960.1230.2675.175<.001
Tessier et al. (2015)Fast0.1960.1230.2675.217<.001
Fang, Sergeeva, et al. (2017)Fast0.1940.120.2655.112<.001
Fang, Ray, et al. (2017)Fast0.2060.1330.2775.438<.001
Halonen et al. (in preparation)Fast0.1990.1160.2794.656<.001
Total effect (fixed effects)0.1990.1270.2695.355<.001
Amplitude (slow)
Schabus et al. (2006)Slow0.090.0080.1712.140.032
Hoedlmoser et al. (2014)Slow0.1020.0190.1832.414.016
Ujma et al. (2014)Slow0.120.0290.2092.571.01
Bódizs et al. (2014)Slow0.1110.0310.1902.710.007
Ujma, Bódizs, et al. (2015)Slow0.1350.0510.2173.136.002
Ujma et al. (2016)Slow0.1010.0200.1802.444.015
Tessier et al. (2015)Slow0.1090.0290.1872.674.007
Fang, Sergeeva, et al. (2017)Slow0.1150.0350.1942.798.005
Fang, Ray, et al. (2017)Slow0.1190.0390.1982.901.004
Halonen et al. (in preparation)Slow0.1390.0460.2292.924.003
Total effect (fixed effects)0.1130.0340.1902.809.005
Density (slow)
Bódizs et al. (2005)Slow0.0920.0160.1682.357.018
Schabus et al. (2006)Slow0.0890.0110.1662.228.026
Chatburn et al. (2013)Slow0.0910.0140.1672.312.021
Hoedlmoser et al. (2014)Slow0.063−0.0160.1411.565.118
Ujma et al. (2014)Slow0.052−0.0340.1381.184.236
Bódizs et al. (2014)Slow0.0930.0160.1682.363.018
Ujma, Bódizs, et al. (2015)Slow0.067−0.0140.1471.631.103
Ujma et al. (2016)Slow0.0980.0210.1742.491.013
Nader and Smith (2015)Slow0.0920.0150.1692.338.019
Fang, Sergeeva, et al. (2017)Slow0.0920.0150.1682.342.019
Fang, Ray, et al. (2017)Slow0.0920.0150.1682.342.019
Halonen et al. (in preparation)Slow0.1220.0340.2082.715.007
Total effect (fixed effects)0.0870.0110.1622.256.024

Note. The table contains the meta-analytic correlation of spindle parameters with intelligence with the exclusion of each study, its confidence intervals, Z-transformed values, and its significance level. CI: confidence interval.

While many other single studies found promising associations between other spindle measures and cognitive ability, most frequently frequency or fast spindle density, these associations did not hold up in a meta-analysis of the literature and therefore they can most likely be considered false positives. Similarly, no well-replicated association between spindle duration and cognitive ability was found.

The robustness of these findings is indicated by several measures: the relatively large number of subjects (N > 500 even in the smallest analyses); the low – often non-existent – between-study heterogeneity, despite obvious differences in the methodology including but not limited to the application of very different spindle detectors; and the general absence of publication bias. While publication bias was found to inflate the estimated meta-analytic correlations between cognitive ability, spindle amplitude, and spindle density, this effect was the result of a few small published studies with large effects, while other small studies generally failed to report their non-significant and presumably much lower effect sizes. While direct mathematical corrections of publication bias such as precision-effect test and precision-effect estimate with standard errors (Van Elk et al., 2015) have been forgone due to the shortcomings of such methods (Stanley, 2017), the simple exclusion of these studies resulted in an unbiased collection of effect sizes without substantial loss in statistical power.

Notably, the moderating effects of age and sex for other comparisons were generally small. While all moderating effects were tested using linear regressions, the presence of non-linear effects is not supported by the inspection of the scatterplots of effect sizes as a function of moderator variables (all plots available in the Supplementary Material) and the generally low level of heterogeneity in most comparisons. Meta-analyses using multivariate models may, however, further clarify the relationship between sleep spindle parameters and intelligence.

Why is fast spindle amplitude, but no other spindle measure, correlated with intelligence? One of the possible reasons may be that fast spindle amplitude is the parameter best captured by all available detection methods, including power spectral density calculation. Fast spindle amplitude and power, but no other spindle measure, are highly correlated across spindle detectors, sleep states, and nights (Cox, Schapiro, Manoach, and Stickgold, 2017; Reynolds et al., 2018; Ujma, Gombos, et al., 2015). Furthermore, as it has been demonstrated in this study (Supplementary Table 1), absolute, log-transformed and even relative power spectral density in the sigma range correlates best with fast spindle amplitude, and the same is the case with fast spindle activity. However, low between-study variance of effect sizes (I2) indicates that differences in the detection methods at least do not fully explain the null findings.

Fast spindle amplitude may be associated with intelligence because interindividual differences in brain anatomy associated with higher intelligence may overlap with those associated with higher sleep spindle amplitude. To date, the most reliable biological correlates of cognitive ability are genetic markers: individual polygenic scores constructed from trait-associated SNPs obtained by genome-wide association studies are able to predict about 10% of the variance in cognitive outcomes in independent samples (Plomin and von Stumm, 2018). Genetic polymorphisms associated with cognitive ability are involved in a large, but not infinite number of biological processes: they are significantly expressed in the brain at the organ level; in the frontal cortex, the cerebellum, the cortex in general at the brain structure level; and somatosensory pyramidal cells and medium spiny neurons at the cell level. Furthermore, they are functionally implicated in myelination, synapse regulation, and neurogenesis (Davies et al., 2018; Lam et al., 2017; Savage et al., 2017). At least a part of these structures and functions overlap with those implicated in sleep spindle amplitude, which is positively associated with white matter-based thalamocortical connectivity (Piantoni et al., 2013) and the systematic interactions of reticular thalamic, thalamocortical, and frontal and parietal corticothalamic neuron firing (Lüthi, 2013). The presence of a genetic correlation (dependence on overlapping genetic variants) between white matter architecture and intelligence based on behavior genetic and structural imaging methods (Chiang et al., 2009, 2011) further strengthens the view that white matter-dependent sleep oscillations are correlated with intelligence because they rely on common structural and functional foundations.

The amplitude of sleep oscillations can be expressed as an absolute or relative value. Relative amplitude refers to the amplitude in a frequency range divided by the sum of amplitudes in all other frequency ranges. The advantage of the latter method is that it eliminates intraindividual differences in voltage, which can be the result of skull and scalp thickness instead of actual lower amplitude oscillations. Most spindle detectors calculate absolute amplitude; while absolute spectral power is used somewhat more routinely in many sleep studies, all studies included in this meta-analysis relied on absolute power. Since intelligence is positively associated with head and brain size (Pietschnig et al., 2015), lower voltage sleep oscillations would be expected in more intelligent subjects at all frequencies, with this tendency somewhat reversed at specific frequencies where systematically larger oscillations are generated as a function of higher intelligence. Therefore, it is possible that the correlation between intelligence and sleep spindle amplitude is biased downward by the confounding effect of head size and skull thickness, which is usually not measured in sleep studies, but can be somewhat reduced by using relative amplitudes.

We have previously found only positive correlations between spindle amplitude and intelligence in females (Bódizs et al., 2014; Ujma, Bódizs, et al., 2015; Ujma et al., 2014, 2016), and we have hypothesized that this is a general phenomenon. This meta-analysis failed to confirm this conclusion, as the sex composition of study subjects had no significant association with effect size. While a metaregression of the percentage of female subjects on the effect sizes of a relatively low number of studies is not the best method to test such a null hypothesis, it is notable that the GLAKU cohort study failed to support the hypothesis of significantly larger effect sizes in females (A. K. Pesonen, personal communication, 2018). Researchers with available data sets are encouraged to report effect sizes calculated separately for males and females, so this hypothesis can be more thoroughly investigated.

Suggestions for a replicable psychological sleep science

The conclusions of this meta-analysis markedly differ from that of any of the single studies reviewed herein. The reason for this discrepancy is simple: sleep spindle parameters are only weakly correlated with cognitive abilities, and small studies are underpowered to reliably assess this relationship, resulting in “shooting around” the true effect sizes. The sample size necessary to achieve 80% statistical power to detect a correlation of .15 – that is, in the range of the meta-analytic correlation between sleep spindle amplitude and cognitive ability – is N = 347. This sample size is not only not achieved by any of the individual studies, but also not reached even by combining the two largest data sets (N = 176 and 160, respectively). Therefore, it is unsurprising that even large individual studies failed to correctly approximate the aggregate effect of all studies.

It is unfortunate, but not unsurprising, that most laboratories are forced to work with small data sets. Full-night polysomnography recordings require the voluntary participation of every subject in the time-consuming inconvenient procedure of sleeping in a sleep laboratory with EEG electrodes mounted on their heads, typically for at least two consecutive nights, with even the preceding days reserved for learning tasks or other phenotypic measurements, resulting in a multiday absence from normal activities. Typical labs are single-bed, so the simultaneous recording of multiple subjects at the same time – even if volunteers were available – is not possible. Furthermore, the nearly continuous presence of laboratory assistants and research staff is required for several days in case of every single subject, rendering polysomnography probably one of the most expensive and time-consuming non-invasive human study designs typically employed in scientific research. A data set of N = 40 subjects provides very limited statistical power for the detection of most psychological effect sizes, but its establishment is an arduous task.

However, this limitation could be overcome by adding together the capacities of multiple laboratories and routinely sharing polysomnography and related phenotypic data, including existing data sets. “Open data” regimes, where collected scientific data are made available in an online repository for use by anybody interested, are encouraged (Boulton, Rawlins, Vallance, and Walport, 2011; Wicherts, 2013) and they are becoming routine (Devuyst, Dutoit, Stenuit, and Kerkhofs, 2011; O’Reilly, Gosselin, Carrier, and Nielsen, 2014; Jernigan et al., 2016). Therefore, it is advised that sleep researchers refrain from “hoarding” data, and instead routinely share it with other researchers, so that data sets can be pooled and higher powered individual studies become feasible. Regrettably, the ethical permissions of certain studies sometimes limit the possibility of data sharing, throttling the advance of cumulative science. It is therefore recommended that applications for ethical permissions be written in a way that permits data sharing, and colleagues serving on ethical boards observe the importance of sharing properly anonymized data with fellow researchers and not weigh such requests against applications.

Another approach – if data sharing is not possible for any reasons – is to prepare studies in a way that makes them suitable for their inclusion in subsequent meta-analyses. Unfortunately, several studies had to be excluded even from this meta-analysis, because while the authors clearly stated that they had spindle and intelligence data available, the association was simply reported as non-significant without a proper effect size. An effect size that fails a traditional null-hypothesis statistical test at α = .05 is not meaningless: it may be a true effect, only too small to become significant in a data set of the given size (Schmidt, 1996). Therefore, as a minimum, even non-significant effect sizes should always be reported for all comparisons that the authors perform in a study. It is possible to report large amounts of such results in Supplementary Tables, even if the authors feel that they would disrupt the normal structure of their paper. An excellent example for meta-analysis-ready reporting is found in one study (Tessier et al., 2015), where the authors systematically report the detailed results for all the performed statistical tests. It is recommended for all authors of future sleep studies to report effect sizes in a similar manner, and for all reviewers to request the reporting of them. Effect sizes should preferably approximate the measure likely available to the most independent studies; for example, it should be a correlation coefficient instead of a multiple regression coefficient, corrected for various other covariates, because these covariates may not be available from other studies and the effect sizes cannot be aggregated.

Adhering to the principles of data sharing and detailed reporting is simple and effective ways toward a more reproducible, evidence-based psychological sleep science.

Acknowledgements

This study was funded by the KTIA_NAP_13-1-2013-0001 and 2017-1.2.1-NKP-2017-00002 grants. PPU was supported by the ÚNKP-17-4 New National Excellence Program of the Ministry of Human Capacities. He also expresses his gratitude to Róbert Bódizs, Martin Dresler, Stuart Fogel, Reut Gruber, Kerstin Hoedlmoser, Mark Kohler, Rebecca Nader, Anu-Katriina Pesonen, and Matthew Tucker for providing data and Péter Simor for helpful comments on the manuscript.

References

  • Begg CB and Mazumdar M (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics 50(4):10881101. doi:10.2307/2533446

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bódizs R , Gombos F , Ujma PP , and Kovács I (2014). Sleep spindling and fluid intelligence across adolescent development: sex matters. Front Hum Neurosci 8:952. doi:10.3389/fnhum.2014.00952

    • Search Google Scholar
    • Export Citation
  • Bódizs R , Kis T , Lazar AS , Havran L , Rigo P , Clemens Z , and Halasz P (2005). Prediction of general mental ability based on neural oscillation measures of sleep. J Sleep Res 14(3):285292. doi:10.1111/j.1365-2869.2005.00472.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Boulton G , Rawlins M , Vallance P , and Walport M (2011). Science as a public enterprise: the case for open data. The Lancet 377(9778):16331635. doi:10.1016/S0140-6736(11)60647-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chatburn A , Coussens S , Lushington K , Kennedy D , Baumert M , and Kohler M (2013). Sleep spindle activity and cognitive performance in healthy children. Sleep 36(2):237243. doi:10.5665/sleep.2380

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chiang MC , Barysheva M , Shattuck DW , Lee AD , Madsen SK , Avedissian C , Klunder AD , Toga AW , McMahon KL , de Zubicaray GI , Wright MJ , Srivastava A , Balov N , and Thompson PM (2009). Genetics of brain fiber architecture and intellectual performance. J Neurosci 29(7):22122224. doi:10.1523/JNEUROSCI.4184-08.2009

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chiang MC , McMahon KL , de Zubicaray GI , Martin NG , Hickie I , Toga AW , Wright MJ , and Thompson PM (2011). Genetics of white matter development: a DTI study of 705 twins and their siblings aged 12 to 29. NeuroImage 54(3):23082317. doi:10.1016/j.neuroimage.2010.10.015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clawson BC , Durkin J , and Aton SJ (2016). Form and function of sleep spindles across the lifespan. Neural Plast 2016:116.doi:10.1155/2016/6936381

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clemens Z , Fabo D , and Halasz P (2006). Twenty-four hours retention of visuospatial memory correlates with the number of parietal sleep spindles. Neurosci Lett 403(1–2):5256. doi:10.1016/j.neulet.2006.04.035

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cox R , Schapiro AC , Manoach DS , and Stickgold R (2017). Individual differences in frequency and topography of slow and fast sleep spindles. Front Hum Neurosci 11:433. doi:10.3389/fnhum.2017.00433

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Davies G , Lam M , Harris SE , Trampush J , Luciano M , Hill WD , Hagenaars SP , Ritchie SJ , Marioni RE , Fawns-Ritchie C , Liewald DC , Okely J , Ahola-Olli A , Barnes CLK , Bertram L , Bis JC , Burdick KE , Christoforou A , DeRosse P , Djurovic S , Espeseth T , Giakoumaki S , Giddaluru S , Gustavson DE , Hayward C , Hofer E , Ikram MA , Karlsson R , Knowles E , Lahti J , Leber M , Li S , Mather KA , Melle I , Morris D , Oldmeadow C , Palviainen T , Payton A , Pazoki R , Petrovic K , Reynolds CA , Sargurupremraj M , Scholz M , Smith JA , Smith AV , Terzikhan N , Thalamuthu A , Trompet S , van der Lee SV , Ware EB , Windham BG , Wright MJ , Yang J , Yu J , Ames D , Amin N , Amouyel P , Andreassen OA , Armstrong N , Attia JR , Attix D , Avramopoulos D , Bennett DA , Böhmer AC , Boyle PA , Brodaty H , Campbell H , Cannon TD , Cirulli ET , Congdon E , Conley ED , Corley J , Cox SR , Dale AM , Dehghan A , Dick D , Dickinson D , Eriksson JG , Evangelou E , Faul JD , Ford I , Freimer NA , Gao H , Giegling I , Gillespie NA , Gordon SD , Gottesman RF , Griswold ME , Gudnason V , Harris TB , Hatzimanolis A , Heiss G , Holliday EG , Joshi PK , Kähönen M , Kardia SLR , Karlsson I , Kleineidam L , Knopman DS , Kochan N , Konte B , Kwok JB , Le Hellard S , Lee T , Lehtimäki T , Li S-C , Liu T , Koini M , London E , Longstreth WT Jr , Lopez OL , Loukola A , Luck T , Lundervold AJ , Lundquist A , Lyytikäinen L-P , Martin NG , Montgomery GW , Murray AD , Need AC , Noordam R , Nyberg L , Ollier W , Papenberg G , Pattie A , Polasek O , Poldrack RA , Psaty BM , Riedel-Heller SG , Rose RJ , Rotter JI , Roussos P , Rovio S , Saba Y , Sabb FW , Sachdev PS , Satizabal C , Schmid M , Scott RJ , Scult MA , Simino J , Slagboom PE , Smyrnis N , Soumaré A , Stefanis NC , Stott DJ , Straub RE , Sundet K , Taylor AM , Taylor KD , Tzoulaki I , Tzourio C , Uitterlinden A , Vitart V , Voineskos AN , Vuoksimaa E , Wagner M , Wagner H , Weinhold L , Wen KH , Widen E , Yang Q , Zhao W , Adams HHH , Arking DE , Bilder RM , Bitsios P , Boerwinkle E , Chiba-Falek O , Corvin A , De Jager PL , Debette S , Donohoe G , Elliott P , Fitzpatrick AL , Gill M , Glahn DC , Hägg S , Hansell NK , Hariri A , Ikram MK , Jukema JW , Kaprio J , Keller MC , Kremen WS , Launer L , Lindenberger U , Palotie A , Pedersen NL , Pendleton N , Porteous DJ , Räikkönen K , Raitakari O , Ramirez A , Reinvang I , Rudan I , Rujescu D , Schmidt R , Schmidt H , Schofield PW , Schofield PR , Starr JM , Steen VM , Trollor JN , Turner ST , Van Duijn CM , Villringer A , Weinberger DR , Weir DR , Wilson JF , Malhotra A , McIntosh AM , Gale CR , Seshadri S , Mosley T , Bressler J , Lencz T , and Deary IJ (2018). Study of 300,486 individuals identifies 148 independent genetic loci influencing general cognitive function. Nat Commun 9(1):2098. doi:10.1038/s41467-018-04362-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deary IJ , Penke L , and Johnson W (2010). The neuroscience of human intelligence differences. Nat Rev Neurosci 11(3):201211. doi:10.1038/nrn2793

  • Devuyst S , Dutoit T , Stenuit P , and Kerkhofs M (2011). Automatic sleep spindles detection – overview and development of a standard proposal assessment method. Conf Proc IEEE Eng Med Biol Soc 2011:17131716. doi:10.1109/IEMBS.2011.6090491

    • Search Google Scholar
    • Export Citation
  • Egger M , Davey Smith G , Schneider M , and Minder C (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315:629634. doi:10.1136/bmj.315.7109.629

    • Search Google Scholar
    • Export Citation
  • Fang L , Ray L , Owen AM , and Fogel S (2017). Simultaneous EEG-FMRI reveals spindle-related neural correlates of human intellectual abilities during NREM sleep. Sleep Medicine 40(Suppl. 1):e99. doi:10.1016/j.sleep.2017.11.288

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fang Z , Sergeeva V , Ray LB , Viczko J , Owen AM , and Fogel SM (2017). Sleep spindles and intellectual ability: epiphenomenon or directly related? J Cogn Neurosci 29(1):167182. doi:10.1162/jocn_a_01034

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ferguson CJ and Heene M (2012). A vast graveyard of undead theories: publication bias and psychological science’s aversion to the null. Perspect Psychol Sci 7(6):555561. doi:10.1177/1745691612459059

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fogel SM , Nader R , Cote KA , and Smith CT (2007). Sleep spindles and learning potential. Behav Neurosci 121(1):110. doi:10.1037/0735-7044.121.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fogel SM and Smith CT (2006). Learning-dependent changes in sleep spindles and stage 2 sleep. J Sleep Res 15(3):250255. doi:10.1111/j.1365-2869.2006.00522.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Geiger A , Huber R , Kurth S , Ringli M , Jenni OG , and Achermann P (2011). The sleep EEG as a marker of intellectual ability in school age children. Sleep 34(2):181189. doi:10.1093/sleep/34.2.181

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Giner-Sorolla R (2012). Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspect Psychol Sci 7(6):562571. doi:10.1177/1745691612457576

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gruber R , Wise MS , Frenette S , Knäauper B , Boom A , Fontil L , and Carrier J (2013). The association between sleep spindles and IQ in healthy school-age children. Int J Psychophysiol 89(2):229240. doi:10.1016/j.ijpsycho.2013.03.018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haier RJ (2016). The neuroscience of intelligence. Cambridge, MA: Cambridge University Press.

  • Halonen R , Ujma P , Kuula L , Makkonen T , Lahti J , Räikkönen K , and Pesonen K (in preparation). The associations between spindle characteristics and cognitive ability in adolescence.

    • Search Google Scholar
    • Export Citation
  • Hoedlmoser K , Heib DP , Roell J , Peigneux P , Sadeh A , Gruber G , and Schabus M (2014). Slow sleep spindle activity, declarative memory, and general cognitive abilities in children. Sleep 37(9):15011512. doi:10.5665/sleep.4000

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ioannidis JA (2005a). Contradicted and initially stronger effects in highly cited clinical research. JAMA 294(2):218228. doi:10.1001/jama.294.2.218

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ioannidis JPA (2005b). Why most published research findings are false. PLoS Med 2(8):e124. doi:10.1371/journal.pmed.0020124

  • Jensen AR (1998). The G factor: the science of mental ability. Westport, CT: Praeger.

  • Jensen AR (2006). Clocking the mind: mental chronometry and individual differences. Amsterdam, The Netherlands: Elsevier Science.

  • Jernigan TL , Brown TT , Hagler DJ , Akshoomoff N , Bartsch H , Newman E , Thompson WK , Bloss CS , Murray SS , Schork N , Kennedy DN , Kuperman JM , McCabe C , Chung Y , Libiger O , Maddox M , Casey BJ , Chang L , Ernst TM , Frazier JA , Gruen JR , Sowell ER , Kenet T , Kaufmann WE , Mostofsky S , Amaral DG , and Dale AM (2016). The pediatric imaging, neurocognition, and genetics (PING) data repository. NeuroImage 124:11491154. doi:10.1016/j.neuroimage.2015.04.057

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jung RE and Haier RJ (2007). The parieto-frontal integration theory (P-FIT) of intelligence: converging neuroimaging evidence. Behav Brain Sci 30(2):135154. doi:10.1017/S0140525X07001185

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lam M , Trampush JW , Yu J , Knowles E , Davies G , Liewald DC , Starr JM , Djurovic S , Melle I , Sundet K , Christoforou A , Reinvang I , DeRosse P , Lundervold AJ , Steen VM , Espeseth T , Räikkönen K , Widen E , Palotie A , Eriksson JG , Giegling I , Konte B , Roussos P , Giakoumaki S , Burdick KE , Payton A , Ollier W , Chiba-Falek O , Attix DK , Need AC , Cirulli ET , Voineskos AN , Stefanis NC , Avramopoulos D , Hatzimanolis A , Arking DE , Smyrnis N , Bilder RM , Freimer NA , Cannon TD , London E , Poldrack RA , Sabb FW , Congdon E , Conley ED , Scult MA , Dickinson D , Straub RE , Donohoe G , Morris D , Corvin A , Gill M , Hariri AR , Weinberger DR , Pendleton N , Bitsios P , Rujescu D , Lahti J , Le Hellard S , Keller MC , Andreassen OA , Deary IJ , Glahn DC , Malhotra AK , and Lencz T (2017). Large-scale cognitive GWAS meta-analysis reveals tissue-specific neural expression and potential nootropic drug targets. Cell Rep 21(9):25972613. doi:10.1016/j.celrep.2017.11.028

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lustenberger C , Maric A , Durr R , Achermann P , and Huber R (2012). Triangular relationship between sleep spindle activity, general cognitive ability and the efficiency of declarative learning. PLoS One 7(11):e49561. doi:10.1371/journal.pone.0049561

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lüthi A (2013). Sleep spindles where they come from, what they do. Neuroscientist 20(3):243256. doi:10.1177/1073858413500854

  • McCreesh S (2016). NREM sleep spindles and slow wave sleep in younger and elderly women: an investigation of their influence on declarative memory consolidation. Thesis dissertation Master of Social Science (MSocSc) in Psychology, Faculty of Humanities. University of Cape Town.

    • Search Google Scholar
    • Export Citation
  • Merikanto I , Kuula L , Makkonen T , Bódizs R , Halonen R , Heinonen K , Lahti J , Räikkönen K , and Pesonen A-K (2017). Circadian preference towards morningness is associated with lower slow sleep spindle amplitude and intensity in adolescents. Sci Rep 7(1):14619. doi:10.1038/s41598-017-13846-7

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nader R and Smith C (2001). The relationship between stage 2 sleep spindles and intelligence. Sleep 24:A160.

  • Nader RS and Smith CT (2015). Correlations between adolescent processing speed and specific spindle frequencies. Front Hum Neurosci 9:30. doi:10.3389/fnhum.2015.00030

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Neubauer AC and Fink A (2009). Intelligence and neural efficiency: measures of brain activation versus measures of functional connectivity in the brain. Intelligence 37(2):223229. doi:10.1016/j.intell.2008.10.008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349(6251):aac4716. doi:10.1126/science.aac4716

    • Search Google Scholar
    • Export Citation
  • O’Reilly C , Gosselin N , Carrier J , and Nielsen T (2014). Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res 23:628635. doi:10.1111/jsr.12169

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peters KR , Ray L , Smith V , and Smith C (2008). Changes in the density of stage 2 sleep spindles following motor learning in young and older adults. J Sleep Res 17(1):2333. doi:10.1111/j.1365-2869.2008.00634.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Peters KR , Smith V , and Smith CT (2007). Changes in sleep architecture following motor learning depend on initial skill level. J Cogn Neurosci 19(5):817829. doi:10.1162/jocn.2007.19.5.817

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Piantoni G , Poil SS , Linkenkaer-Hansen K , Verweij IM , Ramautar JR , Van Someren EJ , and Van Der Werf YD (2013). Individual differences in white matter diffusion affect sleep oscillations. J Neurosci 33(1):227233. doi:10.1523/JNEUROSCI.2030-12.2013

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pietschnig J , Penke L , Wicherts JM , Zeiler M , and Voracek M (2015). Meta-analysis of associations between human brain volume and intelligence differences: how strong are they and what do they mean? Neurosci Biobehav Rev 57:411432. doi:10.1016/j.neubiorev.2015.09.017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Plomin R and von Stumm S (2018). The new genetics of intelligence. Nat Rev Genet 19(3):148159. doi:10.1038/nrg.2017.104

  • Ray LB , Sockeel S , Soon M , Bore A , Myhr A , Stojanoski B , Cusack R , Owen AM , Doyon J , and Fogel SM (2015). Expert and crowd-sourced validation of an individualized sleep spindle detection method employing complex demodulation and individualized normalization. Front Hum Neurosci 9:507. doi:10.3389/fnhum.2015.00507

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reynolds CM , Gradisar M , and Short MA (2018). Reliability of sleep spindle measurements in adolescents: how many nights are necessary?. J Sleep Res, e12698. doi:10.1111/jsr.12698

    • Search Google Scholar
    • Export Citation
  • Savage JE , Jansen PR , Stringer S , Watanabe K , Bryois J , de Leeuw CA , Nagel M , Awasthi S , Barr PB , Coleman JRI , Grasby KL , Hammerschlag AR , Kaminski J , Karlsson R , Krapohl E , Lam M , Nygaard M , Reynolds CA , Trampush JW , Young H , Zabaneh D , Hägg S , Hansell NK , Karlsson IK , Linnarsson S , Montgomery GW , Munoz-Manchado AB , Quinlan EB , Schumann G , Skene N , Webb BT , White T , Arking DE , Attix DK , Avramopoulos D , Bilder RM , Bitsios P , Burdick KE , Cannon TD , Chiba-Falek O , Christoforou A , Cirulli ET , Congdon E , Corvin A , Davies G , Deary IJ , DeRosse P , Dickinson D , Djurovic S , Donohoe G , Conley ED , Eriksson JG , Espeseth T , Freimer NA , Giakoumaki S , Giegling I , Gill M , Glahn DC , Hariri AR , Hatzimanolis A , Keller MC , Knowles E , Konte B , Lahti J , Le Hellard S , Lencz T , Liewald DC , London E , Lundervold AJ , Malhotra AK , Melle I , Morris D , Need AC , Ollier W , Palotie A , Payton A , Pendleton N , Poldrack RA , Räikkönen K , Reinvang I , Roussos P , Rujescu D , Sabb FW , Scult MA , Smeland OB , Smyrnis N , Starr JM , Steen VM , Stefanis NC , Straub RE , Sundet K , Voineskos AN , Weinberger DR , Widen E , Yu J , Abecasis G , Andreassen OA , Breen G , Christiansen L , Debrabant B , Dick DM , Heinz A , Hjerling-Leffler J , Ikram MA , Kendler KS , Martin NG , Medland SE , Pedersen NL , Plomin R , Polderman TJC , Ripke S , van der Sluis S , Sullivan PF , Tiemeier H , Vrieze SI , Wright MJ , and Posthuma D (2017). GWAS meta-analysis (N = 279, 930) identifies new genes and functional links to intelligence. bioRxiv. doi:10.1101/184853

    • Search Google Scholar
    • Export Citation
  • Schabus M , Hodlmoser K , Gruber G , Sauter C , Anderer P , Klosch G , Parapatics S , Saletu B , Klimesch W , and Zeitlhofer J (2006). Sleep spindle-related activity in the human EEG and its relation to general cognitive and learning abilities. Eur J Neurosci 23(7):17381746. doi:10.1111/j.1460-9568.2006.04694.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schabus M , Kerstin H , Thomas P , Ander P , Gruber G , Parapatics S , Sauter C , Kloesch G , Klimesch W , Saletu B , and Zeitlhofer J (2008). Interindividual sleep spindle differences and their relation to learning-related enhancements. Brain Res 1191:127135. doi:10.1016/j.brainres.2007.10.106

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schmidt FL (1996). Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychol Methods 1(2):115129. doi:10.1037/1082-989X.1.2.115

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stanley TD (2017). Limitations of PET-PEESE and other meta-analysis methods. Soc Psychol Pers Sci 8(5):581591. doi:10.1177/1948550617693062

  • Sterne JAC , Gavaghan D , and Egger M (2000). Publication and related bias in meta-analysis. J Clin Epidemiol 53(11):11191129. doi:10.1016/S0895-4356(00)00242-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tan X , Campbell IG , and Feinberg I (2001). Internight reliability and benchmark values for computer analyses of non-rapid eye movement (NREM) and REM EEG in normal young adult and elderly subjects. Clin Neurophysiol 112(8):15401552. doi:10.1016/S1388-2457(01)00570-3

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tessier S , Lambert A , Chicoine M , Scherzer P , Soulieres I , and Godbout R (2015). Intelligence measures and stage 2 sleep in typically-developing and autistic children. Int J Psychophysiol 97(1):5865. doi:10.1016/j.ijpsycho.2015.05.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tucker MA and Fishbein W (2009). The impact of sleep duration and subject intelligence on declarative and motor memory performance: how much is enough? J Sleep Res 18:304312. doi:10.1111/j.1365-2869.2009.00740.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ujma PP , Bódizs R , Gombos F , Stintzing J , Konrad BN , Genzel L , Steiger A , and Dresler M (2015). Nap sleep spindle correlates of intelligence. Sci Rep 5(1):17159. doi:10.1038/srep17159

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ujma PP , Gombos F , Genzel L , Konrad BN , Simor P , Steiger A , Dresler M , and Bódizs R (2015). A comparison of two sleep spindle detection methods based on all night averages: individually adjusted versus fixed frequencies. Front Hum Neurosci 9:52. doi:10.3389/fnhum.2015.00052

    • Search Google Scholar
    • Export Citation
  • Ujma PP , Konrad BN , Genzel L , Bleifuss A , Simor P , Potari A , Kormendi J , Gombos F , Steiger A , Bodizs R , and Dresler M (2014). Sleep spindles and intelligence: evidence for a sexual dimorphism. J Neurosci 34(49):1635816368. doi:10.1523/JNEUROSCI.1857-14.2014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ujma PP , Sandor P , Szakadat S , Gombos F , and Bodizs R (2016). Sleep spindles and intelligence in early childhood-developmental and trait-dependent aspects. Dev Psychol 52(12):21182129. doi:10.1037/dev0000233

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Van Elk M , Matzke D , Gronau Q , Guang M , Vandekerckhove J , and Wagenmakers E-J (2015). Meta-analyses are no substitute for registered replications: a skeptical perspective on religious priming. Front Psychol 6:1365. doi:10.3389/fpsyg.2015.01365

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ward MP , Peters KR , and Smith CT (2013). Effect of emotional and neutral declarative memory consolidation on sleep architecture. Exp Brain Res 8:8. doi:10.1007/s00221-013-3781-0

    • Search Google Scholar
    • Export Citation
  • Wicherts J (2013). Science revolves around the data. J Open Psychol Data 1(1):e1. doi:10.5334/jopd.e1

  • Wicherts JM , Veldkamp CLS , Augusteijn HEM , Bakker M , van Aert RCM , and van Assen MALM (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: a checklist to avoid p-hacking. Front Psychol 7:1832. doi:10.3389/fpsyg.2016.01832

    • Crossref
    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Supplementary Materials

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Jul 2020 0 40 7
Aug 2020 0 21 11
Sep 2020 0 17 2
Oct 2020 0 15 8
Nov 2020 0 42 10
Dec 2020 0 32 9
Jan 2021 0 13 0