Abstract
Background and aims
Instances of Internet use disorders (IUD) including Internet gaming disorder (IGD) and non-gaming pathological Internet use (ng-PIU) have the extent that they are now a growing mental health issue. Individuals suffering from IUD show a large range of symptoms, high comorbidities and impairments in different areas of life. To date there is a lack of efficient and evidence-based treatment programs for such adolescents. The present registered single-arm trial (ClinicalTrials.gov: NCT03582839) aimed to investigate the long-term effects of a brief manualized cognitive behavioral therapy (CBT) program for adolescents with IUD.
Methods
N = 54 patients (16.7% female), aged 9–19 years (M = 13.48, SD = 1.72) received the CBT group program PROTECT+. IUD symptom severity (primary outcome variable) as well as comorbid symptoms, risk-related variables and potentially protective skills (secondary outcome variables) were assessed at pretest, posttest, as well as 4 and 12 months after admission.
Results
Patients showed a significant reduction in IUD symptom severity at the 12-month follow-up. Effect sizes were medium to large depending on the measure. Beyond the statistical significance, the clinical significance was confirmed using the reliable change index. Secondary outcome variables showed a significant reduction in self-reported depression, social anxiety, performance anxiety and school anxiety as well as in parental-reported general psychopathology.
Discussion and conclusions
The present study shows long-term effects of a manual-based CBT treatment for adolescents suffering from IUD. The results indicate that even a 4-session brief intervention can achieve a medium to large effect over 12 months. Future work is needed to confirm the efficacy within a randomized controlled trial (RCT).
Introduction
Internet use disorders (IUD), including both (Internet) gaming disorder (IGD) and non-gaming pathological Internet use (ng-PIU), are new phenomena which have increasingly attracted the attention of both clinicians and researchers. In 2013, the American Psychiatric Association at last included “Internet gaming disorder”, comprising both online and offline gaming, in Section III of the DSM-5 to provide a common ground for diagnostic criteria and a calling for further research (American Psychiatric Association, 2013). In 2018, the World Health Organization followed acknowledging IGD as a disease by including the diagnosis “gaming disorder” in the ICD-11 (World Health Organization, 2018). While IGD refers to a pathological use of (Internet) games, ng-PIU comprises the pathological use of other Internet applications, e.g., social networks, communication, online shopping, or online pornography. In the ICD-11, ng-PIU can be classified as “other specified disorders due to addictive behaviors” (World Health Organization, 2018). Although, ng-PIU is not yet explicitly defined as a disease due to insufficient evidence, the scientific community continues to inquire upon this subtype of IUD, since research findings suggest similar psychological strain and impairments (Strittmatter et al., 2015). Hence, we will refer to IUD as an umbrella term for both IGD and ng-PIU.
While global Internet usage has grown considerably, increasing rates of IUD are reported and IUD has become a growing mental health concern. In that matter International prevalence rates vary to high extent in different regions of the world - one reason for this being differing diagnostic criteria and assessment tools (Kuss, Griffiths, Karila, & Billieux, 2014). A meta-analysis of 31 nations revealed a global IUD prevalence estimate of 6.0% amongst 12–41-year-olds with the highest prevalence in the Middle East (Cheng & Li, 2014). Another meta-analysis across three decades revealed a global IGD prevalence of 4.6% amongst adolescents aged between 10 and 19 years (Fam, 2018). Furthermore, data from several studies suggest that IUD are more prevalent in younger age groups (Bakken, Wenzel, Götestam, Johansson, & Oren, 2009; Mentzoni et al., 2011; Wittek et al., 2016). In addition, IUD are associated with a poor mental health and several comorbidities, like depressive disorders (Bargeron & Hormes, 2017; Brunborg, Mentzoni, & Frøyland, 2014; Kim et al., 2016; Király et al., 2014; Männikkö, Billieux, & Kääriäinen, 2015; Mentzoni et al., 2011; Ostovar et al., 2016; Strittmatter et al., 2015; Wang, Cho, & Kim, 2018; Yen, Ko, Yen, Wu, & Yang, 2007), anxiety disorders (Bargeron & Hormes, 2017; Kim et al., 2016; Männikkö et al., 2015; Mentzoni et al., 2011; Ostovar et al., 2016), attention deficit/hyperactivity disorders (Strittmatter et al., 2015; Yen, Ko, et al., 2007; Yen, Liu, et al., 2017), conduct disorders (Strittmatter et al., 2015) as well as substance and alcohol abuse (Bakken et al., 2009; Ko et al., 2008). Moreover, IUD is related to lower life satisfaction (Bargeron & Hormes, 2017; Mentzoni et al., 2011). Since most of the existing literature about comorbidities is based on a cross-sectional design, the direction of causality remains unclear. However, existing longitudinal studies indicate a reciprocal relationship (Anderson, Steen, & Stavropoulos, 2016).
Despite these manifold and severe impairments IUD are treated to date unsystematically and studies investigating intervention effects in individuals with IUD, especially in adolescents, are rare. An intervention study from Li and Wang (2013) targeting adolescents and offering cognitive behavioral therapy (CBT) in the intervention group and basic counseling in the control group showed in both groups no significant reduction in IGD symptom severity at posttest. Another one-armed study investigating the effectiveness of a CBT group program in 18 adolescents showed a significant reduction of IUD symptom severity directly after the intervention (Wartberg, Thomsen, Moll, & Thomasius, 2014). Du and colleagues (Du, Jiang, & Vance, 2010) conducted a randomized controlled study investigating long-term effects of a CBT treatment and found similar effects after six months in the intervention and control group. Although reviews on treatment of IUD reported increasing work in treatment studies over the last years, a lack of well-designed and evidence-based interventions has been stated (King et al., 2017; Zajac, Ginley, Chang, & Petry, 2017). A recent systematic review and meta-analysis suggested a short-term effectiveness of CBT for IGD but a lack of evidence for long-term benefits (Stevens, King, Dorstyn, & Delfabbro, 2018). Nevertheless, CBT has to date the greatest evidence base in comparison to other intervention types and seems to be superior with regard of the benefits (King et al., 2017).
The present study aimed to investigate the effects of the PROTECT+ program, which is a CBT-based group intervention program for adolescents. We assumed, that patients treated with PROTECT+ would show a significant reduction in IUD symptom severity after 12 months (primary outcome measure). Furthermore, it was hypothesized, that participants would also show a significant reduction in risk-related variables and a significant increase in potentially protective variables as well as in functional skills (secondary outcome measures).
Methods
Participants
The patients were 54 adolescents (16.7% female) between 9 and 19 years of age (M = 13.48; SD = 1.72) who had signed up for the group therapy program due to self-reported or parent-reported excessive gaming or Internet use and who reported subjective psychological strain (self-selection). In total, 55.6% of the participants attended a school with a higher educational level (“Gymnasium”), 25.9% attended a school with middle educational level (“Realschule”), 5.6% attended a school with a lower educational level (“Werkrealschule”), and 13.0% were other (students of comprehensive schools, elementary schools or school finishers). The target group consisted of adolescents at the age of 12–18 years with high risk/pathological gaming or Internet use. Inclusion criterion was a subjective psychological strain in the wake of excessive gaming or use of the Internet on the part of the adolescents or their parents. There were no further objective or external inclusion criteria. Four participants aged between 9 and 11 years were included due to their high motivation and above-average cognitive development. One adolescent was 19 years old and was included due to a high degree of psychological strain, as well as his motivation to participate. According to self-report diagnostics, using the Compulsive Internet Use Scale (CIUS; Meerkerk et al., Van Den Eijnden, Vermulst, & Garretsen, 2009), 30.8% of the adolescents fulfilled the criteria for pathological use (defined by a total score ≥ 28), 34.6% for high-risk use (28 > total score ≥ 20) and 34.6% reported unproblematic use at baseline. According to the German Video Game Dependency Scale (CSAS; Rehbein, Baier, Kleimann, & Mößle, 2015; Rehbein, Kliem, Baier, Mößle, & Petry, 2015), which was applied in an adapted self-report (CSAS-SR) and an adapted parental report version (CSAS-PR) including ng-IUD, 1.9% (CSAS-SR) vs. 55.8% (CSAS-PR) being classified as “pathological users”, 25.0% (CSAS-SR) vs. 32.6% (CSAS-PR) as “high-risk users” and 73.1% (CSAS-SR) vs. 11.6% (CSAS-PR) as “unproblematic users”.
Sampling procedures
Participants were recruited by multiple channels in Heidelberg, Mannheim and the Rhine-Neckar area, Germany. The program was promoted in circumjacent child and adolescent psychiatric institutions, youth welfare services, secondary schools (school counselors, school social workers, and parent councils), learning centers, private tutors, children's homes, youth centers, and other nonprofit institutions and facilities for children and youths. Patients were referred by several professionals and practitioners who were informed about PROTECT+ in advance by way of one-to-one conversations, personal letters as well as flyers: treating physicians (pediatrists, family doctors, and psychiatrists), psychologists, psychotherapists, addiction counselors, educational counselors and school psychologists. Additionally, participants were recruited by newspaper articles and flyers. The study was promoted as a randomized controlled trial (RCT) including a waiting control group (over the course of 4 months). Due to the low enrollment rate and a relatively high participation withdrawal rate, the study design was adapted to a one-armed intervention study.
Trained psychologists conducted the intervention in 11 groups of 3–7 participants between April 2016 and December 2017. Participants were divided into groups according to age. The PROTECT+ program took place at the University of Education in Heidelberg and in the addiction counseling center of the Caritas Association in Mannheim. Each group was supervised by the head of the department of child and adolescent psychotherapy of the Center for Psychological Psychotherapy of the Heidelberg University. The baseline assessment and the 1-month follow-up were collected in situ (right before the first session and right after the last session). Data collection at 4-month follow-up was carried out by post (containing coded questionnaires and return envelopes). Moreover, participants were interviewed at 12-month follow-up in situ at the University of Education in Heidelberg or by telephone. Additionally, they completed questionnaires at 12-month follow-up in situ right after the interview or at home (by post), if the interview took place by telephone. If necessary, after 2 and 4 weeks respectively, participants were reminded per post to return the questionnaires at 4-month follow-up and 12-month follow-up. The participation was free of charge, and participants did not receive monetary reward. The participation flow in this study is outlined in Fig. 1.
Participant flow throughout the study
Citation: Journal of Behavioral Addictions 9, 4; 10.1556/2006.2020.00089
The PROTECT+ intervention
The PROTECT+ intervention consisted of 4 group sessions at 100 minutes (1 session per week in 4 subsequent weeks). The PROTECT+ intervention was based on the PROTECT program (Lindenberg, Kindt, & Szász-Janocha, 2020), which was already evaluated in an RCT (ClinicalTrials.gov: NCT02907658) focusing on selective-indicated school-based prevention (Lindenberg, 2018). PROTECT consists of cognitive behavioral techniques like psychoeducation, cognitive restructuring (e.g., identification and modification of dysfunctional beliefs), life skills (problem solving, behavior modification, emotion regulation). The main differences between the PROTECT+ intervention and the PROTECT prevention study were the out-patient (sub-)clinical setting vs. school-based prevention, the (sub-)clinical target group vs. high-risk sample and the extended session duration, which enabled to address more deeply the needs of participants. Each module broached the issue of a potential risk factor of the development and maintenance of IUD. The session goals were (1) handling of boredom and motivational problems, (2) reduction of procrastination and performance anxiety, (3) reduction of social anxiety and promotion of social skills, and (4) promotion of functional emotion regulation skills, respectively. For a detailed description see (Lindenberg et al., 2020).
Measures
Primary outcome measure
The primary outcome measure was the IUD symptom severity 12 months after assessment. Multiple diagnostic tools were used due to the lack of uniform diagnostic criteria and valid psychometric measures.
The CSAS (Rehbein, Baier et al., 2015) assesses IGD symptoms according to DSM-5. Each item of the 18-item questionnaire can be marked from 0 (strongly disagree) to 3 (strongly agree). It has shown excellent internal consistency (Cronbach's α = 0.92), reliability (split-half reliability r = 0.94; 4-week test-retest reliability r = 0.84), and a high face validity. It was administered at baseline, at 1-month follow-up, 4-month follow-up and 12-month follow-up. To provide a more extensive measure (including ng-PIU symptoms according to DSM-5 criteria), we adapted this diagnostic instrument by adding ng-PIU (e.g., “I feel that games/Internet activities are becoming more and more important to me”). In addition to the self-report version, the informant-rated version was applied in the present trial (rated by a reference person, mostly a parent).
The CIUS (Meerkerk et al., 2009) is a 14-item self-report measure assessing IUD symptom severity on a 5-point Likert scale. The response options range from 0 (never) to 4 (very often). We used the German version (Gürtler et al., 2014) at baseline, at 1-month follow-up, 4-month follow-up and 12-month follow-up. The German version (Cronbach’s α = 0.87) and the original version (Cronbach’s α = 0.89 to 0.90) have shown good internal consistency as well as high validity (Gürtler et al., 2014; Meerkerk et al., 2009).
Furthermore, the incidence and remission of IGD and ng-PIU was assessed at the 12-month follow-up with a semi-structured clinical interview according to DSM-5 criteria based on the CSAS. The original version measures symptom severity of IGD with reference to the last year (Kriminologisches Forschungsinstitut Niedersachsen e.V.). Aiming to provide a more differentiated measure, we adapted this interview by adding ng-PIU and by exploring separately the current symptom severity (within the last month) and retrospectively the symptom severity at the time that the participant had enrolled in the study (12 months prior). The interviews were conducted either personally or by telephone and rated by trained clinical psychologists. Additionally, the interviews were audio recorded and rated for a second time by an independent individual.
Secondary outcome measures
The secondary outcome measures comprised the amount of time spent online, symptom severity of comorbid symptoms as well as changes in associated variables (e.g., emotion regulation skills). They were measured at baseline, 1-month follow-up, 4-month follow-up and 12-month follow-up.
Amount of time spent online
The amount of time spent online is not included as criterion for IGD or ng-PIU. However, it represents a relevant indicator of IUD. We asked participants about the average amount of time spent online for weekdays and weekends separately. The amount of time spent online was computed as follows: [(5 × time spent online on a weekday + 2 × time spent online on a weekend day)/7].
General psychopathology
The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997; Goodman, Meltzer, & Bailey, 2003) is a 25-item questionnaire, that provides a total score as well as five subscales [(1) Hyperactivity Scale, (2) Emotional Symptoms Scale, (3) Conduct Problems Scale, (4) Peer Problems Scale, (5) Pro-social Scale]. The questionnaire implies a response format ranging from 0 (not true) to 2 (certainly true) and provides a self-report version (SDQ-SR) as well as a parent-rated version (SDQ-PR). The total score of the German version (Lohbeck, Schultheiß, Petermann, & Petermann, 2015; Woerner, Becker, & Rothenberger, 2004), which was used in the present study, showed moderate to good internal consistency (SDQ-SR Cronbach's α = 0.77; SDQ-PR Cronbach's α = 0.82) as well as reliability (SDQ-SR: 6-week test-retest reliability r = 0.66).
Depressive symptoms
Depressive symptoms were assessed using the German version (DIKJ; Stiensmeier-Pelster, Schürmann, & Duda, 2000) of the Children's Depression Inventory (CDI; Kovacs, 1985). The DIKJ comprises 27 items using three phrases per item (scores of 0–2 per item), from which the participant selects the most accurate phrase. Higher scores represent higher symptom severity. The DIKJ was shown to possess a good internal consistency (Cronbach's α = 0.85–0.90) and a high split-half reliability (r = 0.84–0.91) in a clinical as well as in school-based sample.
Social anxiety
The German version (Stangier, Heidenreich, Berardi, Golbs, & Hoyer, 1999) of the Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998) was used to assess social phobia fears. The response options of the 20 items range from 0 (not at all) to 4 (extremely). The German version has shown excellent internal consistency (Cronbach's α = 0.94) and a high level of 3-week test-retest reliability (r = 0.92).
Performance anxiety and school anxiety
Performance anxiety and school anxiety were measured using a subscale of the German revision (PHOKI; Döpfner, Schnabel, Goletz, & Ollendick, 2006) of the Fear Survey Schedule for Children-Revised (FSSC-R; Ollendick, 1983). The 9-item subscale implies a response format ranging from 0 (none) to 2 (a lot) and has shown moderate internal consistency (Cronbach's α = 0.78).
Procrastination
Procrastination was assessed with the German Questionnaire for Procrastination (APROF; Höcker, Engberding, & Rist, 2013). It measures three subscales and contains 18 items on a 7-point Likert scale ranging from 1 (never) to 7 (always). The authors stated good psychometric properties, publications about the psychometric properties are not available. We used the individual average score by computing the mean value of the subscales.
Emotion regulation
Emotion regulation skills were assessed with the German Questionnaire for Assessment of Emotion Regulation in Children and Adolescents (FEEL-KJ; Grob & Smolenski, 2009). The 90-items questionnaire measures 15 regulation strategies separately for fear, sadness and anger as well as two secondary scales (functional regulation and dysfunctional regulation). The response options range from 1 (almost never) to 5 (almost always). It has shown good to excellent internal consistency (Cronbach's α for secondary scales = 0.82–0.93) and reliability (6-week test-retest reliability for secondary scales r = 0.73–0.81).
Social behavior and learning behavior
Aiming to assess social competent behavior and academic motivation, we used the German Student Assessment List for Social and Learning Behavior (SSL; Petermann & Petermann, 2014). It is a 40-items measure that provides two secondary scales (social behavior and learning behavior) as well as ten subscales. Each item can be rated from 0 (never) to 3 (often). The SSL has shown good internal consistency (Cronbach's α for secondary scales = 0.88) and reliability (7-week test-retest reliability for secondary scales r = 0.74–0.88).
Self-efficacy
Self-efficacy was rated on the German General Self-Efficacy Scale (SWE; Schwarzer & Jerusalem, 1999). It contains 10 items that can be rated on a range from 1 (not at all true) to 4 (exactly true). It has shown good internal consistency (Cronbach's α = 0.80–0.90) and criterion-related validity.
Statistical analysis
Preliminary analyses
We examined the distribution of the outcome variables and applied transformations aiming to normalize the distributions when required. The impact of outliers was reduced by assigning the score a value that is one unit larger/smaller than the next most extreme value of that variable (Tabachnick & Fidell, 2014). Missing values were not replaced. In order to reduce potential biases, we assessed the comparability of participants with and without missing data including data at baseline, 1-month follow-up, 4-month follow-up and 12-month follow-up using two-tailed t-tests. In the event of significance, we included complete datasets only.
Model building
Following the model building procedure of Singer and Willett (2003), firstly, we computed unconditional means models; secondly, unconditional growth models and thirdly, conditional growth models with random intercepts and random slopes. Measurement points (Level 1) were nested within individuals (Level 2). Effect sizes were computed as Cohen's d by using the difference between the raw scores at baseline and 12-month follow-up divided by the baseline standard deviation.
Additional analyses
Clinical significance was assessed by computing the reliable change index (Jacobson & Truax, 1991). The reliable change is defined as the change in a patient's score divided by the standard error of the difference of the test. To assess pretest and 12-month differences in the diagnostic classification (unproblematic user vs. high-risk user or pathological user) according to the clinical interview, the McNemar -Test was applied. Exploratory analyses comprised exploratory conditional growth models including one predictor at a time, comprising gender, age, gamer (measured by stating to play games “often” or “very often”) vs. non-gamer (measured by stating to play games “sometimes”, “rarely” or “never”), pretest CSAS-SR score, and mean-centered secondary outcome measures at pretest.
Preliminary analyses and additional analyses, except of the growth models, were computed using SPSS version 25.0 (IBM Corp. Released, 2017). Intent-to-treat analyses were evaluated with growth models in a hierarchical linear model framework using the lme function in the nlme package from the R project (Pinheiro, Bates, DebRoy, Sarkar, & R Core Team, 2019).
Ethics
The presented data were collected in the context of the registered PROTECT+ study (ClinicalTrials.gov: NCT03582839). The trial was approved by the University of Education Heidelberg Research Ethics Committee and was conducted in consensus with the Declaration of Helsinki. Prior to admission, participants and legal guardians of underage participants received oral and written information about the aim and the scope of the study as well as the procedure including the follow-up assessments, and completed an informed written consent.
Results
Preliminary analyses
Overall, of the 54 patients who started the intervention, 55.6% attended all sessions, 33.3% attended three sessions, 1.9% attended two sessions, and 9.3% attended one session. Attrition for paper-pencil diagnostic data was 3.7% at pretest, 18.5% at posttest, 42.6% at the 4-month follow-up, and 38.9% at the 12-month follow-up (for the clinical interview 33.3%). At pretest, 69.2% (n = 36) specified playing games “often” or “very often”. Table 1 shows the means and standard deviations of all primary and secondary outcome measures for all study data collection points. It is noteworthy, that mean symptom severity improved in all outcome measures after 12 months. Fig. 2 visually presents the descriptive statistics of the primary measures over 12 months. Distributions of the outcome variables approximated normality, except amount of time spent online, which was normalized with a log transformation, and DIKJ as well as SIAS, which were normalized with a square root transformation. Participants with and without missing data differed significantly in the minimum of one measurement point in the following variables, thus complete data sets only were included: CIUS (n = 26 complete data sets), CSAS-PR (n = 16 complete data sets), SDQ-SR (n = 26 complete data sets), APROF (n = 26 complete data sets), and SSL-Learning (n = 26 complete data sets). The exploration of patterns of missing data showed higher strain in non-completers (in CIUS at posttest and at 4-month follow-up; in CSAS-PR at pretest; in SDQ-SR at posttest; in APROF at posttest; in SSL at pretest, at posttest and at 12-month follow-up).
Descriptive data at all measurement points
| Outcome measure | Pretest | Posttest | 4-Month follow-up | 12-Month follow-up | ||||
|---|---|---|---|---|---|---|---|---|
| M | (SD) | M | (SD) | M | (SD) | M | (SD) | |
| Primary outcome measures | ||||||||
| IUD self-report (CIUS; n = 52/44/31/33) | 23.49 | (9.31) | 24.04 | (9.27) | 21.45 | (9.17) | 14.14 | (8.21) |
| IUD self-report (CSAS-SR; n = 52/44/31/33) | 16.75 | (9.43) | 17.21 | (9.35) | 14.58 | (9.09) | 9.75 | (7.35) |
| IUD parental report (CSAS-PR; n = 43/28/32/34) | 35.69 | (9.26) | 27.79 | (11.80) | 29.72 | (12.30) | 26.80 | (11.47) |
| Secondary outcome measures | ||||||||
| Amount of time spent online (n = 50/43/30/33) | 4.14 | (2.30) | 3.87 | (2.61) | 3.94 | (2.47) | 3.49 | (1.84) |
| Psychopathology self-report (SDQ-SR; n = 52/44/31/33) | 10.62 | (5.45) | 11.48 | (5.50) | 11.16 | (5.33) | 9.97 | (5.27) |
| Psychopathology parental report (SDQ-PR; n = 44/28/32/34) | 15.75 | (6.61) | 12.36 | (6.34) | 13.41 | (6.23) | 12.00 | (5.79) |
| Depressive symptoms (DIKJ; n = 52/43/31/32) | 13.25 | (7.09) | 12.58 | (7.43) | 10.84 | (6.74) | 9.78 | (8.40) |
| Social anxiety (SIAS; n = 52/44/31/32) | 21.69 | (13.25) | 20.57 | (14.92) | 20.71 | (12.99) | 17.69 | (14.82) |
| Performance anxiety and school anxiety (PHOKI; n = 52/43/31/32) | 5.58 | (4.25) | 5.88 | (4.83) | 5.35 | (4.40) | 4.31 | (4.43) |
| Procrastination (APROF; n = 52/43/31/32) | 3.40 | (1.16) | 3.33 | (1.18) | 3.31 | (1.08) | 3.38 | (1.26) |
| Functional regulation (FEEL-KJ; n = 52/44/30/33) | 124.94 | (27.84) | 129.36 | (31.08) | 128.50 | (28.83) | 130.82 | (31.09) |
| Dysfunctional regulation (FEEL-KJ; n = 52/44/30/33) | 74.10 | (19.32) | 75.89 | (15.81) | 74.60 | (15.68) | 72.23 | (18.38) |
| Social behavior (SSL; n = 52/43/31/32) | 58.44 | (8.70) | 58.50 | (10.04) | 59.07 | (7.98) | 60.38 | (8.75) |
| Learning behavior (SSL; n = 52/43/31/32) | 33.98 | (7.75) | 34.88 | (8.06) | 37.42 | (6.24) | 35.03 | (8.19) |
| Self-efficacy (SWE; n = 52/43/31/32) | 28.15 | (6.66) | 28.62 | (5.73) | 30.19 | (4.89) | 30.07 | (5.55) |
Note: Amount of Time Spent Online is presented in hours per day. CIUS = Compulsive Internet Use Scale; CSAS-SR = Video Game Dependency Scale Self-Report; CSAS-PR = Video Game Dependency Scale Parental Report; SDQ-SR=Strengths and Difficulties Questionnaire Self-Report; SDQ-PR = Strengths and Difficulties Questionnaire Parental Report; DIKJ = German version of the Children's Depression Inventory; SIAS = Social Interaction Anxiety Scale; PHOKI = German revision of the Fear Survey Schedule for Children-Revised; APROF = German Questionnaire for Procrastination; FEEL-KJ = German Questionnaire for Assessment of Emotion Regulation in Children and Adolescents; SSL = German Student Assessment List for Social and Learning Behavior; SWE = German General Self-Efficacy Scale.
Descriptive statistics chart of the primary outcome variables over the period of 12 months. CIUS = Compulsive Internet Use Scale; CSAS-SR = Video Game Dependency Scale Self-Report; CSAS-PR = Video Game Dependency Scale Parental Report
Citation: Journal of Behavioral Addictions 9, 4; 10.1556/2006.2020.00089
Primary and secondary outcome analyses
Changes in IUD symptoms as well as in comorbid symptoms were examined using growth models in a hierarchical linear model framework. Table 2 presents the results of the unconditional growth models of all primary and secondary outcome measures. All primary outcome variables showed a significant reduction in symptom severity over time and medium to large effect sizes between pretest and 12-month follow-up. The reduction in the amount of time spent online was not significant. Considering second outcome measures, there was a significant reduction in symptom severity in SDQ-PR, DIKJ, SIAS, and PHOKI showing small to medium effect sizes.
Results of fixed effects parameters for change in IUD symptoms and secondary outcomes
| Variable | Parameter | Estimate | SE | t value | P value | d |
|---|---|---|---|---|---|---|
| Primary outcome measures | ||||||
| IUD self-report (CIUS; n = 26) | Intercept | 22.395 | 1.573 | 14.24 | <0.001 | |
| Time | −0.664 | 0.119 | −5.59 | <0.001 | 1.004 | |
| IUD self-report (CSAS-SR; n = 54) | Intercept | 17.283 | 1.261 | 13.71 | <0.001 | |
| Time | −0.534 | 0.091 | −5.86 | <0.001 | 0.743 | |
| IUD parental report (CSAS-PR; n = 16) | Intercept | 29.240 | 2.280 | 12.82 | <0.001 | |
| Time | −0.514 | 0.159 | −3.24 | 0.002 | 0.961 | |
| Secondary outcome measures | ||||||
| Amount of time spent online (n = 54) | Intercept | 0.622 | 0.028 | 21.86 | <0.001 | |
| Time | −0.003 | 0.002 | −1.18 | 0.243 | 0.283 | |
| Psychopathology self-report (SDQ-SR; n = 26) | Intercept | 10.183 | 0.819 | 12.43 | <0.001 | |
| Time | −0.033 | 0.078 | −0.42 | 0.674 | 0.118 | |
| Psychopathology parental report (SDQ-PR; n = 50) | Intercept | 14.732 | 0.857 | 17.19 | <0.001 | |
| Time | −0.177 | 0.064 | −2.76 | 0.007 | 0.567 | |
| Depressive symptoms (DIKJ; n = 54) | Intercept | 3.709 | 0.143 | 25.90 | <0.001 | |
| Time | −0.186 | 0.043 | −4.29 | <0.001 | 0.489 | |
| Social anxiety (SIAS; n = 54) | Intercept | 4.714 | 0.235 | 20.04 | <0.001 | |
| Time | −0.177 | 0.063 | −2.81 | 0.006 | 0.302 | |
| Performance anxiety and school anxiety (PHOKI; n = 54) | Intercept | 5.686 | 0.584 | 9.73 | <0.001 | |
| Time | −0.112 | 0.043 | −2.62 | 0.010 | 0.297 | |
| Procrastination (APROF; n = 26) | Intercept | 3.069 | 0.198 | 15.48 | <0.001 | |
| Time | 0.022 | 0.022 | 1.00 | 0.322 | 0.022 | |
| Functional regulation (FEEL-KJ; n = 54) | Intercept | 126.576 | 3.442 | 36.78 | <0.001 | |
| Time | 0.299 | 0.351 | 0.85 | 0.396 | −0.211 | |
| Dysfunctional regulation (FEEL-KJ; n = 54) | Intercept | 74.441 | 2.223 | 33.48 | <0.001 | |
| Time | −0.167 | 0.221 | −0.759 | 0.450 | 0.097 | |
| Social behavior (SSL; n = 54) | Intercept | 58.210 | 1.162 | 50.09 | <0.001 | |
| Time | 0.155 | 0.111 | 1.40 | 0.164 | −0.222 | |
| Learning behavior (SSL; n = 26) | Intercept | 37.283 | 1.231 | 30.27 | <0.001 | |
| Time | −0.035 | 0.118 | −0.296 | 0.768 | −0.135 | |
| Self-efficacy (SWE; n = 54) | Intercept | 28.692 | 0.760 | 37.75 | <0.001 | |
| Time | 0.115 | 0.066 | 1.74 | 0.085 | −0.289 | |
Note: Intercept reflects pretest score, and time reflects linear change per month. Amount of Time Spent Online is presented in hours per day, the variable was log transformed. CIUS = Compulsive Internet Use Scale; CSAS-SR = Video Game Dependency Scale Self-Report; CSAS-PR = Video Game Dependency Scale Parental Report; SDQ-SR = Strengths and Difficulties Questionnaire Self-Report; SDQ-PR = Strengths and Difficulties Questionnaire Parental Report; DIKJ = German version of the Children's Depression Inventory; SIAS = Social Interaction Anxiety Scale; PHOKI = German revision of the Fear Survey Schedule for Children – Revised; APROF = German Questionnaire for Procrastination; FEEL-KJ = German Questionnaire for Assessment of Emotion Regulation in Children and Adolescents; SSL = German Student Assessment List for Social and Learning Behavior; SWE = German General Self-Efficacy Scale; SE = standard error; d = Cohen's d statistic.
Additional analyses
Aiming to assess variables predicting the individual symptom courses, we conducted exploratory analyses for the CSAS-SR. Conditional growth models including gender, age, gaming vs. non-gaming, the centered pretest CSAS-SR score, as well as the centered pretest scores of the secondary outcomes (amount of time spent online, total score of general psychopathology, total score of depressive symptoms, total score of social anxiety, total score of performance anxiety and school anxiety, average score of procrastination, total scores of functional and dysfunctional emotion regulation skills, total scores of social behavior and learning behavior, and total score of self-efficacy) were computed by including one at a time. Thereof the pretest CSAS-SR score was the only significant predictor indicating an interaction effect with time, i.e. individuals with higher pretest CSAS-SR scores showed a higher rate of change over time (P = 0.005). Beyond the statistical significance, we assessed clinical significance of the reduction in symptom severity of the primary outcomes at 12-month follow-up. The reliable change was computed for CIUS (n = 32) only, given to lacking norms for the adapted version of the CSAS. According to CIUS, 88.9% (n = 8) of the pathological users improved reliably (11.9% showed no reliable change), 80.0% (n = 8) of the high-risk improved reliably (10.0% showed no reliable change and 10.0% showed a reliable deterioration), and 61.5% (n = 8) of the unproblematic users improved reliably (15.4% showed no reliable change and 23.1% showed a reliable deterioration). In total, 25.0% (n = 8) recovered (pathological use at pretest and maximum high-risk use at 12-month follow-up), the incidence rate was 0.0%.
The semi-structured clinical interview measured IGD and ng-PIU separately. The presence of each symptom was rated for the past 4 weeks and for 12 months prior. In case of symptom-onset, the change-point was evaluated using a calendar. In the IGD section 38.9% (n = 14) were classified as high-risk/pathological users at 12 months prior and as unproblematic users during the past 4 weeks. 2.8% (n = 1) were classified as unproblematic users at 12 months prior and as high-risk/pathological users during the past 4 weeks. In the ng-PIU section 33.3% (n = 12) were classified as high-risk/pathological users at 12 months prior and as unproblematic users during the past 4 weeks. 0% was classified as unproblematic users at 12 months prior and as high-risk/pathological users during the past 4 weeks. The McNemar-Test showed a significant difference in the proportion of the categories unproblematic use vs. high-risk/pathological use (IGD P = 0.001; ng-PIU P < 0.001) between the current classification and the retrospective classification from 12 months prior.
Discussion and conclusions
The aim of the current study was the evaluation of the long-term effects of the manualized early intervention program PROTECT+ for adolescents suffering from IUD. The self-selected sample participated in a 4-session brief therapy program and was assessed at pretest, posttest, 4 months and 12 months after admission. It was hypothesized, that the participants would show a reduction in IUD symptom severity after 12 months (primary outcome measure). Furthermore, a reduction in risk-related factors as well as an increase in associated protective variables was hypothesized (secondary outcome measures).
The results showed a significant reduction in both self-reported (CIUS and CSAS-SR) and parental reported (CSAS-PR) symptom severity after 12 months with medium to large effects. The presented results are in line with earlier research which showed beneficial effects of CBT-based interventions in individuals suffering from IUD (Stevens et al., 2018). Moreover, to our knowledge, this is the first study to show large effects over a 12 month period in adolescents.
Although we used multiple primary variables given to the lacking gold standard in measuring IUD, the results consistently show a long-term symptom reduction after 12 months. The reliable change index confirmed the clinical significance of the provided results. Most of the participants showed a reliable improvement, however, a minority showed no reliable change or even a reliable deterioration.
Exploratory growth models revealed more beneficial effects in more impaired individuals. The other potential predictors showed no effect on the symptom course, which could be attributed to the small sample size. Further studies should inquire moderating and mediating effects of third variables for detecting underlying relatedness.
Moreover, changes in the amount of time spent online were not significant. The amount of time spent online or playing games is not a part of the DSM-5 criteria, nevertheless it is an indicator of pathological use. Comparing the pretest scores of the PROTECT+ patients with the German national average, we found that the pretest scores were not elevated. The PROTECT+ patients spent 4.1 hours per day online on average and the national average in adolescents aged 14–29 years is 4.2 hours (Koch & Frees, 2017). Since the goal of the program is not a total abstinence of Internet use and/or gaming activities, but a controlled use, a significant reduction below the national average was not made to be a target.
Looking at the descriptive statistics, an unexpected finding is the initial worsening from pretest to posttest in the self-reported measures. The increase (on average 0.6 CIUS points and 0.5 CSAS-SR points) was not significant, nevertheless it was unexpected. A possible explanation for this might be an increasing problem awareness of the patients during the treatment. Another possible explanation is that adolescents trivialized their Internet use and gaming behavior at the beginning. This would be in line with the relatively low motivation for participation, which also is in line with previous research findings (Lindenberg, Szász-Janocha, Schoenmaekers, Wehrmann, & Vonderlin, 2017; Wölfling et al., 2019). Arguments against a potentially harmful effect of the intervention are first that the initial worsening arised in the self-report only and second that afterward symptom severity declined considerably.
Regarding the comorbid symptoms, descriptive statistics showed an improvement after 12 months in all variables, a significant improvement was found in the parental report of the general psychopathology (SDQ-PR), in depressive symptoms (DIKJ), in social anxiety (SIAS) as well as performance anxiety and school anxiety (PHOKI). This is consistent with previous findings that show an elevated psychopathology as well as a high comorbidity of IUD (Anderson et al., 2016; Carli et al., 2013; Ho et al., 2014; Ko, Yen, Yen, Chen, & Chen, 2012; Mihara & Higuchi, 2017), especially with depression and anxiety disorders (Bargeron & Hormes, 2017; Männikkö et al., 2015; Mentzoni et al., 2011), as well as a reduction in these comorbid symptoms after CBT for treating IUD (Stevens et al., 2018).
Furthermore, the high discrepancy between self-reports and parental reports are noteworthy. One possible explanation might be a lack of problem awareness on part of the adolescents. Another reason could be different limits of tolerance. Encouragingly and most importantly, over 12 months both, parents and adolescents, reported a significant reduction in symptom severity.
Two major limitations of the present study should be considered. First, because of the low motivation for participation, we had to adapt the originally planned randomized controlled study design to a one-armed intervention study without a control group. Although, the efficacy of the PROTECT program has been shown in a school-based prevention trial within the scope of a RCT (Lindenberg, 2018), we do not know how symptom severity would have developed without the intervention in the present trial, taking into account the chance that spontaneous remission could occur in adolescents (Lau, Wu, Gross, Cheng, & Lau, 2017; Wartberg & Lindenberg, 2020). This limitation is a crucial point, since recent reviews criticized a lack of well-designed IUD treatment studies (King et al., 2017; Zajac et al., 2017). The low enrollment rates as well as the high dropout rate seem to be a common difficulty in the treatment of patients suffering from IUD (Lindenberg et al., 2017; Wölfling et al., 2019) and this requires special attention. The dropout rate between enrollment and actual participation (29%) corresponds to other intervention studies treating IUD (Wartberg et al., 2014; Wölfling et al., 2019), and the dropout in general mental health treatment (Wells et al., 2013). All in all, the dropout and absenteeism during the sessions might have biased the presented results. Furthermore, exploratory analyses of differences in participants with complete and with missing data showed higher strain in non-completers in various variables at different measurement points. This indicates an underestimation of symptom severity but should be interpreted with caution given to the small sample sizes. As a result, further research should focus on multicentered and international randomized controlled clinical trials to confirm the presented results.
Secondly, we did not use objective inclusion criteria for study participation, e.g., a diagnostic interview to assure an IUD diagnosis. On the one hand, this was in order to provide the intervention at low-threshold to counteract the low treatment motivation of individuals suffering from IUD (Lindenberg et al., 2017; Wölfling et al., 2019). On the other hand, the study aimed to target participants in early stages of the disease (early intervention program) characterized by suffering from single symptoms but not necessarily fulfilling the full diagnosis of IUD. Furthermore, exploratory analyses showed a higher reduction in symptom severity in individuals with higher IUD symptom scores indicating even more beneficial effects in more impaired individuals. Despite this, a generalization of the presented results cannot be made. Another limitation of the present study is the limited validity of several diagnostic tools. While the existing validation of the psychometric properties of some tools (e.g., CSAS, CIUS) do not cover the total age span of the present sample, the CSAS has been adapted by including IUD in general, which might affect the validity additionally. Furthermore, the clinical interview was only conducted at 12-month follow-up and might be biased by the retrospective application. Since uniform criteria are still lacking, and valid psychometric measures are rare, we applied several primary outcome measures. Nevertheless, there is a need for valid measures of IUD, especially for younger age groups.
In conclusion, the provided clinical trial is to the best of our knowledge the first study evaluating the long-term effects of an early intervention program for adolescents with IUD, that is registered and based on a manual (Lindenberg et al., 2020). Although the efficacy of PROTECT+ should be confirmed in a RCT, the presented results are promising in respect of the effects of CBT treatment of IUD.
Funding sources
This research was funded by University of Education Heidelberg. No conditions were imposed on the financing.
Authors' contribution
Conceptualization, C.S.-J. and K.L.; methodology C.S.-J.; software, C.S.-J.; validation, C.S.-J. and K.L.; formal analysis, C.S.-J.; investigation, C.S.-J. and K.L.; resources, K.L.; data curation, C.S.-J. and K.L.; writing—original draft preparation, C.S.-J.; writing—review and editing, C.S.-J., E.V., K.L.; visualization, C.S.-J.; supervision, E.V. and K.L.; project administration, C.S.-J.; funding acquisition, K.L.”,
Conflict of interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
References
American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders: DSM-5 (5. ed.). Washington, DC: American Psychiatric Assoc.
Anderson, E. L., Steen, E., & Stavropoulos, V. (2016). Internet use and problematic internet use: A systematic review of longitudinal research trends in adolescence and emergent adulthood. International Journal of Adolescence and Youth, 22(4), 430–454. https://doi.org/10.1080/02673843.2016.1227716.
Bakken, I. J., Wenzel, H. G., Götestam, K. G., Johansson, A., & Oren, A. (2009). Internet addiction among Norwegian adults: A stratified probability sample study. Scandinavian Journal of Psychology, 50(2), 121–127. https://doi.org/10.1111/j.1467-9450.2008.00685.x.
Bargeron, A. H., & Hormes, J. M. (2017). Psychosocial correlates of internet gaming disorder: Psychopathology, life satisfaction, and impulsivity. Computers in Human Behavior, 68, 388–394. https://doi.org/10.1016/j.chb.2016.11.029.
Brunborg, G. S., Mentzoni, R. A., & Frøyland, L. R. (2014). Is video gaming, or video game addiction, associated with depression, academic achievement, heavy episodic drinking, or conduct problems?. Journal of Behavioral Addictions, 3(1), 27–32. https://doi.org/10.1556/JBA.3.2014.002.
Carli, V., Durkee, T., Wasserman, D., Hadlaczky, G., Despalins, R., Kramarz, E., et al. (2013). The association between pathological internet use and comorbid psychopathology: A systematic review. Psychopathology, 46(1), 1–13. https://doi.org/10.1159/000337971.
Cheng, C., & Li, A. Y.-l. (2014). Internet addiction prevalence and quality of (real) life: A meta-analysis of 31 nations across seven world regions. Cyberpsychology, Behavior, and Social Networking, 17(12), 755–760. https://doi.org/10.1089/cyber.2014.0317.
Döpfner, M., Schnabel, M., Goletz, H., & Ollendick, T. H. (2006). Phobiefragebogen für Kinder und Jugendliche: PHOKI. Göttingen: Hogrefe.
Du, Y.-S., Jiang, W., & Vance, A. (2010). Longer term effect of randomized, controlled group cognitive behavioural therapy for Internet addiction in adolescent students in Shanghai. Australian and New Zealand Journal of Psychiatry, 44(2), 129–134. https://doi.org/10.3109/00048670903282725.
Fam, J. Y. (2018). Prevalence of internet gaming disorder in adolescents: A meta-analysis across three decades. Scandinavian Journal of Psychology, 59(5), 524–531. https://doi.org/10.1111/sjop.12459.
Goodman, R. (1997). The strengths and difficulties questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38(5), 581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x.
Goodman, R., Meltzer, H., & Bailey, V. (2003). The strengths and Difficulties questionnaire: A pilot study on the validity of the self-report version. International Review of Psychiatry (Abingdon, England), 15(1–2), 173–177. https://doi.org/10.1080/0954026021000046137.
Grob, A., & Smolenski, C. (2009). FEEL-KJ: Fragebogen zur Erhebung der Emotionsregulation bei Kindern und Jugendlichen. Bern.
Gürtler, D., Rumpf, H.-J., Bischof, A., Kastirke, N., Meerkerk, G.-J., John, U., et al. (2014). Psychometrische Eigenschaften und Normierung der deutschen Version der Compulsive Internet Use Scale (CIUS). Diagnostica, 61(4), 210–221. https://doi.org/10.1026/0012-1924/a000127.
Höcker, A., Engberding, M., & Rist, F. (2013). Prokrastination: Ein Manual zur Behandlung des pathologischen Aufschiebens; [mit CD-ROM]. Therapeutische Praxis. Göttingen: Hogrefe.
Ho, R. C., Zhang, M. W. B., Tsang, T. Y., Toh, A. H., Pan, F., Lu, Y., et al. (2014). The association between internet addiction and psychiatric co-morbidity: A meta-analysis. BMC Psychiatry, 14, 183. https://doi.org/10.1186/1471-244X-14-183.
IBM Corp. Released (2017). IBM SPSS statistics for windows (Version 25.0) [Computer software]. Armonk, NY: IBM Corp.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19.
Kim, N. R., Hwang, S. S.-H., Choi, J.-S., Kim, D.-J., Demetrovics, Z., Király, O., et al. (2016). Characteristics and psychiatric symptoms of internet gaming disorder among adults using self-reported DSM-5 criteria. Psychiatry Investigation, 13(1), 58–66. https://doi.org/10.4306/pi.2016.13.1.58.
King, D. L., Delfabbro, P. H., Wu, A. M. S., Doh, Y. Y., Kuss, D. J., Pallesen, S., et al. (2017). Treatment of internet gaming disorder: An international systematic review and CONSORT evaluation. Clinical Psychology Review, 54, 123–133. https://doi.org/10.1016/j.cpr.2017.04.002.
Király, O., Griffiths, M. D., Urbán, R., Farkas, J., Kökönyei, G., Elekes, Z., et al. (2014). Problematic internet use and problematic online gaming are not the same: Findings from a large nationally representative adolescent sample. Cyberpsychology, Behavior, and Social Networking, 17(12), 749–754. https://doi.org/10.1089/cyber.2014.0475.
Koch, W., & Frees, B. (2017). ARD/ZDF-Onlinestudie 2017: Neun von zehn Deutschen online. Media Perspektiven, 9, 434–446. Retrieved from http://www.ard-zdf-onlinestudie.de/files/2017/Artikel/917_Koch_Frees.pdf.
Kovacs, M. (1985). The children's depression, inventory (CDI). Psychopharmacology Bulletin, 21(4), 995–998.
Ko, C. H., Yen, J. Y., Yen, C. F., Chen, C. S., & Chen, C. C. (2012). The association between internet addiction and psychiatric disorder: A review of the literature. European Psychiatry: The Journal of the Association of European Psychiatrists, 27(1), 1–8. https://doi.org/10.1016/j.eurpsy.2010.04.011.
Ko, C.-H., Yen, J.-Y., Yen, C.-F., Chen, C.-S., Weng, C.-C., & Chen, C.-C. (2008). The association between internet addiction and problematic alcohol use in adolescents: The problem behavior model. CyberPsychology and Behavior: The Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society, 11(5), 571–576. https://doi.org/10.1089/cpb.2007.0199.
Kriminologisches Forschungsinstitut Niedersachsen e.V. Deutschlandweite repräsentativbefragung computerspielabhängigkeit, unpublished manuscript.
Kuss, D. J., Griffiths, M. D., Karila, L., & Billieux, J. (2014). Internet addiction: A systematic review of epidemiological research for the last decade. Current Pharmaceutical Design, 20(25), 4026–4052.
Lindenberg, K. (2018). 12-Monats-Effekte der PROTECT Studie: Wirksamkeit eines kognitiv-verhaltenstherapeutischen Trainings zur indizierten Prävention von Internetbezogenen Störungen. SUCHT, 64(S1), 78–79.
Lindenberg, K., Kindt, S., & Szász-Janocha, C. (2020). Internet addiction in adolescents: The PROTECT program for evidence-based prevention and treatment. Cham: Springer.
Lindenberg, K., Szász-Janocha, C., Schoenmaekers, S., Wehrmann, U., & Vonderlin, E. (2017). An analysis of integrated health care for Internet Use Disorders in adolescents and adults. Journal of Behavioral Addictions, 6(4), 579–592. https://doi.org/10.1556/2006.6.2017.065.
Lau, J. T. F., Wu, A. M. S., Gross, D. L., Cheng, K.-M., & Lau, M. M. C. (2017). Is Internet addiction transitory or persistent? Incidence and prospective predictors of remission of Internet addiction among Chinese secondary school students. Addictive Behaviors, 74, 55–62. https://doi.org/10.1016/j.addbeh.2017.05.034.
Li, H., & Wang, S. (2013). The role of cognitive distortion in online game addiction among Chinese adolescents. Children and Youth Services Review, 35(9), 1468–1475. https://doi.org/10.1016/j.childyouth.2013.05.021.
Lohbeck, A., Schultheiß, J., Petermann, F., & Petermann, U. (2015). Die deutsche Selbstbeurteilungsversion des Strengths and Difficulties Questionnaire (SDQ-Deu-S). Diagnostica, 61(4), 222–235. https://doi.org/10.1026/0012-1924/a000153.
Männikkö, N., Billieux, J., & Kääriäinen, M. (2015). Problematic digital gaming behavior and its relation to the psychological, social and physical health of Finnish adolescents and young adults. Journal of Behavioral Addictions, 4(4), 281–288. https://doi.org/10.1556/2006.4.2015.040.
Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour Research and Therapy, 36(4), 455–470.
Meerkerk, G.-J., Van Den Eijnden, R. J. J. M., Vermulst, A. A., & Garretsen, H. F. L. (2009). The compulsive internet use scale (CIUS): Some psychometric properties. CyberPsychology and Behavior, 12(1), 1–6.
Mentzoni, R. A., Brunborg, G. S., Molde, H., Myrseth, H., Skouveroe, K. J. M., Hetland, J., et al. (2011). Problematic video game use: Estimated prevalence and associations with mental and physical health. Cyberpsychology, Behavior, and Social Networking, 14(10), 591–596. https://doi.org/10.1089/cyber.2010.0260.
Mihara, S., & Higuchi, S. (2017). Cross-sectional and longitudinal epidemiological studies of internet gaming disorder: A systematic review of the literature. Psychiatry and Clinical Neurosciences, 71(7), 425–444. https://doi.org/10.1111/pcn.12532.
Ollendick, T. H. (1983). Reliability and validity of the revised fear survey schedule for children (FSSC-R). Behaviour Research and Therapy, 21(6), 685–692. https://doi.org/10.1016/0005-7967(83)90087-6.
Ostovar, S., Allahyar, N., Aminpoor, H., Moafian, F., Nor, M. B. M., & Griffiths, M. D. (2016). Internet addiction and its psychosocial risks (depression, anxiety, stress and loneliness) among Iranian adolescents and young adults: A structural equation model in a cross-sectional study. International Journal of Mental Health and Addiction, 14(3), 257–267. https://doi.org/10.1007/s11469-015-9628-0.
Petermann, U., & Petermann, F. (2014). Schülereinschätzliste für Sozial-und Lernverhalten: SSL .Göttingen: Hogrefe.
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & R Core Team (2019). NLME: Linear and nonlinear mixed effects models (version R package version 3.1-143) [Computer software]. Retrieved from https://CRAN.R-project.org/package=nlme.
Rehbein, F., Baier, D., Kleimann, M., & Mößle, T. (2015). CSAS: Computerspielabhängigkeitsskala. Ein Verfahren zur Erfassung der Internet Gaming Disorder nach DSM-5. Göttingen: Hogrefe Verlag GmbH & Co. KG.
Rehbein, F., Kliem, S., Baier, D., Mößle, T., & Petry, N. M. (2015). Prevalence of internet gaming disorder in German adolescents: Diagnostic contribution of the nine DSM-5 criteria in a state-wide representative sample. Addiction (Abingdon, England), 110(5), 842–851. https://doi.org/10.1111/add.12849.
Schwarzer, R., & Jerusalem, M. (1999). Skalen zur Erfassung von Lehrer- und Schülermerkmalen. Dokumentation der psychometrischen Verfahren im Rahmen der Wissenschaftlichen Begleitung des Modellversuchs Selbstwirksame Schulen. Berlin: Freie Universität Berlin.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. Oxford: Oxford Univ. Press. Retrieved from http://www.loc.gov/catdir/enhancements/fy0612/2002007055-d.html.
Stangier, U., Heidenreich, T., Berardi, A., Golbs, U., & Hoyer, J. (1999). Die Erfassung sozialer Phobie durch die Social Interaction Anxiety Scale (SIAS) und die Social Phobia Scale (SPS). Zeitschrift für Klinische Psychologie und Psychotherapie, 28(1), 28–36. https://doi.org/10.1026//0084-5345.28.1.28.
Stevens, M. W. R., King, D. L., Dorstyn, D., & Delfabbro, P. H. (2018). Cognitive-behavioral therapy for internet gaming disorder: A systematic review and meta-analysis. Clinical Psychology & Psychotherapy, 1–13. https://doi.org/10.1002/cpp.2341.
Stiensmeier-Pelster, J., Schürmann, M., & Duda, K. (2000). Depressions-Inventar für Kinder und Jugendliche:(DIKJ). Göttingen: Hogrefe.
Strittmatter, E., Kaess, M., Parzer, P., Fischer, G., Carli, V., Hoven, C. W., et al. (2015). Pathological internet use among adolescents: Comparing gamers and non-gamers. Psychiatry Research, 228(1), 128–135. https://doi.org/10.1016/j.psychres.2015.04.029.
Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics (6th ed., Pearson new international edition). Pearson custom library. Harlow, Essex: Pearson Education.
Wang, H. R., Cho, H., & Kim, D.-J. (2018). Prevalence and correlates of comorbid depression in a nonclinical online sample with DSM-5 internet gaming disorder. Journal of Affective Disorders, 226, 1–5. https://doi.org/10.1016/j.jad.2017.08.005.
Wartberg, L., & Lindenberg, K. (2020). Predictors of Spontaneous Remission of Problematic Internet Use in Adolescence: A One-Year Follow-Up Study. International Journal of Environmental Research and Public Health, 17(2). https://doi.org/10.3390/ijerph17020448.
Wartberg, L., Thomsen, M., Moll, B., & Thomasius, R. (2014). Pilotstudie zur Effektivität eines kognitiv-verhaltenstherapeutischen Gruppenprogramms mit psychoedukativen Anteilen für Jugendliche mit pathologischem Internetgebrauch [Pilot study on the effectiveness of a cognitive behavioural group programme for adolescents with pathological internet use]. Praxis der Kinderpsychologie und Kinderpsychiatrie, 63(1), 21–35.
Wells, J. E., Browne, M. O., Aguilar-Gaxiola, S., Al-Hamzawi, A., Alonso, J., Angermeyer, M. C., et al. (2013). Drop out from out-patient mental healthcare in the world health organization's world mental health survey initiative. The British Journal of Psychiatry: The Journal of Mental Science, 202(1), 42–49. https://doi.org/10.1192/bjp.bp.112.113134.
Wittek, C. T., Finserås, T. R., Pallesen, S., Mentzoni, R. A., Hanss, D., Griffiths, M. D., et al. (2016). Prevalence and predictors of video game addiction: A study based on a national representative sample of gamers. International Journal of Mental Health and Addiction, 14(5), 672–686. https://doi.org/10.1007/s11469-015-9592-8.
Woerner, W., Becker, A., & Rothenberger, A. (2004). Normative data and scale properties of the German parent SDQ. European Child & Adolescent Psychiatry, 13(Suppl 2), ii3-ii10. https://doi.org/10.1007/s00787-004-2002-6.
Wölfling, K., Müller, K. W., Dreier, M., Ruckes, C., Deuster, O., Batra, A., et al.(2019). Efficacy of short-term treatment of internet and computer game addiction: A randomized clinical trial. JAMA Psychiatry, 76(10), 1018–1025. https://doi.org/10.1001/jamapsychiatry.2019.1676.
World Health Organization (2018). ICD-11: International classification of diseases 11th revision. Retrieved from https://icd.who.int/browse11/l-m/en.
Yen, J.-Y., Ko, C.-H., Yen, C.-F., Wu, H.-Y., & Yang, M.-J. (2007). The comorbid psychiatric symptoms of internet addiction: Attention deficit and hyperactivity disorder (ADHD), depression, social phobia, and hostility. Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, 41(1), 93–98. https://doi.org/10.1016/j.jadohealth.2007.02.002.
Yen, J.-Y., Liu, T.-L., Wang, P.-W., Chen, C.-S., Yen, C.-F., & Ko, C.-H. (2017). Association between Internet gaming disorder and adult attention deficit and hyperactivity disorder and their correlates: Impulsivity and hostility. Addictive Behaviors, 64, 308–313. https://doi.org/10.1016/j.addbeh.2016.04.024.
Zajac, K., Ginley, M. K., Chang, R., & Petry, N. M. (2017). Treatments for internet gaming disorder and internet addiction: A systematic review. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 31(8), 979–994. https://doi.org/10.1037/adb0000315.
