Abstract
Background and aims
Given the growing epidemiological research interest concerning Internet addiction, brief instruments with a robust theoretical basis are warranted. The Internet Disorder Scale (IDS-15) is one such instrument that can be used to quickly assess the Internet addiction in an individual. However, only two language versions of the IDS-15 have been developed. This study translated the IDS-15 into Persian and examined its psychometric properties using comprehensive psychometric testing.
Methods
After ensuring the linguistic validity of the Persian IDS-15, 1,272 adolescents (mean age = 15.53 years; 728 males) completed the IDS-15, Depression Anxiety Stress Scale (DASS), Internet Gaming Disorder Scale – Short Form (IGDS9-SF), and the Bergen Social Media Addiction Scale (BSMAS). Confirmatory factor analysis (CFA), Rasch models, regression analysis, and latent profile analysis (LPA) were carried out to test the psychometric properties of the Persian IDS-15.
Results
Both CFA and Rasch supported the construct validity of the Persian IDS-15. Multigroup analysis in CFA and differential item functioning in Rasch indicated that male and female adolescents interpreted the IDS-15 items similarly. Regression analysis showed that the IDS-15 correlated with IGDS9-SF and BSMAS (ΔR2 = .12 and .36, respectively) is stronger than the DASS (ΔR2 = .03–.05). LPA based on IDS-15 suggests three subgroups for the sample. Significant differences in depression, anxiety, IGDS9-SF, and BSMAS were found among the three LPA subgroups.
Conclusion
The Persian IDS-15 has robust psychometric properties as evidenced by both classical test theory and Rasch analysis.
Introduction
Recent research has reported that the median percentage of Internet usage in 2015 was 54% across 21 emerging and developing countries (e.g., Malaysia and mainland China) and 87% across 11 advanced economies (e.g., the United States and Canada; Pew Research Center, 2016). Given the high prevalence of Internet use and smartphone use, research has also indicated that children and adolescents are at risk of addiction to Internet-related activities (Pontes, Szabo, & Griffiths, 2015; Yang, Chen, Huang, Lin, & Chang, 2017). Consequently, over the past two decades, Internet addiction has received growing interest and attention from healthcare professionals, including researchers and clinicians (Kuss, Griffiths, Karila, & Billieux, 2014; Pontes, Kuss, & Griffiths, 2015). Although Internet-related activities provide convenience and enjoyment to individuals, a large amount of research consistently indicated the negative impacts of Internet addiction among a small minority of individuals (e.g., Rücker, Akre, Berchtold, & Suris, 2015; Sariyska, Reuter, Lachmann, & Montag, 2015; Wartberg, Kriston, Kammerl, Petersen, & Thomasius, 2015; Weinstein, Dorani, Elhadif, Bukovaz, & Yarmulnik, 2015). For example, Sariyska et al. (2015) reported that Internet addiction is associated with attention-deficithyperactivity disorder and depression. Wartberg et al. (2015) reported that Internet addiction was associated with impaired family functioning, lowered life satisfaction, and problematic family interaction among adolescents. Weinstein et al. (2015) reported that Internet addiction was related to social anxiety among young adults. Rücker et al. (2015) suggested that Internet addiction was correlated with substance abuse, poor emotional well-being, and decreased academic performance among adolescents. Moreover, a recent nationally representative study by Macur and Pontes (2018) found that Slovenian adolescents presenting high risk of Internet addiction exhibited poorer levels of self-control and were generally less satisfied with their lives in comparison with adolescents presenting low risk of Internet addiction.
The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) working group led by Petry acknowledged that the healthy discussion among experts and the available empirical evidence led the DSM-5 to include a subtype of Internet addiction, Internet gaming disorder (IGD; Griffiths et al., 2016; Petry et al., 2014). However, the research in the Internet addiction field lacks consensus in terms of definition and characterization of the phenomenon. Consequently, many different instruments assessing Internet addiction have been developed, often with inconsistent criteria (Petry, Zajac, & Ginley, 2018). The discrepancies among the Internet addiction instruments often relate to the diverse theoretical basis, different factorial structure and psychometric properties (Király, Nagygyörgy, Koronczai, Griffiths, & Demetrovics, 2014; Pontes & Griffiths, 2017). Another flaw of these instruments is that most of them (as noted by Kuss et al., 2014) are based on outdated criteria of pathological gambling and/or substance dependence in the DSM-IV (American Psychiatric Association [APA], 1994) rather than the more updated clinical criteria, such as those developed for IGD in the latest (fifth) edition of the DSM-5 (APA, 2013) and those for Gaming Disorder in the latest (11th) revision of the International Classification of Diseases (Petry et al., 2018). Therefore, Internet addiction instruments developed to date using DSM-IV criteria may not be directly capturing the concept of Internet addiction.
Researchers have suggested the appropriateness of applying the IGD criteria to assess Internet addiction (e.g., Pontes & Griffiths, 2017; Rumpf et al., 2015). To the best of the authors’ knowledge, only two instruments [the Internet Addiction Scale and the Internet Disorder Scale (IDS-15)] based on the DSM-5 IGD criteria have been developed to assess Internet addiction (Cho et al., 2014; Pontes & Griffiths, 2017). However, the Internet Addiction Scale (Cho et al., 2014) suffers from several weaknesses, including being lengthy (41 items), which is a disadvantage for time-limited research, as well as rather weak psychometric properties (e.g., low internal consistency of .49 and .65 reported by Cronbach’s α). In contrast, IDS-15 has only 15 items with much stronger psychometric properties (α = .79–.85; Pontes & Griffiths, 2017).
The IDS-15 has been validated only in English and Italian (Monacis, Sinatra, Griffiths, & de Palo, 2018; Pontes & Griffiths, 2017). Therefore, this study translated the IDS-15 into Persian due to the large population of Persian-speaking people globally (approximately 120 million people in Iran, Afghanistan, and Tajikistan). In addition, Internet-based behaviors may differ across different cultures. Studies examining the psychometric properties of the IDS-15 in other countries and languages are thus warranted. Moreover, to be confident that the IDS-15 is psychometrically robust, the psychometric properties should be tested repetitively, especially using a variety of statistical methods in different populations (or ethnicities). Because the two previous psychometric studies on IDS-15 mainly used classical test theory, this study used Rasch models to test the psychometric properties of the IDS-15 as a way of providing additional support to the limited literature concerning the IDS-15.
Because adolescence is a critical transition period (i.e., rendering them more vulnerable to addictive behaviors and other disorders), many adolescents experience extensive physical and psychological changes (Lin & Tsai, 2016) that lead to emotional distress (Strong, Tsai, Lin, & Cheng, 2016) and excessive Internet use (Skoog, Stattin, & Kerr, 2009). In addition, because of the significant physical, biological, cognitive, and social changes during adolescence, many mental health disorders first emerge during this period (Davidson, Grigorenko, Boivin, Rapa, & Stein, 2015). Therefore, adolescents are at high risk for developing Internet addiction. Fortunately, Internet addiction can be diagnosed early and early intervention may be provided, if there are brief and psychometrically robust instruments to screen those with potential problems. Given the aforementioned rationale, this study rigorously tested the psychometric properties of the IDS-15 among adolescents in a Middle East country (i.e., Iran). Two types of psychometric testing (classical test theory and Rasch) were applied to comprehensively and thoroughly examine the psychometric properties of the IDS-15. Given that males and females have different characteristics and patterns in using the Internet (Tsai, Strong, Chen, Lee, & Lin, 2018; Yang, Lin, Huang, & Chang, 2018), the psychometric testing also investigated whether the IDS-15 can be appropriately used for both genders. In addition, latent profile analysis (LPA) was conducted to cluster the participants into several latent subgroups across different levels of Internet addiction and to further assess the characteristics of the subgroups.
Methods
Participants and procedures
This study was conducted from September to December 2017. A multistage sampling approach was used to select the study sample from 58 high schools in Qazvin. Twenty-two high schools were randomly selected from a list of high schools provided by the Organization for Education at Qazvin. Two classes were then randomly selected in each school. Of the 1,483 adolescents who were initially approached, only 1,272 (85.7%) agreed to participate in the study.
With respect to the translation process, a standardized procedure was adopted with three stages (Beaton, Bombardier, Guillemin, & Ferraz, 2000; Khoshnevisan et al., 2012) to translate the English IDS-15 into Persian. First, the English IDS-15 was independently translated into two Persian forward translation versions by two bilingual translators whose native language was Persian. Second, a merged Persian version was synthesized from the two forward translations by the two aforementioned translators and a project manager. Third, the merged Persian version was independently back-translated into two English backward translations by two native English translators who were fluent in Persian. All translated versions (two forward translation versions, one merged Persian version, and two backward translation versions) were then reviewed by an expert committee comprising a psychologist, pediatrician, psychiatrist, psychometrician, public health specialist, health education specialist, and nurse to ensure semantic, idiomatic, experiential, and conceptual equivalencies. After the review from the committee, an interim Persian version was generated and pretested on a sample of Iranian adolescents (n = 34, 19 boys; mean age = 15.10 years).
Instruments
Internet Disorder Scale (IDS-15)
The IDS-15 consists of four subscales comprising escapism and dysfunctional emotional coping (EDEC; four items), withdrawal symptoms (WS; four items), impairments and dysfunctional self-regulation (IDSR; four items), and dysfunctional Internet-related self-control (DISC; three items). All the items are rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) with higher scores on the IDS-15 indicating a higher likelihood of Internet addiction (Monacis et al., 2018). The English IDS-15 had satisfactory internal consistency (α = .79–.85), construct validity, and criterion-related validity (Pontes & Griffiths, 2017).
Depression Anxiety Stress Scale (DASS)
The DASS consists of three 7-item subscales comprising depression, anxiety, and stress. All items are rated on a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time) with higher scores indicating higher levels of psychological distress (Lovibond & Lovibond, 1995). The Persian DASS had acceptable internal consistency (α = .84 for depression, .91 for anxiety, and .86 for stress) and supported the factorial structure reported in previous studies (Asghari, Saed, & Dibajnia, 2008; Lin, Broström, Nilsen, Griffiths, & Pakpour, 2017).
Internet Gaming Disorder Scale – Short Form (IGDS9-SF)
The IGDS9-SF assesses IGD using nine items and was developed based on the IGD criteria in the DSM-5 (Pontes & Griffiths, 2015). All the items are rated on a 5-point Likert scale ranging from 1 (never) to 5 (very often) with a higher score indicating a higher likelihood of IGD. The Persian IGDS9-SF has satisfactory internal consistency (α = .90), promising construct validity, and supported measurement invariant across gender (Wu et al., 2017).
Bergen Social Media Addiction Scale (BSMAS)
The BSMAS (Andreassen et al., 2016) was adapted from the Bergen Facebook Addiction Scale (Andreassen, Torsheim, Brunborg, & Pallesen, 2012) and consists of six items comprising six core addiction elements (salience, mood modification, tolerance, withdraw, conflict, and relapse) proposed by Griffiths (2005). All the items are rated on a 5-point Likert scale ranging from 1 (very rarely) to 5 (very often), and a higher score indicates a greater likelihood of addiction to social media (Andreassen et al., 2016). More recently, Bányai et al. (2017) reported that a score on the BSMAS of over 19 (out of 30) represents at-risk problematic use of social media. The Persian BSMAS has satisfactory internal consistency (α = .86), promising construct validity, and supported measurement invariance across gender (Lin et al., 2017).
Data analysis
Psychometric testing using classical test theory
Ceiling and floor effects were conducted for each subscale of the IDS-15 and the IDS-15 total score, and a percentage of <20% is deemed as acceptable (Jette, Warren, & Wirtalla, 2005). Internal consistency was performed using Cronbach’s α and a value >.7 indicates satisfactory reliability (Cheng, Luh, Yang, Su, & Lin, 2016). Corrected item-total correlation was calculated to demonstrate how an item adheres to its construct and a value of >.4 is deemed as acceptable (Wang, Wang, & Shee, 2007). Standard error of measurement was computed to assess how the observed scores were attributed from the measurement errors, and a low value is anticipated (Cheng et al., 2016). Test–retest reliability was assessed using intraclass correlational coefficient and a value >.7 represents being acceptable.
Construct validity was examined via confirmatory factor analysis (CFA) using diagonally weighted least squares means and variance estimation, and average variance extracted and composite reliability were additionally computed using the factor loadings and uniqueness values retrieved from the CFA; average variance extracted >0.5 and composite reliability >0.6 are the recommended cutoffs (Bagozzi & Yi, 1988; Fornell & Larcker, 1981). Two CFA structures were examined according to the suggestions from previous studies on IDS-15 (Monacis et al., 2018; Pontes & Griffiths, 2017). More specifically, (a) a second-order structure with four first-order factors (EDEC, WS, IDSR, and DISC) and one second-order factor (IDS) and (b) a first-order structure with the aforementioned four first-order factors, which are mutually correlated in the structure. Each IDS-15 item was then evaluated using both loading and significance levels, where a loading >0.4 (Brown, 2006) with statistical significance indicates an acceptable item. A series of fit indices were used to determine whether the structure of the IDS-15 was supported: comparative fit index (CFI = 0.90) and Tucker–Lewis index (TLI >0.95) indicate acceptable fit and excellent fit, respectively (Hu & Bentler, 1998); root mean square error of approximation (RMSEA) <0.05, <0.08, and >0.10 indicate close, reasonable, and poor fit, respectively (Browne & Cudeck, 1993); standardized root mean square residual (SRMR) <0.08 (Cheng et al., 2016; Lin et al., 2012). Concurrent validity was assessed using the associations between the IDS-15 and other psychometric instruments (DASS, IGDS9-SF, and BSMAS). More specifically, several regression models were constructed using IDS-15 as the dependent variable with one of the following independent variables: DASS depression subscale, DASS anxiety subscale, DASS stress subscale, IGDS9-SF, and BSMAS. Moreover, two confounders (age and gender) were included for all the regression models as suggested in a previous study (Pontes, 2017).
Multigroup CFA with three models was used to examine the measurement invariance of the IDS-15 in the scale level. The three models included a configural model (Model 1 – a first-order IDS-15 framework with all items loading on the underlying concept of Internet addiction), a metric invariance model (Model 2 – a model based on Model 1 to constrain all factor loadings being equal across gender), and a scalar invariance model (Model 3 – a model based on Model 2 to additionally constrain all item intercepts being equal across gender). Models 1 and 2 were compared for metric invariance and Models 2 and 3 for scalar invariance (Bagheri, Jafari, Tashakor, Kouhpayeh, & Riazi, 2014); ΔCFI >−0.01, ΔSRMR <0.01, and ΔRMSEA <0.015 suggested measurement invariance across groups (Chen, 2007).
Psychometric testing using Rasch analysis
Rasch partial credit model was used to report item difficulty with the unit of logit and to examine item validity using information-weighted fit statistic (infit) mean square (MnSq) and outlier-sensitive fit statistic (outfit) MnSq. The main reason for using a partial credit model rather than a graded response model is because the intervals between every two descriptors in the Likert-type scale were different in every item. A good fit of an item has both infit and outfit MnSq between 0.5 and 1.5 (Jafari, Bagheri, & Safe, 2012). A low MnSq indicates redundancy of an item, and a high MnSq indicates misfit of an item (Khan, Chien, & Brauer, 2013; Strong, Lin, Tsai, & Lin, 2018). Item and person reliability were generated and a value >.7 is suggested as acceptable (Chang, Wang, Tang, Cheng, & Lin, 2014).
In addition, differential item functioning (DIF) was conducted based on the Rasch models to test measurement invariance across gender at item level. More specifically, DIF tests the measurement invariance for each IDS-15 item. Hence, DIF clearly identify which items for males as compared with females are easier or harder to fulfill the item description (Lin, Ou, et al., 2018; Lin, Pakpour, et al., 2018). A commonly used cutoff of >0.5 was used in the DIF contrast (i.e., the difference of difficulty between two groups) for indicating substantial DIF (Shih & Wang, 2009).
LPA and comparisons among different subgroups
LPA is a person-centered approach that helps to identify latent subgroups of individuals within a population. LPA was used to classify the adolescents into categories based on the four self-reported level of the risk of Internet disorder using IDS-15 subscale scores. The optimal number of classes was selected based on five fit statistics: the Akaike information criterion (AIC), the Bayesian information criterion (BIC), the sample-size adjusted BIC (SSABIC), Lo–Mendell–Rubin (LMR) Likelihood Ratio Test (LRT), entropy and bootstrap LRT. Lower AIC, BIC, and SSABIC values suggest better-fitting models while the entropy measure ranges from 0 to 1 with higher scores demonstrating a better classification quality. In addition, the LMR in the LRT examines whether a model with k profiles provides a significant improvement in fit over a model with k−1 profiles. A non-significant value (i.e., p > .05) indicates that the model with one fewer class provides a more parsimonious fit to the data. Age and gender were examined if they were associated with each profile of membership. To examine sociodemographic characteristics as well as addiction scales as predictors of risk Internet addiction, Multinomial regression using the R3Step method was used. The data were analyzed using Mplus (version 7.4; Los Angeles, CA) and Winstep software (version 3.70.0.2; Winstep Software Technologies, Chicago, IL).
Ethics
The study procedures were carried out in accordance with the Declaration of Helsinki, and the study was approved by the Ethics committee of Qazvin University of Medical Sciences. All participants were informed about the study, fully understood the study purpose, and all provided informed consent. It should also be noted that parental consent was sought for those participants younger than 18 years of age. For this reason, all parents of adolescents younger than 18 years were encouraged to attend a session providing information about the study. Those adolescents whose parents gave their consent were contacted and asked to participate in the study after describing the study aims and procedures.
Results
Participant characteristics
The mean age of the participants was 15.53 years (SD = 1.2; n = 1,272; 728 males). Their average scores on the questionnaires were 46.97 for IDS-15 (SD = 9.78), 7.5 for DASS depression subscale (SD = 4.6), 8.1 for DASS anxiety subscale (SD = 5.0), 7.6 for DASS stress subscale (SD = 4.7), 12.6 for IGDS9-SF (SD = 3.8), and 13.6 for BSMAS (SD = 2.8) (see Table 1 for further details).
Participants’ characteristics (N = 1,272)
| Mean ± SD or n (%) | |
|---|---|
| Age (years) | 15.53 ± 1.2 |
| Gender (male) | 728 (57.2) |
| Family income | |
| Poor (<$600) | 136 (10.7) |
| Fair ($600–1,200) | 814 (64.0) |
| Good (>$1,200) | 320 (25.2) |
| Father’s educational year | 7.8 ± 3.8 |
| Mother’s educational year | 6.1 ± 3.5 |
| Internet Disorder Scale total score | 46.97 ± 9.78 |
| EDEC subscale score | 4.14 ± 0.71 |
| WS subscale score | 2.62 ± 0.77 |
| IDSR subscale score | 3.67 ± 1.03 |
| DISC subscale score | 2.66 ± 0.84 |
| DASS depression subscale score | 7.5 ± 4.6 |
| DASS anxiety subscale score | 8.1 ± 5.0 |
| DASS stress subscale score | 7.6 ± 4.7 |
| Internet gaming disorder | 12.6 ± 3.8 |
| Social media addiction | 13.6 ± 2.8 |
Note. SD: standard deviation; EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; DASS: Depression Anxiety Stress Scale.
Psychometric properties of the IDS-15 at item level
All items in the IDS-15 had acceptable factor loadings derived from the second-order CFA (0.64–0.86), adequate values in corrected item-total correlation (0.76–0.89), and satisfactory test–retest reliability (0.70–0.88). In addition, all items exhibited excellent fit in the Rasch analyses: infit MnSq between 0.81 and 1.23 and outfit MnSq between 0.80 and 1.34. No substantial DIF was displayed for all the items across gender (DIF contrasts = −0.50–0.44). Moreover, the difficulty of the item was between −1.69 and 1.27 logit (Table 2).
Psychometric properties of the Internet Disorder Scale (IDS-15) at the item level
| Analyses from classical test theory | Rasch analyses | ||||||
|---|---|---|---|---|---|---|---|
| Item no. | Factor loadinga* | Item-total correlation | Test–retest reliabilityb | Infit MnSq | Outfit MnSq | Difficulty | DIF contrast across genderc,d |
| EDEC-1 | 0.77 | 0.83 | 0.81 | 0.99 | 1.05 | −1.69 | 0.00 |
| EDEC-2 | 0.64 | 0.78 | 0.72 | 1.23 | 1.34 | −1.05 | −0.16 |
| EDEC-3 | 0.80 | 0.83 | 0.78 | 0.85 | 0.90 | −1.31 | 0.08 |
| EDEC-4 | 0.76 | 0.82 | 0.77 | 0.99 | 1.00 | −1.38 | 0.17 |
| WS-1 | 0.65 | 0.86 | 0.82 | 0.94 | 0.97 | 1.20 | −0.33 |
| WS-2 | 0.68 | 0.89 | 0.88 | 1.01 | 1.08 | 1.27 | −0.13 |
| WS-3 | 0.71 | 0.88 | 0.79 | 0.95 | 1.02 | 0.98 | −0.49 |
| WS-4 | 0.70 | 0.87 | 0.73 | 0.86 | 0.92 | 0.88 | −0.50 |
| IDSR-1 | 0.80 | 0.76 | 0.74 | 1.08 | 1.08 | −0.54 | 0.31 |
| IDSR-2 | 0.83 | 0.78 | 0.70 | 1.15 | 1.13 | −0.45 | 0.21 |
| IDSR-3 | 0.86 | 0.79 | 0.86 | 0.92 | 0.89 | −0.50 | 0.33 |
| IDSR-4 | 0.83 | 0.78 | 0.81 | 1.07 | 1.03 | −0.48 | 0.44 |
| DISC-1 | 0.69 | 0.83 | 0.73 | 0.99 | 1.02 | 1.18 | 0.17 |
| DISC-2 | 0.65 | 0.81 | 0.79 | 1.05 | 1.10 | 1.06 | −0.06 |
| DISC-3 | 0.80 | 0.83 | 0.80 | 0.81 | 0.80 | 0.82 | 0.35 |
Note. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; MnSq: mean square error; DIF: differential item functioning.
aBased on the second-order confirmatory factor analysis. bUsing intraclass correlation coefficient. cDIF contrast >0.5 indicates substantial DIF. dDIF contrast across gender = Difficulty for females − Difficulty for males.
All factor loadings were significant at p = .001.
Psychometric properties of the IDS-15 at scale level
No substantial ceiling effect (0%–17.9%) or floor effect (0%–3.9%) was found in the IDS-15 total score and the subscale scores. Reliability conducted using classical test theory (Cronbach’s α = .76–.90 and composite reliability = 0.76–0.90) and Rasch analyses (person separation reliability = 0.71–0.85 and item separation reliability = 0.71–0.98) was satisfactory. CFA showed that the fit statistics of the IDS-15 were excellent in CFI (0.96) and TLI (0.95) and acceptable in RMSEA [90% confidence interval (CI) = 0.054–0.064]. The CFA results support the structure of the IDS-15 (Figure 1). Average variance extracted values were also adequate (0.47–0.56) and standard errors of measurement were relatively low (Table 3).
Second-order confirmatory factor analysis with standardized factor loadings. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control; IDS-15: Internet Disorder Scale
Citation: Journal of Behavioral Addictions 7, 3; 10.1556/2006.7.2018.88
Psychometric properties of the Internet Disorder Scale (IDS-15) at the scale level
| Psychometric testing | EDEC | WS | IDSR | DISC | IDS-15 |
|---|---|---|---|---|---|
| Composite reliability | 0.83 | 0.78 | 0.90 | 0.76 | 0.86 |
| Average variance extracted | 0.55 | 0.47 | 0.69 | 0.51 | 0.56 |
| Ceiling effects (%) | 17.90 | 14.40 | 0 | 0 | 0 |
| Floor effects (%) | 0.20 | 1.40 | 2.40 | 3.90 | 0 |
| Internal consistency (Cronbach’s α) | .83 | .90 | .78 | .76 | .90 |
| Person separation reliability | 0.72 | 0.85 | 0.73 | 0.71 | – |
| Item separation reliability | 0.98 | 0.71 | 0.97 | 0.97 | – |
| Standard error of measurement | 0.32 | 0.33 | 0.36 | 0.41 | 3.10 |
Note. EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control.
Concurrent validity of the IDS-15 was supported by the significant associations between IDS-15 and DASS depression subscale (β = 0.114, p < .05), DASS anxiety subscale (β = 0.148, p < .01), DASS stress subscale (β = 0.120, p < .01), IGDS9-SF (β = 0.371, p < .001), and BSMAS (β = 2.308, p < .001). Moreover, the associations between IDS-15 and two Internet-related questionnaires (IGDS9-SF and BSMAS; ΔR2 = .12 and .36, respectively) were stronger than those between IDS-15 and DASS subscales (Table 4). Multigroup CFA demonstrated that male and female adolescents interpreted the IDS-15 items similarly (ΔCFI = −0.002 and −0.007, ΔSRMR = 0.001 and 0.004, ΔRMSEA = 0.003 and 0.002; Table 5).
Concurrent validity of the Internet Disorder Scale (IDS-15) using regression models with adjustment for age and gender
| Criterion | β (p value) | ΔR2 | Overall R2 (adjusted R2) |
|---|---|---|---|
| Depressiona | 0.114 (<.05) | .03 | .167 (.165) |
| Anxietya | 0.148 (<.01) | .05 | .274 (.241) |
| Stressa | 0.120 (<.01) | .03 | .324 (.301) |
| IGDS9-SF | 0.317 (<.001) | .12 | .474 (.470) |
| BSMAS | 2.308 (<.001) | .36 | .763 (.461) |
Note. ΔR2: overall R2 − the R2 derived from age and gender; that is, indicating the explained variance of the criterion; IGDS9-SF: Internet Gaming Disorder Scale – Short Form; BSMAS: Bergen Social Media Addiction Scale.
Depression, anxiety, and stress were measured using Depression Anxiety Stress Scale.
Measurement invariance across gender on Internet Disorder Scale (IDS-15) through confirmatory factor analysis
| Models and comparisons | Fit statistics | |||||||
|---|---|---|---|---|---|---|---|---|
| χ2 (df) | Δχ2 (Δdf) | CFI | ΔCFI | SRMR | ΔSRMR | RMSEA | ΔRMSEA | |
| M1: configural | 769.46 (168)* | 0.928 | 0.051 | 0.053 | ||||
| M2: plus all loadings constrained | 790.55 (183)* | 0.921 | 0.052 | 0.056 | ||||
| M3: plus all intercepts constrained | 810.51 (198)* | 0.920 | 0.056 | 0.058 | ||||
| M2−M1 | 21.09 (15) | −0.007 | 0.001 | 0.003 | ||||
| M3−M2 | 19.96 (15) | −0.002 | 0.004 | 0.002 | ||||
Note. M1: Model 1, a configural model; M2: Model 2, a model based on M1 with all factor loadings constrained being equal across groups; M3: Mode 3, a model based on M2 with all item intercepts constrained being equal across groups; CFI: comparative fit index; SRMR: standardized root mean square residual; RMSEA: root mean square error of approximation.
p < .05.
LPA and comparisons among different subgroups
Table 6 presents the fit statistics for the two- to four-profile solutions. Consistent improvement was observed up to the three-profile solution and the four-profile solution decreased the classification quality. Although AIC, BIC, and SSABIC were lowest for the four-profile solution, the LMR value was not significant for the four-profile solution. Therefore, the three-profile solution was the model that best-fitted the data. More specifically, the LMR p value associated with the four-profile model was significant indicating that the three-profile model provides the best fit to the data. Given that the lines in Figure 2 did not intersect, the profile was named based on quantitative differences. In the three-class model, the average latent class probabilities for the most likely class were 0.92, 0.86, and 0.90 for the three latent classes, indicating good model fit and very strong probabilities of class assignment. The first profile (i.e., low Internet addiction risk) included 13.8% adolescents (n = 176), the second profile (i.e., medium Internet addiction risk) had 44.7% adolescents (n = 569), and the final profile (i.e., high Internet addiction risk) contained 41.4% adolescents (n = 527).
Latent profile analysis to identify subgroups of participants
| AIC | BIC | SSABIC | Entropy | LMR test (p value) | |
|---|---|---|---|---|---|
| Two classes | 11,332.729 | 11,399.658 | 11,358.363 | 0.818 | 1,192.523 (<.0001) |
| Three classes | 11,034.981 | 11,127.651 | 11,070.474 | 0.748 | 299.372 (<.0001) |
| Four classes | 10,926.217 | 11,044.629 | 10,971.570 | 0.713 | 115.531 (.115) |
Note. The bold values indicate the values of the group we used for the final decided group number. AIC: Akaike information criterion; BIC: Bayesian information criterion; SSABIC: sample-size adjusted BIC; LMR test: Lo–Mendell–Rubin Likelihood Ratio Test.
Latent profile analysis with three subgroups (low, medium, and high Internet addiction). EDEC: escapism and dysfunctional emotional coping; WS: withdrawal symptoms; IDSR: impairments and dysfunctional self-regulation; DISC: dysfunctional Internet-related self-control
Citation: Journal of Behavioral Addictions 7, 3; 10.1556/2006.7.2018.88
Table 7 shows the results of the multinomial logistic regression for examining associations between sociodemographic variables (independent variables) and Internet addiction profile (dependent variable). The analysis treated the high-risk addiction latent subgroup as the reference group. Factors associated with high-risk addiction included being female and older, higher father’s education, higher anxiety and stress, higher BSMAS total score, IGD weekly Internet usage, and total score of the IDS-15. Regarding the IDS-15 comparison among three subgroups, the first profile yielded the lowest Internet addiction mean score (IDS-15 = 32.83, 95% CI [32.19–33.47]), followed by the second (IDS-15 = 46.27, 95% CI [45.89–46.66]) and the third profiles (IDS-15 = 58.17, 95% CI [57.80–58.53]). In terms of subscales in the IDS-15, the adolescents in the three profiles reported the highest scores on the EDEC compared to three dimensions of the IDS-15 (Figure 2).
Predictors of membership in latent profile of Internet addiction risks
| Low addiction risk (n = 176) | Medium addiction risk (n = 569) | |||||
|---|---|---|---|---|---|---|
| b | SE | OR [95% CI] | b | SE | OR | |
| Age (years) | −1.61 | 0.71 | 0.20 [0.05–0.82] | 0.10 | 0.06 | 1.12 [0.93–1.54] |
| Gender (female) | 1.64 | 0.31 | 5.17 [3.17–11.10] | 0.38 | 0.16 | 1.47 [0.84–3.05] |
| Mother’s education year | −0.03 | 0.04 | 0.97 [0.89–1.06] | −0.02 | 0.02 | 0.98 [0.94–1.03] |
| Father’s education year | −0.11 | 0.04 | 0.89 [0.82–0.97] | −0.01 | 0.02 | 1.01 [0.97–1.06] |
| DASS depression score | −0.03 | 0.02 | 0.97 [0.93–1.01] | −0.06 | 0.02 | 0.94 [0.91–0.98] |
| DASS anxiety score | −0.13 | 0.02 | 0.88 [0.84–0.92] | −0.07 | 0.02 | 0.93 [0.90–0.96] |
| DASS stress score | −0.12 | 0.02 | 0.89 [0.85–0.92] | −0.05 | 0.01 | 0.96 [0.93–0.98] |
| BSMAS total score | −1.20 | 0.07 | 0.30 [0.26–0.35] | −0.46 | 0.04 | 0.63 [0.59–0.68] |
| Weekly Internet usage | −0.19 | 0.02 | 0.83 [0.80–0.86] | −0.08 | 0.01 | 0.92 [0.90–0.95] |
| IGDS9-SF total score | −0.17 | 0.04 | 0.85 [0.79–0.91] | −0.05 | 0.02 | 0.95 [0.91–0.99] |
| IDS-15 total score | −1.89 | 0.13 | 0.15 [0.12–0.20] | −0.88 | 0.07 | 0.42 [0.37–0.47] |
Note. The latent profile of high addiction risk was set as the reference group. All odds ratios were adjusted for all variables included in the table. SE: standard error; OR: odds ratio; CI: confidence interval; BSMAS: Bergen Social Media Addiction Scale; IGDS9-SF: Internet Gaming Disorder Scale – Short Form; IDS-15: Internet Disorder Scale; DASS: Depression Anxiety Stress Scale.
Discussion
This study comprehensively and thoroughly assessed the psychometric properties of the Persian IDS-15, an instrument developed using the modified criteria of IGD in the DSM-5 (APA, 2013). Overall, the results obtained for the Persian IDS-15 at the factorial structure and measurement invariance levels were adequate across both item and scale levels. The psychometric properties of the Persian IDS-15 were ascertained using different psychometric frameworks (i.e., classical test theory and Rasch analysis), and the results evidenced the robustness of the IDS-15 in assessing Internet addiction among Iranian adolescents. Male and female participants interpreted the IDS-15 items similarly, and the results supported the use of IDS-15 across gender. Moreover, the LPA showed that the study’s participants could be classified into three latent subgroups based on participants’ risk for developing Internet addiction.
The CFA results were comparable to the previous two studies assessing the psychometric properties of IDS-15 (English version by Pontes & Griffiths, 2017 and Italian version by Monacis et al., 2018). However, the results of this study were more comparable to the findings from Pontes and Griffiths (2017) than those from Monacis et al. (2018). More specifically, Pontes and Griffiths (2017) reported satisfactory CFA without correlating any residual of the IDS-15 item and a three-subgroup finding in their LPA. Conversely, Monacis et al. (2018) presented their CFA with an unexpected covariance (the residual of Item 9 to that of Item 10) and a four-subgroup finding in the LPA. A possible explanation proposed by Monacis et al. (2018) was based on the composition of the recruited sample. In addition to the general populations regardless of their age, Monacis et al. recruited a subgroup from gaming halls. Given that the participants recruited from gaming halls are very likely to have different levels of Internet addiction from those who were recruited from other places, Monacis et al. additionally classified a subgroup as those with critical addiction risk. That is, the participants from gaming halls are likely to include actual addicts and may have also been affected by other behavioral addictions. As a result, the structure of IDS-15 may also rather be influenced by the specific subgroup.
Similar to the studies by Pontes and Griffiths (2017) and Monacis et al. (2018), this study found that the IDS-15 had promising reliability as demonstrated by internal consistency and composite reliability. The study also extended the satisfactory reliability results to test–retest reliability and separation reliability reported by Rasch, providing novel findings. Therefore, a variety of interested stakeholders (e.g., researchers, clinicians, and other healthcare providers) can be assured that the reproducibility of the IDS-15 is stable and that items are coherent to each other. In addition, the findings share similarity with those reported by Monacis et al. (2018) in terms of measurement invariance and concurrent validity. Findings from this study supported the notion that the IDS-15 is measurement invariant across gender, and the study extended the results to the item level; that is, no DIF items were displayed in the Rasch findings. The absence of DIF items and the presence of measurement invariance for the structure of the IDS-15 facilitate meaningful comparisons between groups (Gregorich, 2006; Lin, Ou, et al., 2018). From a statistical and clinical perspective, establishing invariance indicated that respondents in the comparison groups (e.g., male and female groups in this study) responded to the IDS-15 items in the same way (i.e., they share similar processes in the underlying constructs), which suggests that generic treatment and prevention programs are likely to be effective for the comparison groups. In contrast, when DIF occurs (i.e., invariance is not established), the comparison groups may need specifically tailored assessments, treatment, and prevention programs.
In terms of the characteristics of the three subgroups with different risks of Internet addiction, key differences were found in gender, emotional distress, weekly Internet usage, and other addictive behaviors (Internet gaming and social media use). The group at high risk of Internet addiction was generally male, prone to high emotional distress, engaging more Internet usage, and having other Internet-related addiction behaviors. The findings parallel to the results of the previous studies in which being male (e.g., Sasmaz et al., 2014), high weekly Internet usage (e.g., Bouna-Pyrrou, Mühle, Kornhuber, & Lenz, 2015), impaired emotional distress (e.g., Pontes, Caplan, & Griffiths, 2016; Weinstein et al., 2015), and addiction to other behaviors (e.g., Pontes, 2017; Rücker et al., 2015) were found to be associated with Internet addiction.
However, unlike the finding from Pontes and Griffiths (2017) that younger age was related to higher risk of Internet addiction, this study found that age was not significantly different between the three subgroups across different levels of addiction risk. A possible reason is because of the different age ranges. Because in the study by Pontes and Griffiths (2017), subgroups had a wider age range compared to that of this study (15.53 ± 1.2 vs. 33 ± 12.3), it is possible that the age effect was not obvious in the findings here. Nevertheless, this study showed a three-profile solution in the LPA, which corresponds to the LPA solution found by Pontes and Griffiths (2017). Additionally, the results demonstrated that more than 40% of the participants (527 out of 1,272) were classified as a high addiction risk by the LPA. Given that our study is focused on adolescent population, this finding may reflect the results from Pontes and Griffiths (2017) regarding the association between young age and higher risk of addiction. Indeed, other studies have found that adolescents frequently use Internet excessively (e.g., Strong, Lee, Chao, Lin, & Tsai, 2018; Tsai et al., 2018).
There are several limitations in the study. First, the study recruited only adolescents; therefore, the generalizability of our results cannot be applied to those in different life stages (e.g., adults and the elderly). Second, the data were self-report and subject to well-known biases such as memory recall biases and social desirability biases. Third, the adolescent sample data were only collected in one Iranian city and may not necessarily by representative of all Iranian adolescents. Fourth, the participants were recruited from schools; thus, the data were nested and therefore could have impacted on the findings. Finally, because the participants were Iranian, the study is unable to directly compare the IDS-15 between Western and Eastern countries (e.g., whether Western people are more Internet-addicted) or to examine the measurement invariance across different cultures. Researchers and other interested stakeholders (e.g., healthcare professionals) are reminded that using the IDS-15 to compare Western people to Eastern people in assessing Internet addiction requires caution. Based on the aforementioned potential limitations, the present authors would encourage future research to examine the psychometric properties of the IDS-15 across different age groups and different ethnicities that are underresearched such as the one investigated in this study.
Conclusions
This study showed that the IDS-15 had a clear structure of four distinct and interrelated domains, and all items were perfectly embedded in their belonging domain. The Persian IDS-15 also had robust psychometric properties as evidenced by both classical test theory and Rasch analysis. Moreover, no substantial DIF was displayed across gender in this study’s sample. Similarly, measurement invariance was supported for IDS-15 across gender. LPA clustered the participants into three subgroups, and the high-risk Internet addiction group had more males, more weekly Internet usage, higher DASS scores in anxiety and depression, and higher BSMAS and IGDS9-SF scores compared with medium and low Internet addiction groups.
Authors’ contribution
AHP designed and organized the study and collected the data. C-YL and AHP wrote the first draft and analyzed and initially interpreted the data. MG supervised the data analysis. MDG supervised the entire study. HMP, VI, AB, and MDG critically reviewed the manuscript and provided constructive comments.
Conflict of interest
The authors declare no conflict of interest.
References
American Psychiatric Association [APA]. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.
American Psychiatric Association [APA]. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Association.
Andreassen, C. S. , Billieux, J. , Griffiths, M. D. , Kuss, D. J. , Demetrovics, Z. , Mazzoni, E. , & Pallesen, S. (2016). The relationship between addictive use of social media and video games and symptoms of psychiatric disorder: A large-scale cross-sectional study. Psychology of Addictive Behaviors, 30(2), 252–262. doi:10.1037/adb0000160
Andreassen, C. S. , Torsheim, T. , Brunborg, G. S. , & Pallesen, S. (2012). Development of a Facebook Addiction Scale. Psychological Reports, 110(2), 501–517. doi:10.2466/02.09.18.PR0.110.2.501-517
Asghari, A. , Saed, F. , & Dibajnia, P. (2008). Psychometric properties of the Depression Anxiety Stress Scales-21 (DASS-21) in a non-clinical Iranian sample. International Journal of Psychology, 2(2), 82–102.
Bagheri, Z. , Jafari, P. , Tashakor, E. , Kouhpayeh, A. , & Riazi, H. (2014). Assessing whether measurement invariance of the KIDSCREEN-27 across child-parent dyad depends on the child gender: A multiple group confirmatory factor analysis. Global Journal of Health Science, 6(5), 142–153. doi:10.5539/gjhs.v6n5p142
Bagozzi, R. P. , & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. doi:10.1007/BF02723327
Bányai, F. , Zsila, Á. , Király, O. , Maraz, A. , Elekes, Z. , Griffiths, M. D. , Andreassen, C. S. , & Demetrovics, Z. (2017). Problematic social media use: Results from a large-scale nationally representative adolescent sample. PLoS One, 12(1), e0169839. doi:10.1371/journal.pone.0169839
Beaton, D. E. , Bombardier, C. , Guillemin, F. , & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. doi:10.1097/00007632-200012150-00014
Bouna-Pyrrou, P. , Mühle, C. , Kornhuber, J. , & Lenz, B. (2015). Internet gaming disorder, social network disorder and laterality: Handedness relates to pathological use of social networks. Journal of Neural Transmission, 122(8), 1187–1196. doi:10.1007/s00702-014-1361-5
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: The Guilford Press.
Browne, M. W. , & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.
Chang, K.-C. , Wang, J.-D. , Tang, H.-P. , Cheng, C.-M. , & Lin, C.-Y. (2014). Psychometric evaluation, using Rasch analysis, of the WHOQOL-BREF in heroin-dependent people undergoing methadone maintenance treatment: Further item validation. Health and Quality of Life Outcomes, 12(1), 148. doi:10.1186/s12955-014-0148-6
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. doi:10.1080/10705510701301834
Cheng, C.-P. , Luh, W.-M. , Yang, A.-L. , Su, C.-T. , & Lin, C.-Y. (2016). Agreement of children and parents scores on Chinese version of Pediatric Quality of Life Inventory version 4.0: Further psychometric development. Applied Research in Quality of Life, 11(3), 891–906. doi:10.1007/s11482-015-9405-z
Cho, H. , Kwon, M. , Choi, J.-H. , Lee, S.-K. , Choi, J. S. , Choi, S.-W. , & Kim, D.-J. (2014). Development of the Internet Addiction Scale based on the Internet gaming disorder criteria suggested in DSM-5. Addictive Behaviors, 39(9), 1361–1366. doi:10.1016/j.addbeh.2014.01.020
Davidson, L. L. , Grigorenko, E. L. , Boivin, M. J. , Rapa, E. , & Stein, A. (2015). A focus on adolescence to reduce neurological, mental health and substance-use disability. Nature, 527(7578), S161–S166. doi:10.1038/nature16030
Fornell, C. , & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. doi:10.2307/3151312
Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44, S78–S94. doi:10.1097/01.mlr.0000245454.12228.8f
Griffiths, M. D. (2005). A ‘components’ model of addiction within a biopsychosocial framework. Journal of Substance Use, 10(4), 191–197. doi:10.1080/14659890500114359
Griffiths, M. D. , Van Rooij, A. J. , Kardefelt-Winther, D. , Starcevic, V. , Király, O. , Pallesen, S. , Müller, K. , Dreier, M. , Carras, M. , Prause, N. , King, D. L. , Aboujaoude, E. , Kuss, D. J. , Pontes, H. M. , Lopez Fernandez, O. , Nagygyorgy, K. , Achab, S. , Billieux, J. , Quandt, T. , Carbonell, X. , Ferguson, C. J. , Hoff, R. A. , Derevensky, J. , Haagsma, M. C. , Delfabbro, P. , Coulson, M. , Hussain, Z. , & Demetrovics, Z. (2016). Working towards an international consensus on criteria for assessing Internet gaming disorder: A critical commentary on Petry et al. (2014). Addiction, 111(1), 167–175. doi:10.1111/add.13057
Hu, L.-T , & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. doi:10.1037/1082-989X.3.4.424
Jafari, P. , Bagheri, Z. , & Safe, M. (2012). Item and response-category functioning of the Persian version of the KIDSCREEN-27: Rasch partial credit model. Health and Quality of Life Outcomes, 10(1), 127. doi:10.1186/1477-7525-10-127
Jette, D. U. , Warren, R. L. , & Wirtalla, C. (2005). Functional independence domains in patients receiving rehabilitation in skilled nursing facilities: Evaluation of psychometric properties. Archives of Physical Medicine and Rehabilitation, 86(6), 1089–1094. doi:10.1016/j.apmr.2004.11.018
Khan, A. , Chien, C.-W. , & Brauer, S. G. (2013). Rasch-based scoring offered more precision in differentiating patient groups in measuring upper limb function. Journal of Clinical Epidemiology, 66(6), 681–687. doi:10.1016/j.jclinepi.2012.12.014
Khoshnevisan, A. , Yekaninejad, M. S. , Ardakani, S. K. , Pakpour, A. H. , Mardani, A. , & Aaronson, N. K. (2012). Translation and validation of the EORTC brain cancer module (EORTC QLQ-BN20) for use in Iran. Health and Quality of Life Outcomes, 10(1), 54. doi:10.1186/1477-7525-10-54
Király, O. , Nagygyörgy, K. , Koronczai, B. , Griffiths, M. D. , & Demetrovics, Z. (2014). Assessment of problematic Internet use and online video gaming. In V. Starcevic & E. Aboujaoude (Eds.), Mental health in the digital age: Grave dangers, great promise (pp. 46–68). Oxford, UK: Oxford University Press.
Kuss, D. J. , Griffiths, M. D. , Karila, L. , & Billieux, J. (2014). Internet addiction: A systematic review of epidemiological research for the last decade. Current Pharmaceutical Design, 20(25), 4026–4052. doi:10.2174/13816128113199990617
Lin, C.-Y. , Broström, A. , Nilsen, P. , Griffiths, M. D. , & Pakpour, A. H. (2017). Psychometric validation of the Persian Bergen Social Media Addiction Scale using classic test theory and Rasch models. Journal of Behavioral Addiction, 6(4), 620–629. doi:10.1556/2006.6.2017.071
Lin, C.-Y. , Luh, W.-M. , Yang, A.-L. , Su, C.-T. , Wang, J.-D. , & Ma, H.-I. (2012). Psychometric properties and gender invariance of the Chinese version of the self-report Pediatric Quality of Life Inventory version 4.0: Short Form is acceptable. Quality of Life Research, 21(1), 177–182. doi:10.1007/s11136-011-9928-1
Lin, C.-Y. , Ou, H.-T. , Nikoobakht, M. , Broström, A. , Årestedt, K. , & Pakpour, A. H. (2018). Validation of Medication Adherence Report Scale (MARS-5) in older stroke patients in Iran. Journal of Cardiovascular Nursing. Advance online publication. doi:10.1097/JCN.0000000000000488
Lin, C.-Y. , Pakpour, A. H. , Broström, A. , Fridlund, B. , Årestedt, K. , Strömberg, A. , Jaarsma, T. , & Mårtensson, J. (2018). Psychometric properties of the 9-item European Heart Failure Self-Care Behavior Scale using confirmatory factor analysis and Rasch analysis among Iranian patients. Journal of Cardiovascular Nursing, 33(3), 281–288. doi:10.1097/JCN.0000000000000444
Lin, C.-Y. , & Tsai, M.-C. (2016). Effects of family context on adolescents’ psychological problems: Moderated by pubertal timing, and mediated by self-esteem and interpersonal relationships. Applied Research in Quality of Life, 11(3), 907–923. doi:10.1007/s11482-015-9410-2
Lovibond, P. F. , & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335–343. doi:10.1016/0005-7967(94)00075-U
Macur, M. , & Pontes, H. M. (2018). Individual differences and the development of Internet addiction: A nationally representative study. In B. Bozoglan (Ed.), Psychological, social, and cultural aspects of Internet addiction (pp. 221–235). Hershey, PA: IGI Global.
Monacis, L. , Sinatra, M. , Griffiths, M. D. , & de Palo, V. (2018). Assessment of the Italian version of the Internet Disorder Scale (IDS-15). International Journal of Mental Health and Addiction, 16(3), 680–691. doi:10.1007/s11469-017-9823-2
Petry, N. M. , Rehbein, F. , Gentile, D. A. , Lemmens, J. S. , Rumpf, H. J. , Mößle, T. , Bischof, G. , Tao, R. , Fung, D. S. , Borges, G. , Auriacombe, M. , González Ibáñez, A. , Tam, P. , & O’Brien, C. P. (2014). An international consensus for assessing Internet gaming disorder using the new DSM-5 approach. Addiction, 109(9), 1399–1406. doi:10.1111/add.12457
Petry, N. M. , Zajac, K. , & Ginley, M. K. (2018). Behavioral addictions as mental disorders: To be or not to be? Annual Review of Clinical Psychology, 14(1), 399–423. doi:10.1146/annurev-clinpsy-032816-045120
Pew Research Center. (2016). Smartphone ownership and Internet usage continues to climb in emerging economies. Retrieved September 8, 2018, from http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/
Pontes, H. M. (2017). Investigating the differential effects of social networking site addiction and Internet gaming disorder on psychological health. Journal of Behavioral Addictions, 6(4), 601–610. doi:10.1556/2006.6.2017.075
Pontes, H. M. , Caplan, S. E. , & Griffiths, M. D. (2016). Psychometric validation of the Generalized Problematic Internet Use Scale 2 in a Portuguese sample. Computers in Human Behavior, 63, 823–833. doi:10.1016/j.chb.2016.06.015
Pontes, H. M. , & Griffiths, M. D. (2015). Measuring DSM-5 Internet gaming disorder: Development and validation of a Short Psychometric Scale. Computers in Human Behavior, 45, 137–143. doi:10.1016/j.chb.2014.12.006
Pontes, H. M. , & Griffiths, M. D. (2017). The development and psychometric evaluation of the Internet Disorder Scale (IDS-15). Addictive Behaviors, 64, 261–268. doi:10.1016/j.addbeh.2015.09.003
Pontes, H. M. , Kuss, D. J. , & Griffiths, M. D. (2015). Clinical psychology of Internet addiction: A review of its conceptualization, prevalence, neuronal processes, and implications for treatment. Neuroscience and Neuroeconomics, 4, 11–23. doi:10.2147/NAN.S60982
Pontes, H. M. , Szabo, A. , & Griffiths, M. D. (2015). The impact of Internet-based specific activities on the perceptions of Internet addiction, quality of life, and excessive usage: A cross-sectional study. Addictive Behaviors Reports, 1, 19–25. doi:10.1016/j.abrep.2015.03.002
Rücker, J. , Akre, C. , Berchtold, A. , & Suris, J.-C. (2015). Problematic Internet use is associated with substance use in young adolescents. Acta Paediatrica, 104(5), 504–507. doi:10.1111/apa.12971
Rumpf, H. , Bischof, G. , Bischof, A. , Besser, B. , Meyer, C. , & John, U. (2015). Applying DSM-5 criteria for Internet gaming disorder for the broader concept of Internet addiction. Journal of Behavioral Addictions, 4(Suppl. 1), 36. doi:10.1556/JBA.4.2015.Suppl.1
Sariyska, R. , Reuter, M. , Lachmann, B. , & Montag, C. (2015). Attention deficit/hyperactivity disorder is a better predictor for problematic Internet use than depression: Evidence from Germany. Journal of Addiction Research & Therapy, 6, 209. doi:10.4172/2155-6105.1000209
Sasmaz, T. , Oner, S. , Kurt, A. O. , Yapici, G. , Yazici, A. E. , Bugdayci, R. , & Sis, M. (2014). Prevalence and risk factors of Internet addiction in high school students. European Journal of Public Health, 24(1), 15–20. doi:10.1093/eurpub/ckt051
Shih, C.-L. , & Wang, W.-C. (2009). Differential item functioning detection using the multiple indicators, multiple causes method with a pure short anchor. Applied Psychological Measurement, 33(3), 184–199. doi:10.1177/0146621608321758
Skoog, T. , Stattin, H. , & Kerr, M. (2009). The role of pubertal timing in what adolescent boys do online. Journal of Research on Adolescence, 19(1), 1–7. doi:10.1111/j.1532-7795.2009.00578.x
Strong, C. , Lee, C.-T. , Chao, L.-H. , Lin, C.-Y. , & Tsai, M.-C. (2018). Adolescent Internet use, social integration, and depressive symptoms: Analysis from a longitudinal cohort survey. Journal of Developmental & Behavioral Pediatrics, 39(4), 318–324. doi:10.1097/DBP.0000000000000553
Strong, C. , Lin, Y.-C. , Tsai, M.-C. , & Lin, C.-Y. (2018). Factor structure of Sizing Me Up, a self-reported weight-related quality of life instrument, in community children across weight status. Childhood Obesity, 13(2), 111–119. doi:10.1089/chi.2016.0259
Strong, C. , Tsai, M.-C. , Lin, C.-Y. , & Cheng, C.-P. (2016). Childhood adversity, timing of puberty and adolescent depressive symptoms: A longitudinal study in Taiwan. Child Psychiatry and Human Development, 47(3), 347–357. doi:10.1007/s10578-015-0570-y
Tsai, M.-C. , Strong, C. , Chen, W.-T. , Lee, C.-T. , & Lin, C.-Y. (2018). Longitudinal impacts of pubertal timing and weight status on adolescent Internet use: Analysis from a cohort study of Taiwanese youths. PLoS One, 13(5), e0197860. doi:10.1371/journal.pone.0197860
Wang, Y.-S. , Wang, H.-Y. , & Shee, D. Y. (2007). Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior, 23(4), 1792–1808. doi:10.1016/j.chb.2005.10.006
Wartberg, L. , Kriston, L. , Kammerl, R. , Petersen, K. U. , & Thomasius, R. (2015). Prevalence of pathological Internet use in a representative German sample of adolescents: Results of a latent profile analysis. Psychopathology, 48(1), 25–30. doi:10.1159/000365095
Weinstein, A. , Dorani, D. , Elhadif, R. , Bukovza, Y. , & Yarmulnik, A. (2015). Internet addiction is associated with social anxiety in young adults. Annals of Clinical Psychiatry, 27(1), 2–7. doi:10.1093/med/9780199380183.003.0001
Wu, T.-Y. , Lin, C.-Y. , Årestedt, K. , Griffiths, M. D. , Broström, A. , & Pakpour, A. H. (2017). Psychometric validation of the Persian nine-item Internet Gaming Disorder Scale – Short form: Does gender and hours spent online gaming affect the interpretations of item descriptions? Journal of Behavioral Addictions, 6(2), 256–263. doi:10.1556/2006.6.2017.025
Yang, S.-Y. , Chen, M.-D. , Huang, Y.-C. , Lin, C.-Y. , & Chang, J.-H. (2017). Association between smartphone use and musculoskeletal discomfort in adolescent students. Journal of Community Health, 42(3), 423–430. doi:10.1007/s10900-016-0271-x
Yang, S.-Y. , Lin, C.-Y. , Huang, Y.-C. , & Chang, J.-H. (2018). Gender differences in the association of smartphone use with the vitality and mental health of adolescent students. Journal of American College Health. Advance online publication. doi:10.1080/07448481.2018.1454930
