Abstract
Background and aims
Although the Ten-Item Internet Gaming Disorder Test (IGDT-10) has been translated into Japanese and widely used, the Japanese version has not previously been validated. We used the clinical diagnosis of IGD as a gold standard for validating the test.
Methods
The Japanese version was validated using 244 gamers drawn from the general young population in Japan. Expert interviews using the Japanese version of the Structured Clinical Interview for Internet Gaming Disorder evaluated diagnoses of Internet gaming disorder (IGD). This resulted in a diagnosis of IGD for eight individuals, categorized as the gold standard group. The screening performance of the two Japanese versions with different scoring conditions was examined: the scoring method proposed by the original study (original version) and a less stringent scoring method where responses of either “often” or “sometimes” were regarded as affirmative (modified version).
Results
The results of the sensitivity and specificity analyses, the Cronbach's alpha and the receiver operating characteristics analysis revealed a higher screening performance for the modified versus the original version. The optimum cutoff for the modified version was 5 or more – the sensitivity, specificity, and Youden's index were 87.5, 85.2, and 72.7%, respectively. The rate of probable IGD using the original and modified versions were 1.8% and 11.3%, respectively.
Discussion and conclusion
A less stringent scoring method for the Japanese version of IGDT-10 showed a higher screening performance than the original scoring method. Future studies comprising different ethnic groups and gaming cultures should further examine the suggested scoring method.
Introduction
In recent years digital gaming has become increasingly popular worldwide and a parallel phenomenon has been the rise on a global scale of health, social, and familial issues associated with excessive gaming (Darvesh et al., 2020; Sugaya, Shirasaka, Takahashi, & Kanda, 2019). In response to this increasingly problematic situation, the American Psychiatric Association (APA) developed and published the diagnostic criteria of internet gaming disorder (IGD) in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) in 2013 (APA, 2013). These comprised nine criteria: 1) preoccupation, 2) withdrawal symptoms, 3) tolerance, 4) unsuccessful control, 5) giving up other activities, 6) continuation despite problems, 7) deception, 8) escape, and 9) negative consequences. At least five criteria were required to be met in order for a diagnosis of IGD to be made. Subsequently, the World Health Organization (WHO) included gaming disorder (GD) as a disorder due to addictive behaviors in the eleventh revision of the International Classification of Diseases (ICD-11) in May 2019 (WHO, 2019). According to the definition of GD in ICD-11, three clinical manifestation criteria and one functional impairment criterion must be met in order to make a diagnosis of GD (WHO, 2019).
Several assessment tools and screening tests based on the diagnostic criteria of IGD were developed following the publication of DSM-5 (Jo et al., 2018; Király et al., 2017; Lemmens, Valkenburg, & Gentile, 2015; Pearcy, Roberts, & McEvoy, 2016; Pontes & Griffiths, 2015; Pontes, Király, Demetrovics, & Griffiths, 2014; van Rooij, Schoenmakers, & van de Mheen, 2017). Some of these have been extensively validated and translated into other languages (Chen, Su, Dang, & Wu, 2021; Király et al., 2017; Pontes & Griffiths, 2015). One such test, the Ten-Item Internet Gaming Disorder Test (IGDT-10) has been translated into multiple languages, including Japanese (King et al., 2020). The IGDT-10 assesses nine criteria of DSM-5 IGD and was developed by selecting one item from each of the IGD criteria with the final criterion being divided into two sub-criteria. The items are answered using a three-point Likert scale with three available responses in relation to the frequency of items occurring in the previous 12 months: “never”, “sometimes”, and “often”. With regard to the scoring condition of the scale, the original study indicated that items were to be recorded in a dichotomous format to measure the DSM-5 criteria according to the following: the answers “never” and “sometimes” are evaluated as the criterion not being met, while “often” is evaluated as the criterion being met (Király et al., 2017). In the case of questions 9 and 10, the answer “often” on either item or both items is regarded as the final item being met. The cutoff for screening IGD was set at meeting five or more criteria (Király et al., 2017).
As previously mentioned, the IGDT-10 was translated into Japanese and has been widely used in Japan. This has included for the purposes of estimating the prevalence of probable IGD and to screen individuals with probable IGD in various settings (Kinjo & Osaki, 2019; Nakayama, Matsuzaki, Mihara, Kitayuguchi, & Higuchi, 2020). However, validation of this scale had yet to be conducted in Japan. In this context the main purpose of this study was to validate the Japanese version of the scale and to confirm whether the scoring condition proposed by the original study is applicable to Japanese samples for screening purposes. Additionally, we sought to estimate the rate of probable IGD using the validated version of IGDT-10 among subjects representing the general population, aged between 10 and 29 years, in Japan.
Methods
Study participants
The subjects of this study were from among the general Japanese population residing in Japan who responded to a self-reported questionnaire survey about internet game use and daily habits from January through March 2019 (initial survey). For this, 300 national census spots were randomly extracted and 9,000 individuals aged 10–29 years randomly selected from the register of local community residents comprising these census spots. Of the survey participants, 5,096 individuals gave valid responses to the questionnaire (response rate: 56.6%). Despite the relatively low response rate, the age, gender, and geographical distributions of these respondents were similar to those of the initial survey invitees (N = 9,000). Of the respondents, 2,953 gave consent to participate in the follow-up survey and 766 of those living in the broader Tokyo metropolitan area were invited to participate in this study for the validation of the IGDT-10 Japanese version and the development of a screening test for ICD-11 GD (Higuchi et al., 2021). A total of 281 individuals who had consented to participate in the interview actually attended the survey venue (interview survey). Among these, 244 participants had played games in the previous 12 months prior to the survey and were enrolled in this study.
Content of the self-reported questionnaire and the instrument utilized to make an IGD diagnosis
A self-reported questionnaire including the Japanese version of IGDT-10, shown in appendix, was administered to the study participants. IGDT-10 was translated into Japanese and the accuracy of translation was confirmed by comparing the original English version and the English version retranslated from the Japanese version. In addition to these questions, several others were included in the questionnaire to obtain background information on: gender, age, occupation, family profile, school life, internet usage, age at the start of gaming, devices used for gaming, favorite game genre, time spent on gaming per day, and involvement in e-sport.
In examining the diagnosis of DSM-5 IGD, the Japanese version of the Structured Clinical Interview for Internet Gaming Disorder (SCI-IGD) was used (Koo, Han, Park, & Kwon, 2017). This is a short interview instrument for IGD, the validity and reliability of which has been confirmed using Korean adolescents. Like IGDT-10, the accuracy of translation was confirmed by comparing the original English version and the English version retranslated from the Japanese version. However, validation of the instrument has yet to be assessed.
Procedures
In the initial survey, professional interviewers visited each household containing a respondent identified using the aforementioned process, requesting that the questionnaire be completed and arranging for its subsequent collection.
IGD in study participants was definitively diagnosed on the basis of face-to-face interviews by psychiatrists and clinical psychologists at the Center with expertise in the diagnosis and treatment of IGD. These professionals have extensive clinical experiences in seeing both outpatients and inpatients with IGD. To minimize differences in symptom evaluation arising between interviewers, the interviewers were trained in advance using model cases, so that evaluations could be performed in a standardized manner. The interviews were carried out in July and August 2019 in Tokyo, and a total of 281 individuals participated in the survey. All experts involved in the interviews later met, confirming the symptoms of those individuals who were suspected to have IGD, to judge the presence or absence of IGD, thereby identifying eight participants with IGD. The same self-reported questionnaire that had been used in the initial survey was administered on the same day as but prior to the interviews and the results of this survey was utilized for the analysis of this study.
Statistical analysis
The sensitivity and specificity of each item of IGDT-10 was calculated using the presence or absence of each diagnostic item of DSM-5 IGD, based on the gold standard interviews. For items 9 and 10 of IGDT-10 corresponding to the 9th diagnostic item of IGD, an answer was judged to be positive if at least one of two items produced an affirmative response.
In this study, we examined two different scoring conditions in the Japanese version of IGDT-10. The first was the original condition in which each item was judged to be positive if “often” out of three choices was selected, as proposed in the original study (called the “original version”) (Király et al., 2017). Similarly, items 9 and 10 were judged as positive if one of the two responses was in the affirmative. The second was the modified condition in which each item was judged to be positive if either “often” or “sometimes” was selected (called the “modified version”). The sensitivity, specificity, positive and negative predictive values, and diagnostic accuracy of these two versions were calculated using the diagnosis of IGD based on the result of the SCI-IGD and the subsequent panel discussion as the gold standard. Screening performance was analyzed using the receiver operating characteristics (ROC) analysis of the two IGDT-10 versions with the DSM-5 IGD diagnosis as the gold standard. The optimum cutoff level was determined using the Youden's Index (Sensitivity + Specificity - 1) (Bantis, Nakas, & Reiser, 2014). The confirmatory factor analysis was conducted to evaluate the factor structure of the two versions.
The confirmatory factor analysis (CFA) was conducted to evaluate the factor structure of the two versions using the R lavaan package (version 4.2.1). Since the variables treated in this study were binary variables that were not normally distributed, a CFA analysis was conducted using the adjusted diagonally weighted least squares (WLSAMV) estimation method.
In order to estimate the rate of probable IGD, the original and modified versions of IGDT-10 were applied to 5,096 respondents of the initial survey. The data were adjusted by every one-year age group and gender for the estimated population in Japan as of October 1, 2018 (Statistics Bureau of Japan, 2018). The t-test was used for continuous data and the chi-square test and the Fischer's exact test were used for categorical data to examine statistical differences. Effect sizes were evaluated using Cohen's d for the t-test and Cramer's V for the chi-square test. Cronbach's alpha and composite reliability were used for evaluating the internal consistency of the Japanese version of the IGDT-10.
All statistical analyses of the data were conducted using the Statistical Analysis System, Version 9.4 (SAS Institute Inc., 2016), except for the estimation of the rate of probable IGD and a confirmatory factor analysis. These were carried out using SPSS Version 25.0 (IBM Corp., 2017).
Ethics
This study was approved by the Ethics Committee of Kurihama Medical and Addiction Center which is called “the Center” in the text of this article (Approval No. 340). Informed consent to participate in this study was obtained from all participants. If participants were younger than 18 years of age, informed consent was obtained from not only the participants, but also their guardians. All study procedures were performed in accordance with the Declaration of Helsinki.
Results
Background information
Background information on the interview survey participants is summarized in Table 1. The percentage of males was approximately 50%, the mean ages were about 17 years old, and more than 70% were students, both in the case of participants with IGD (IGD group) and those who were diagnosed as not having IGD (non-IGD group). Regarding the types of internet application, both information search and video viewing were the most popular in both groups. Online games were played by 87.5% for the IGD group and 52.5% for the non-IGD group within the previous 30 days prior to the interview. One participant in the IGD group who had not played online games in the previous 30 days had played online games prior to the 30 days leading up to the interview. There was little difference between the two groups with regard to game genres. Action and social games were preferred by the IGD group. As expected, the mean time spent on gaming on a weekday was significantly longer in the IGD than the non-IGD group. A similar tendency was observed for a weekend day, but it did not reach statistically significant levels due to the wide range of time spent on gaming. However, effect sizes for comparisons between the two groups in the time spent on gaming were large not only on a weekday but on a weekend day (Table 1).
Background information of study participants (N = 244), by IGD status.
| Non-IGDa) (N = 236) | IGDa) (N = 8) | Results of chi-square and Fisher's exact tests or t-test | Effect sizeb) | |
| Gender | ||||
| Male | 121 (51.3%) | 5 (62.5%) | χ2 = 0.391, df = 1, P = 0.532; Fisher, P = 0.235 | 0.040 |
| Age (years) | ||||
| Mean (SD) | 16.6 (5.3) | 17.1 (5.9) | t = 0.29, df = 241 P = 0.776 | 0.10 |
| Range | 10–30 | 10–30 | ||
| Occupation | ||||
| Full-time worker | 33 (14.1%) | 1 (12.5%) | χ2 = 2.709, df = 3, P = 0.439 | 0.106 |
| Part-time worker | 28 (12.0%) | 0 (0.0%) | ||
| Student | 165 (70.5%) | 6 (75.0%) | ||
| Unemployed | 8 (3.4%) | 1 (12.5%) | ||
| Types of internet applications usedc,d) | ||||
| Information search | 202 (85.6%) | 8 (100%) | χ2 = 1.339, df = 1, P = 0.247; Fisher, P = 0.295 | −0.074 |
| 111 (47.0%) | 2 (25.0%) | χ2 = 1.511, df = 1, P = 0.219; Fisher, P = 0.143 | 0.079 | |
| Chat, Skype, and Messenger | 28 (11.9%) | 3 (37.5%) | χ2 = 4.585, df = 1, P = 0.032; Fisher, P = 0.057 | −0.137 |
| Electronic bulletin board and Blog | 20 (8.5%) | 0 (0.0%) | χ2 = 0.739, df = 1, P = 0.390; Fisher, P = 0.499 | 0.055 |
| SNS | 172 (72.9%) | 5 (62.5%) | χ2 = 0.419, df = 1, P = 0.518; Fisher, P = 0.236 | 0.041 |
| Online games | 124 (52.5%) | 7 (87.5%) | χ2 = 7.018, df = 1, P = 0.008; Fisher, P = 0.007 | −0.170 |
| Video viewing | 203 (86.0%) | 8 (100%) | χ2 = 1.294, df = 1, P = 0.255; Fisher, P = 0.307 | −0.073 |
| Game genre playedc,d) | ||||
| RPG | 64 (26.7%) | 2 (25.0%) | χ2 = 0.011, df = 1, P = 0.915; Fisher, P = 0.315 | 0.007 |
| Shooting | 54 (22.9%) | 3 (37.5%) | χ2 = 0.924, df = 1, P = 0.337; Fisher, P = 0.190 | −0.062 |
| Strategic simulation | 28 (11.9%) | 1 (12.5%) | χ2 = 0.003, df = 1, P = 0.956; Fisher, P = 0.399 | −0.004 |
| Action | 75 (31.8%) | 2 (25.0%) | χ2 = 0.165, df = 1, P = 0.685; Fisher, P = 0.290 | 0.026 |
| Caring | 58 (24.6%) | 4 (50.0%) | χ2 = 2.639, df = 1, P = 0.104; Fisher, P = 0.089 | −0.104 |
| Puzzle | 58 (24.6%) | 1 (12.55) | χ2 = 0.616, df = 1, P = 0.424; Fisher, P = 0.279 | 0.050 |
| Social | 67 (28.4%) | 4 (50.0%) | χ2 = 1.752, df = 1, P = 0.188; Fisher, P = 0.126 | −0.085 |
| Rhythm | 66 (28.0%) | 2 (25.0%) | χ2 = 0.034, df = 1, P = 0.854; Fisher, P = 0.311 | 0.012 |
| Mean time spent on gaming on a weekday (mins) | ||||
| Mean (SD) | 77.5 (91.5) | 262.5 (207.1) | t = 2.52, df = 7.09, P = 0.039 | 1.91 |
| Range | 30–600 | 0–600 | ||
| Mean time spent on gaming on a weekend day (mins) | ||||
| Mean (SD) | 129.1 (133.2) | 307.5 (218.6) | t = 2.29, df = 7.18, P = 0.0545 | 1.31 |
| Range | 0–780 | 60–600 |
Note:
a) IGD: participants with IGD. Non-IGD: participants who were diagnosed as not having IGD.
b) Effect sizes were evaluated using Cohen's d for the t-test and Cramer's V for the chi-square test.
c) Multiple choice questions were used.
d) For the 30 days prior to the survey.
Sensitivity and specificity of IGDT-10 items
Sensitivity, specificity, positive predictive value (PPV) and negative predictive values of each IGDT-10 item were examined for the modified and original versions (Tables 2 and 3). Both tables also include information on the endorsement rate of each criterion based on the self-report survey for both versions. In the modified version, sensitivity ranged from 75% to 100% for the majority of items. However, it was lower for items related to negative consequences - sensitivity of continuation of gaming despite problems was 62.5% and for negative consequences as low as 39.2%. Specificity of each item was stable across the items - between 73% (loss of control) and 92% (giving up other activities), except for preoccupation (51%). In contrast, the sensitivity of each item in the original version was quite low - between 0% and 33% - with the exception of withdrawal (60.0%). With regard to negative consequences, no positive cases were found, not only for the non-IGD but also for the IGD groups in the original version. In addition, specificity of each item was fairly high - between 82% and almost 100%.
Criterion endorsement and sensitivity, specificity, PPV and NPV of each IGDT-10 item using the original scoring condition against the result of the interview on IGD among study participants (N = 244)a,b)
| IGDT-10 item | Criterion endorsement N (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | |
| IGD (N = 8) | Non-IGD (N = 236) | |||||
| Preoccupation | 2 (25.0%) | 16 (6.8%) | 31.6 | 97.1 | 66.7 | 88.5 |
| Withdrawal | 3 (37.5%) | 4 (1.7%) | 60.0 | 98.3 | 42.9 | 99.2 |
| Tolerance | 0 (0.0%) | 7 (3.0%) | 6.3 | 97.4 | 14.3 | 93.7 |
| Loss of control | 3 (37.5%) | 15 (6.4%) | 32.6 | 98.5 | 83.3 | 86.3 |
| Giving up other activities | 1 (12.5%) | 0 (0.0%) | 0.0 | 99.6 | 0.0 | 98.4 |
| Continuation despite problems | 1 (12.5%) | 3 (1.3%) | 25.0 | 99.2 | 50.0 | 97.5 |
| Deception | 1 (12.5%) | 4 (1.7%) | 17.7 | 99.1 | 60.0 | 94.1 |
| Escape | 2 (25.0%) | 13 (5.5%) | 22.9 | 98.0 | 73.3 | 83.8 |
| Negative consequences | 0 (0.0%) | 0 (0.0%) | –c) | –c) | –c) | –c) |
a) Response of “never” and “sometimes” was regarded as “no” and that of “often” as “yes”.
b) PPV = positive predictive value, NPV = negative predictive values.
c) Unable to be calculated because there were no participants who reported positive for this item of the IGDT-10.
Criterion endorsement and sensitivity, specificity, PPV and NPV of each IGDT-10 item using the modified scoring condition against the result of the interview on IGD among study participants (N = 244)a,b)
| IGDT-10 item | Criterion endorsement N (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | |
| IGD (N = 8) | Non-IGD (N = 236) | |||||
| Preoccupation | 8 (100.0%) | 132 (55.9%) | 100.0 | 50.5 | 27.1 | 100.0 |
| Withdrawal | 5 (62.5%) | 44 (18.6%) | 80.0 | 81.2 | 8.2 | 99.5 |
| Tolerance | 5 (62.5%) | 37 (15.7%) | 75.0 | 86.8 | 28.6 | 98.0 |
| Loss of control | 4 (50.0%) | 85 (36.0%) | 78.3 | 73.2 | 40.5 | 93.6 |
| Giving up other activities | 8 (100.0%) | 16 (6.8%) | 100.0 | 91.7 | 16.7 | 100.0 |
| Continuation despite problems | 5 (62.5%) | 47 (19.9%) | 62.5 | 80.8 | 9.6 | 98.4 |
| Deception | 3 (37.5%) | 32 (13.6%) | 82.4 | 90.8 | 40.0 | 98.6 |
| Escape | 5 (62.5%) | 71 (30.1%) | 79.2 | 80.6 | 50.0 | 94.1 |
| Negative consequences | 3 (37.5%) | 30 (12.8%) | 39.1 | 89.1 | 27.3 | 93.3 |
a) Response of “never” and “sometimes” was regarded as “no” and that of “often” as “yes”.
b) PPV = positive predictive value, NPV = negative predictive values.
Internal consistency, ROC analysis, and cutoff evaluation
The Cronbach's alpha for the Japanese version of IGDT-10 with three response options among 244 subjects was 0.799, while that for the original and modified versions with 9 items and dichotomous response options was 0.579 and 0.782, respectively. The composite reliability for the modified version was 0.789. The area under the ROC curve for the original version was 0.820, while it was 0.898 for the modified version (Fig. 1). With regard to the original version, the distributions of sensitivity, specificity, PPVs and Youden's index were skewed. The highest sensitivity was found when the cutoff was set at 1. If the cutoff was set at 5, as originally proposed, the sensitivity was 0% and the Youden's index was –0.4 (Table 4). In the modified version, the highest Youden's index was found when the cutoff value was set at 5 and it became lower if the cutoff values decreased or increased. The sensitivity, specificity, and diagnostic accuracy were 87.5, 85.2, and 85.3%, respectively, but the PPV was 16.7% (Table 5). These results indicate that the modified Japanese version was superior in screening performance to the original Japanese version and the optimum cutoff value for the former version was 5 or more.
Receiver operating characteristics curves for the original and modified versions
Citation: Journal of Behavioral Addictions 11, 4; 10.1556/2006.2022.00070
True positives and negatives, false positives and negatives, sensitivity, specificity, PPV, NPV, DA and Youden's index using the original scoring condition in IGDT-10 between diagnostic positive and negative groups among study participants (N = 244)a,b)
| Cutoff points | True positives | True negatives | False positives | False negatives | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | DA (%) | Youden's Index (%) |
| 1 | 6 | 198 | 38 | 2 | 75.0 | 83.9 | 13.6 | 99.0 | 83.6 | 58.9 |
| 2 | 5 | 222 | 14 | 3 | 62.5 | 94.1 | 26.3 | 98.7 | 93.0 | 56.6 |
| 3 | 2 | 229 | 7 | 6 | 25.0 | 97.0 | 22.2 | 97.5 | 94.7 | 22.0 |
| 4 | 0 | 234 | 2 | 8 | 0.0 | 99.2 | 0.0 | 96.7 | 95.9 | −0.1 |
| 5 | 0 | 235 | 1 | 8 | 0.0 | 99.6 | 0.0 | 96.7 | 96.3 | 0.0 |
| 6 | 0 | 236 | 0 | 8 | 0.0 | 100.0 | 0.0 | 96.7 | 96.7 | 0.0 |
| 7 | 0 | 236 | 0 | 8 | 0.0 | 100.0 | 0.0 | 96.7 | 96.7 | 0.0 |
| 8 | 0 | 236 | 0 | 8 | 0.0 | 100.0 | 0.0 | 96.7 | 96.7 | 0.0 |
a) Response of “never” and “sometimes” was regarded as “no” and that of “often” as “yes”.
b) PPV = positive predictive value, NPV = negative predictive values, DA = diagnostic accuracy.
True positives and negatives, false positives and negatives, sensitivity, specificity, PPV, NPV, DA and Youden's index using the modified scoring condition in IGDT-10 between diagnostic positive and negative groups among study participants (N = 244)a,b)
| Cutoff points | True positives | True negatives | False positives | False negatives | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | DA (%) | Youden's Index (%) |
| 1 | 8 | 71 | 165 | 0 | 100.0 | 30.1 | 4.6 | 100.0 | 32.4 | 30.1 |
| 2 | 8 | 121 | 115 | 0 | 100.0 | 51.3 | 6.5 | 100.0 | 52.9 | 51.3 |
| 3 | 8 | 151 | 85 | 0 | 100.0 | 63.9 | 8.6 | 100.0 | 65.2 | 63.9 |
| 4 | 7 | 178 | 58 | 1 | 87.5 | 75.4 | 10.8 | 99.4 | 75.8 | 62.9 |
| 5 | 7 | 201 | 35 | 1 | 87.5 | 85.2 | 16.7 | 99.5 | 85.3 | 72.7 |
| 6 | 4 | 215 | 21 | 4 | 50.0 | 91.1 | 16.0 | 98.2 | 89.8 | 41.1 |
| 7 | 2 | 228 | 8 | 6 | 25.0 | 96.6 | 20.0 | 97.4 | 94.3 | 21.6 |
| 8 | 1 | 232 | 4 | 7 | 12.5 | 98.3 | 20.0 | 97.1 | 95.5 | 10.8 |
a) Response of “never” was regarded as “no” and that of “sometimes” and “often” as “yes”.
b) PPV = positive predictive value, NPV = negative predictive values, DA = diagnostic accuracy.
Confirmatory factor analysis
A confirmatory factor analysis (CFA) confirmed the single factor solution of the modified Japanese version. For the CFA goodness of fit, the chi-square value was 60.98 with a P value of less than 0.01. The values of the comparative fit index (CFI) and Tucker-Lewis Index (TLI), indicators of the goodness of fit of the model, were 0.920 and 0.894, respectively. The root mean square error of approximation (RMSA) was 0.072. The composite reliability score was 0.975. All of these indices suggested that the goodness of fit of this model was acceptable. In contrast, we were unable to compute the CFA for the original version because of the absence of participants reporting positive for either item 9 or 10 of the IGDT-10.
A confirmatory factor analysis (CFA) confirmed the single factor solution of the modified Japanese version. For the CFA goodness of fit, the chi-square value was 60.73 with P the value less than 0.01, the goodness of fit index (GFI) and the adjusted goodness of fit index (AGFI) were 0.948 and 0.914, respectively. In addition, the comparative fit index (CFI) was 0.920 and the root mean square error of approximation (RMSA) was 0.071. All of these indices suggested that the goodness of fit of this model was acceptable. In contrast, the CFA for the original version was unable to be computed, because there were no participants who reported positive for item 9 nor 10 of the IGDT-10 (Fig. 2).
Graphical summary of confirmatory factor analysis results obtained from the modified Japanese version of the IGDT-10
Citation: Journal of Behavioral Addictions 11, 4; 10.1556/2006.2022.00070
Rate of individuals with probable IGD in the general young population
Table 6 shows the rate of probable IGD cases among the young population in Japan by gender, based on the results of the modified and original Japanese versions. The estimated rate among male participants was 16.2% (95% confidence interval, 14.7–17.7%), 6.3% (3.5–7.3%) for female participants and 11.3% (10.4–12.2%) in total. If the rate was estimated using the original Japanese version with the cutoff at 5, it would equate to 2.2% (1.5–2.8%) for males, 1.3% (0.8–1.8%) for females and 1.8% (1.3–2.2%) overall.
Estimated rate of probable IGD among the general young population using IGDT-10 with original and modified scoring conditions
| IGDT-10 with original cutoff conditiona) | IGDT-10 with modified cutoff conditionb) | ||||
| Males (N = 2,546) % (95% CI)a) | Females (N = 2,550) % (95% CI) | Total (N = 5,096) % (95% CI) | Males (N = 2,546) % (95% CI) | Females (N = 2,550) % (95% CI) | Total (N = 5,096) % (95% CI) |
| 2.2 (1.5–2.8) | 1.3 (0.8–1.8) | 1.8 (1.3–2.2) | 16.2 (14.7–17.7) | 6.3 (5.3–7.3) | 11.3 (10.4–12.2) |
Note. 95% CI: 95% confidence interval.
a) Response of “never” and “sometimes” was regarded as “no” and that of “often” as “yes”.
b) Response of “never” was regarded as “no” and that of “sometimes” and “often” as “yes”.
Discussion
In this study we validated the Japanese version of the IGDT-10 using 244 participants derived from the younger, general population in Japan, aged between 10 and 29 years old. Participants were interviewed by experts in the diagnosis and treatment of IGD and GD using the Japanese version of the SCI-IGD to determine a diagnosis of DSM-5 IGD. These procedures identified eight participants with a definitive diagnosis of IGD and these served as the reference group. We examined the screening performance of the two Japanese versions with different scoring conditions for IGDT-10. The response “often” was regarded as positive for each IGDT-10 question item in the original version (Király et al., 2017), whereas either “often” or “sometimes” were regarded as an affirmative response in the modified version. One of the main reasons why this study examined less stringent scoring conditions was because the endorsement rate of criteria for the original Japanese version was suggested to be relatively low by clinicians, even when the version was used for clinically diagnosed IGD cases. This belief was confirmed by relatively low sensitivity for the majority of the IGDT-10 criteria of the original version in this study. Low sensitivity was especially true for the negative consequences criterion.
An endorsement rate for the negative consequences item may be lower than the rates for other criteria due to low recognition of negative consequences by young gamers. In addition, the small number subjects with IGD may have made it difficult to identify IGD subjects who would have endorsed the negative consequences item.
The results of the Cronbach's alpha coefficient and the ROC analysis revealed that the modified version showed a higher screening performance than the original version. The sensitivity and specificity analyses of both the original and modified versions against a DSM-5 IGD diagnosis and the presence or absence of each IGD item also supported the superiority of the modified version. The Youden's index indicated that the optimum cutoff for the modified version was 5 or more in order to screen IGD.
The psychometric properties of IGDT-10 have been extensively examined using different language samples. These studies revealed that the Cronbach's alpha coefficient for the 10-item questionnaire with three response options was above the acceptable limit (0.77–0.87), and this ranged from 0.62 to 0.79 in the case of the nine dichotomous items (Chiu, Chen, & Lin, 2018; Evren, Evren, Dalbudak, Topcu, & Kutlu, 2020; Király et al., 2019; Männikkö, Ruotsalainen, Tolvanen, & Kääriäinen, 2019). Confirmatory factor analyses of these studies consistently suggested that the IGDT-10 was a unidimensional construct and factor loadings of all items were positive, statistically significant, and within the conventionally acceptable threshold (Evren et al., 2020; Király et al., 2017; Király et al., 2019; Männikkö et al., 2019). These data were consistent with the data obtained from the modified version in this study. In the case of the original version, Cronbach's alpha for the nine dichotomous items was slightly below the lower limit of the previous studies and the CFA structure was not elucidated.
In contrast, the optimum scoring and cutoff conditions for IGDT-10 has not been well studied. In the original study, the cutoff of the test was established using the results of latent class analysis of the sample studied (Király et al., 2017). However, this analysis may not have captured a meaningful target group of subjects and if the model is incorrect, the value of the resulting estimates is less certain (Albert & Dodd, 2004; Pepe & Janes, 2007).
One study examining the cutoff of the IGDT-10 was undertaken using a reference group in Taiwan (Chiu et al., 2018). Researchers used a structured interview form they had developed to assess DSM-5 IGD and conducted interviews with 76 senior high school students. Eight students were found to have IGD and they were used as the reference group to evaluate the cutoff of the Chinese version of IGDT-10. The authors concluded that the optimum cutoff to differentiate the IGD group from the non-IGD group was 5 or more, which was consistent with the results of the original study (Király et al., 2017). However, the sensitivity of this model was only 44%, while the Youden's index was 42%. These figures are far lower than those of the modified version of this study. The highest Youden's index was found when the cutoff was set at 2 or more in the study (Chiu et al., 2018). In addition, the area under the ROC curve of their study was 0.810, which was comparable to the figure of the original version of this study. These observations suggest that the screening performance of the Chinese version would have increased if a less stringent scoring condition was employed. The data obtained from our study were generated from a younger Japanese demographic utilizing a self-reported administration of the Japanese version of IGDT-10. Therefore, the examination of the cutoff requires further studies to be conducted in other cultures and in their respective languages.
The estimated rate of probable IGD was 1.8% among participants representative of the Japanese young population, if the original Japanese version with a cutoff of 5 or more was used. The estimated rate of probable IGD using the modified Japanese version was far higher than the rate for the original version. The estimated prevalence of reported IGD was diverse, influenced by study design, measurement, and study population. One recent review suggested that the prevalence of problematic gaming was between 0.7% and 27.5% (Mihara & Higuchi, 2017). More recently, a comprehensive review revealed that the prevalence of IGD ranged from 0.21% to 57.5% (Darvesh et al., 2020). Another study based on a meta-analysis of 16 studies reported that the pooled prevalence of problematic gaming among adolescents was 4.6% (Fam, 2018). A similar and more recent study showed that the worldwide prevalence of gaming disorder was 3.05% based on a systematic review and meta-analysis (Stevens, Dorstyn, Delfabbro, & King, 2021). The PPV of the modified Japanese version with a cutoff of 5 or more was 16.7%, meaning that only one out of six individuals screened as positive using this version is believed to actually have IGD. Therefore, the rate obtained using this version may be inflated when the prevalence of probable IGD is estimated. It also indicated that the modified Japanese version is adequate for broadly identifying probable IGD cases for screening purposes rather than for estimating the prevalence of probable IGD.
Finally, the methodological limitations of this study should be summarized. Firstly, the sample size of interviewees was limited, resulting in a participant group diagnosed as having IGD numbering eight, which could lower the reliability of the findings derived from these data. In addition, these samples are not necessarily representative of gamers in the younger Japanese population of this age group. Secondly, an unvalidated instrument (SCI-IGD) for examining the diagnosis of IGD was used due to the lack of a validated instrument. However, the process of examining the diagnosis of IGD by experts who have clinically seen large numbers of patients with IGD is assumed to significantly reduce the risk of misdiagnosis. Thirdly, psychiatric comorbidity which may influence the performance of a screening instrument was not evaluated in study participants. Lastly, the new scoring method for the IGDT-10 was proposed based on the results of the study using only Japanese participants. The suggested scoring and cutoff conditions need to be examined in future studies using other ethnic groups with different gaming cultures.
Conclusion
The Japanese version of the IGDT-10 was validated utilizing 244 participants derived from younger members of the general Japanese population. The scoring and cutoff conditions for the Japanese version was examined using eight participants with a definitive diagnosis of DSM-5 IGD, based on face-to-face interviews, as a gold standard group. Compared to the scoring method proposed by the original study, the less stringent scoring method, where the response of either “often” or “sometimes” to each item was regarded as affirmative (modified version) with a cutoff of 5 or more correlated with higher screening performance. As these data were obtained using only Japanese samples, the suggested scoring and cutoff conditions need to be examined in future studies using other ethnic groups with different gaming cultures.
Funding sources
This study was funded by a research grant on addiction provided to the National Center for Addiction Services Administration by the Ministry of Health, Labour and Welfare of Japan.
Authors' contributions
SM and SH developed the concept and design of the study. All authors contributed to the collection, analysis, and interpretation of data. SM and SH drafted the article. All authors contributed to the revision and approved the final version of the article. All authors had full access to all data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Conflict of interest disclosure
All authors report no financial conflict or other relationship relevant to the subject of this article.
SM, TM, HN, TK and SH have been engaged in the treatment of disorders due to substance use and addictive behaviors including gaming disorder.
SM, TM and SH are members of the World Health Organization (WHO) Work Group on “Behavioural Disorders Associated with Excessive Use of the Internet, Computers, Smartphones and Similar Electronic Devices”, which was established in 2014. They have been involved in developing and reviewing the diagnostic guidelines of gaming disorder in the ICD-11.
Acknowledgments
The authors wish to thank members of the Treatment of Internet Addiction and Research Section of Kurihama Medical Addiction Center for contributing to the data collection of this study.
References
Albert, P. S., & Dodd, L. E. (2004). A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60, 427–435. http://doi.org/10.1111/j.0006-341X.2004.00187.x.
American Psychiatric Association. (2013). The diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Arlington, VA: American Psychiatric Association.
Bantis, L. E., Nakas, C. T., & Reiser, B. (2014). Construction of confidence regions in the ROC space after the estimation of the optimal Youden index-based cut-off point. Biometrics, 70, 212–223. http://doi.org/10.1111/biom.12107.
Chen, J. H., Su, X., Dang, L., & Wu, A. M. S. (2021). Evaluation of the psychometric properties of the Chinese internet gaming disorder checklist (C-IGDC) among Chinese adolescents. Frontiers in Psychiatry, 12, 721397. http://doi.org/10.3389/fpsyt.2021.721397.
Chiu, Y.-C., Pan, Y.-C., & Lin, Y.-H. (2018). Chinese adaptation of the Ten-Item Internet Gaming Disorder Test and prevalence estimate of Internet gaming disorder among adolescents in Taiwan. Journal of Behavioral Addictions, 7, 719–726. http://doi.org/10.1556/2006.7.2018.92.
Darvesh, N., Radhakrishnan, A., Lachance, C. C., Nincic, V., Sharpe, J. P., Ghassemi, M., … Tricco, A. C. (2020). Exploring the prevalence of gaming disorder and internet gaming disorder: A rapid scoping review. Systematic Reviews, 9, 68. http://doi.org/10.1186/s13643-020-01329-2.
Evren, C., Evren, B., Dalbudak, E., Topcu, M., & Kutlu, N. (2020). Psychometric validation of the Turkish ten-item internet gaming disorder test (IGDT-10). Dusunen Adam The Journal of Psychiatry and Neurological Sciences, 33, 19–28. http://doi.org/10.14744/DAJPNS.2019.00057.
Fam, J. Y. (2018). Prevalence of internet gaming disorder in adolescents: A meta-analysis across three decades. Scandinavian Journal of Psychology, 59, 524–531. http://doi.org/10.1111/sjop.12459.
Higuchi, S., Osaki, Y., Kinjo, A., Mihara, S., Maezono, M., Kitayuguchi, T., … Saunders, J. B. (2021). Development and validation of a nine-item short screening test for ICD-11 gaming disorder (GAMES test) and estimation of the prevalence in the general young population. Journal of Behavioral Addictions, 10, 263–280. http://doi.org/10.1556/2006.2021.00041.
IBM Corp. (2017). IBM SPSS statistics for windows, Version 25.0. Armonk, NY: IBM Corp.
Jo, S.-J., Yim, H. W., Lee, H.-K., Lee, H.-C., Choi, J.-S., Baek, K.-Y. (2018). The internet Game Use-Elicited Symptom Screen proved to be a valid tool for adolescents aged 10–19 years. Acta Paediatrica, 107, 511–516. http://doi.org/10.1111/apa.14087.
King, D. L., Chamberlain, S. R., Carragher, N., Billieux, J., Stein, D., Mueller, K., … Delfabbro, P. H. (2020). Screening and assessment tools for gaming disorder: A comprehensive systematic review. Clinical Psychology Review, 77, 101831. http://doi.org/10.1016/j.cpr.2020.101831.
Kinjo, A., & Osaki, Y. (2019). Epidemiology related to gaming disorder. Journal of Clinical and Experimental Medicine (IGAKU NO AYUMI), 271, 567–571. (In Japanese)
Király, O., Bőthe, B., Ramos-Diaz, J., Rahimi-Movaghar, A., Lukavska, K., Hrabec, O., … Demetrovics, Z. (2019). The Ten-Item Internet Gaming Disorder Test (IGDT-10): Measurement invariance and cross-cultural validation across seven language-based samples. Psychologists in Addictive Behaviors, 33, 91–103. http://doi.org/10.1037/adb0000433.
Király, O., Sleczka, P., Pontes, H. M., Urbán, R., Griffiths, M. D., & Demetrovics, Z. (2017). Validation of the ten-item internet gaming disorder test (IGDT-10) and evaluation of the nine DSM-5 internet gaming disorder criteria. Addictive Behaviors, 64, 253–260. http://doi.org/10.1016/j.addbeh.2015.11.005.
Koo, H. J., Han, D. H., Park, S. Y., Kwon, J. H. (2017). The structured clinical interview for DSM-5 internet gaming disorder: Development and validation for diagnosing IGD in adolescents. Psychiatry Investigation, 14, 21–29. http://doi.org/10.4306/pi.2017.14.1.21.
Lemmens, J. S., Valkenburg, P. M., & Gentile, D. A. (2015). The internet gaming disorder scale. Psychological Assessment, 27, 567–582. http://doi.org/10.1037/pas0000062.
Männikkö, N., Ruotsalainen, H., Tolvanen, A., & Kääriäinen, M. (2019). Psychometric properties of the Internet Gaming Disorder Test (IGDT-10) and problematic gaming behavior among Finnish vocational school students. Scandinavian Journal of Psychology, 60, 252–260. http://doi.org/10.111/sjop.12533.
Mihara, S., & Higuchi, S. (2017). Cross-sectional and longitudinal epidemiological studies of internet gaming disorder: A systematic review of the literature. Psychiatry and Clinical Neurosciences, 71, 425–444. http://doi.org/10.1111/pcn.12532.
Nakayama, H., Matsuzaki, T., Mihara, S., Kitayuguchi, T., & Higuchi, S. (2020). Relationship between problematic gaming and age at the onset of habitual gaming. Pediatrics International, 62, 1275–1281. http://doi.org/10.1111/ped.14290.
Pearcy, B. T. D., Roberts, L. D., & McEvoy, P. M. (2016). Psychometric testing of the personal internet gaming disorder evaluation-9: A new measure designed to assess internet gaming disorder. Cyberpsychology, Behavior, and Social Networking, 19, 335–341. http://doi.org/10.1089/cyber.2015.0534.
Pepe, M. S., & Janes, H. (2007). Insights into latent class analysis of diagnostic test performance. Biostatistics, 8, 474–484. http://doi.org/10.1093/biostatistics/kxl038.
Pontes, H. M., & Griffiths, M. D. (2015). Measuring DSM-5 internet gaming disorder: Development and validation of a short psychometric scale. Computers in Human Behavior, 45, 137–143. http://dx.doi.org/10.1016/j.chb.2014.12.006.
Pontes, H. M., Király, O., Demetrovics, Z., & Griffiths, M. D. (2014). The conceptualisation and measurement of DSM-5 internet gaming disorder: The development of the IGD-20 test. Plos One, 9, e110137. http://doi.org/10.1371/journal.pone.0110137.
SAS Institute Inc. (2016). Statistical analysis System for PC, Version 9.4. Cary, NC: SAS Institute Inc.
Statistics Bureau of Japan. (2018). Current population estimates as of October 1st, 2018 .Available from: https://www.stat.go.jp/english/data/jinsui/2.html.
Stevens, M., Dorstyn, D., Delfabbro, P. H., & King, D. L. (2021). Global prevalence of gaming disorder: A systematic review and meta-analysis. Australian and New Zealand Journal of Psychiatry, 55, 553–568. http://doi.org/10.1177/0004867420962851.
Sugaya, N., Shirasaka, T., Takahashi, K., & Kanada, H. (2019). Bio-psychosocial factors of children and adolescents with internet gaming disorder: A systematic review. BioPsychoSocial Medicine, 13, 3. https://doi.org/10.1186/s13030-019-0144-5.
van Rooij, A. J., Schoenmakers, T. M., & van de Mheen, D. (2017). Clinical validation of the C-VAT 2.0 assessment tool for gaming disorder: A sensitivity analysis of the proposed DSM-5 criteria and the clinical characteristics of young patients with ‘video game addiction’. Addictive Behaviors, 64, 269–274. https://doi.org/10.1016/j.addbeh.2015.10.018.
World Health Organization. (2019). ICD-11 for mortality and morbidity statistics, 2019. Available from https://icd.who.int/browse11/l-m/en.
Appendix
The Japanese version of IGDT-10
 |
