View More View Less
  • 1 National Hospital Organization Kurihama Medical and Addiction Center, Yokosuka, Kanagawa, , Japan
  • | 2 Tottori University, Faculty of Medicine, Yonago, Tottori, , Japan
  • | 3 Asahiyama Hospital, Sapporo, Hokkaido, , Japan
  • | 4 University of Lübeck, Lübeck, , Germany
  • | 5 National Centre for Youth Substance Use Research, The University of Queensland, Brisbane, Queensland, , Australia
Open access

Abstract

Background and Aims

A definition of gaming disorder (GD) was introduced in ICD-11. The purpose of this study was to develop a short screening test for GD, utilizing a reference GD group. It also sought to estimate the prevalence of GD among individuals, representative of the general young population in Japan.

Methods

Two hundred eighty one men and women selected from the general population, aged between 10 and 29 years, and 44 treatment seekers at our center completed a self-reported questionnaire comprising candidate questions for the screening test. The reference group with ICD-11 GD was established, based on face-to-face interviews with behavioral addiction experts, using a diagnostic interview instrument. The questions in the screening test were selected to best differentiate those who had GD from those who did not, and the cutoff value was determined using the Youden index.

Results

A nine-item screening test (GAMES test) was developed. The sensitivity and specificity of the test were both 98% and the positive predictive value in the study sample was 91%. The GAMES test comprised two factors, showed high internal consistency and was highly reproducible. The estimated prevalence of GD among the general young population was 7.6% (95% confidence interval; 6.6–8.7%) for males and 2.5% (1.9–3.2%) for females, with a combined prevalence of 5.1% (4.5–5.8%).

Discussion and Conclusion

The GAMES test shows high validity and reliability for screening of ICD-11 GD. The estimated prevalence of 5.1% among the general young population was comparable to the pooled estimates of young people globally.

Abstract

Background and Aims

A definition of gaming disorder (GD) was introduced in ICD-11. The purpose of this study was to develop a short screening test for GD, utilizing a reference GD group. It also sought to estimate the prevalence of GD among individuals, representative of the general young population in Japan.

Methods

Two hundred eighty one men and women selected from the general population, aged between 10 and 29 years, and 44 treatment seekers at our center completed a self-reported questionnaire comprising candidate questions for the screening test. The reference group with ICD-11 GD was established, based on face-to-face interviews with behavioral addiction experts, using a diagnostic interview instrument. The questions in the screening test were selected to best differentiate those who had GD from those who did not, and the cutoff value was determined using the Youden index.

Results

A nine-item screening test (GAMES test) was developed. The sensitivity and specificity of the test were both 98% and the positive predictive value in the study sample was 91%. The GAMES test comprised two factors, showed high internal consistency and was highly reproducible. The estimated prevalence of GD among the general young population was 7.6% (95% confidence interval; 6.6–8.7%) for males and 2.5% (1.9–3.2%) for females, with a combined prevalence of 5.1% (4.5–5.8%).

Discussion and Conclusion

The GAMES test shows high validity and reliability for screening of ICD-11 GD. The estimated prevalence of 5.1% among the general young population was comparable to the pooled estimates of young people globally.

Introduction

Digital gaming has become increasingly popular around the globe (King & Delfabbro, 2019). However, gaming when unrestricted, can be highly absorbing, time-consuming, and potentially addictive for vulnerable individuals. The World Health Organization (WHO) included gaming disorder (GD) as a disorder due to addictive behaviors in the eleventh revision of the International Classification of Diseases (ICD-11) in May 2019 (WHO, 2019). The decision to do so was based on the accumulated evidence and followed intensive discussion among experts from all over the world (WHO, 2016; Saunders et al., 2017). According to the definition of GD in ICD-11, three clinical manifestation criteria and one functional impairment criterion need to be met to make a diagnosis of GD (WHO, 2019). In addition, these behavior patterns and impairments are required to have persisted for at least 12 months, except in cases where severe symptoms are present.

Prior to the inclusion in ICD-11, preliminary diagnostic criteria of internet gaming disorder (IGD) were published by the American Psychiatric Association in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) in 2013 (American Psychiatric Association, 2013). These comprised nine criteria with at least five criteria needing to be met for a diagnosis of IGD to be made.

In a recent review of studies, the prevalence of problematic gaming ranged from 0.7% to 27.5%, depending on study design, measurement, and study population (Mihara & Higuchi, 2017). The prevalence was higher among males than females and tended to be higher among younger rather than older people in some studies. In a meta-analysis of 16 studies published prior to 2017, the pooled prevalence of problematic gaming was 4.6% among adolescents, confirming the extent of the disorder (Fam, 2018). More recently, a comprehensive review revealed that the prevalence of IGD ranged from 0.21% to 57.5%. However, no study has reported the estimated prevalence of ICD-11 GD (Darvesh et al., 2020).

The development of a screening instrument for a newly conceptualized disorder is of great importance. Firstly, it can serve as an instrument to identify individuals with the disorder for the purposes of preventive and treatment interventions, and it may be used as an adjunctive tool with which to make a diagnosis. Secondly, it can be used to make a provisional estimate of the prevalence of the disorder in a general or specific population. In the case of DSM-5 IGD, several assessment tools and screening tests based on the diagnostic criteria of IGD have been developed (Pontes, Király, Demetrovics, & Griffiths, 2014; Lemmens, Valkenburg, & Gentile, 2015; Pontes & Griffiths, 2015; Pearcy, Roberts, & McEvoy, 2016; Király et al., 2017; van Rooij, Schoenmakers, & van de Mheen, 2017; Jo et al., 2018). Some of these have been extensively validated and translated into other languages (Pontes & Griffiths, 2015; Király et al., 2017). One such test, the Ten-Item Internet Gaming Disorder Test (IGDT-10) has been translated into 11 languages with cross-cultural validation conducted for seven of these (Király et al., 2019).

In a recent comprehensive and systematic review of screening and assessment tools for IGD and GD, the authors identified all available English-language tools (N = 32), and evaluated these in relation to 1) conceptual and practical considerations, 2) alignment with DSM-5 and ICD-11 criteria, 3) type and quantity of studies and samples, and 4) psychometric properties (King et al., 2020). The result showed that the coverage of IGD and GD criteria is inconsistent and tools converge on the importance of screening for control over gaming and functional impairment. It also highlighted how no single instrument was clearly superior to any other. Currently, a gold standard tool that allows for the identification of individuals with GD correctly and accurately has yet to be created, which is unsurprising, as the definition of GD was first published by WHO only in 2019. The authors suggest that the development of a gold standard instrument would be invaluable in enabling professionals to correctly identify gaming-related harms, and develop more effective intervention strategies for those in need (King et al., 2020).

With regard to ICD-11 GD, two assessment tools have been developed. The first tool, called the Gaming Disorder Test (GDT), consists of 4 items representing the three diagnostic guidelines and the functional impairment (Pontes et al., 2019). This test was validated using the results of online surveys given to subjects in the UK and China. However, the purpose of this instrument was not to diagnose GD but to assess its severity and consequences, and the suggested cutoff score was not validated (Pontes et al., 2019). The second instrument - the Gaming Disorder Scale for Adolescents (GADIS-A) – comprised 10 items and was created based on the consensus of clinical and scientific experts (Paschke, Austermann, & Thomasius, 2020). It was designed to exclusively be used with adolescents. The classification of the presence of GD was determined by latent class analysis and the 4 criteria of IGD, similar in content to the key elements of GD, were used as a reference standard (Ko, Lin, Lin, & Yen, 2019).

These two instruments employed polytomous scales using the Likert approach, whereas the screening test this study has developed utilized a dichotomous (yes/no) approach. A study comparing the psychometric properties of the polytomous and dichotomous approaches has been conducted, using the same 9-item and 27-item instruments for IGD (Lemmens et al., 2015). It revealed that both types of assessment were reliable, showed good criteria-related validity, and additionally, the dichotomous 9-item IGD scale showed solid psychometric properties and was the most practical scale for diagnostic purposes.

In this context we have developed a screening instrument for ICD-11 GD using a novel but standardized approach. We conducted a self-reported survey using a questionnaire containing a set of questions relating to the four key elements of the ICD-11 GD definition of gaming behavior among randomly selected community sample groups and treatment seekers. Due to the absence of a gold standard instrument for GD, we developed a new interview tool for establishing a reference group in order to develop a screening test. Survey participants with GD were identified using this instrument based on face-to-face interviews conducted by behavioral addiction experts followed by a panel discussion among the interviewers. Question items for the instrument and the cutoff score to screen for GD were statistically determined to best differentiate participants with GD from those without GD. The psychometric properties were also explored to ascertain if it could be a valid and reliable screening instrument for assessing GD. The new instrument comprising 9 items was named the GAMing Engagement Screener test (GAMES test) (Appendix). In this study we also estimated the prevalence of GD using the GAMES test among subjects representing the general population, aged between 10 and 29 years, in Japan.

Methods

Study participants

The subjects of this study were drawn from a representative cohort of the Japanese population residing in Japan who responded to a self-reported questionnaire survey about internet game use and daily habits from January through March 2019. For that, 300 national census spots were randomly identified and 9,000 individuals aged 10–29 years randomly selected from the inhabitant register of the local communities containing these census spots. Professional interviewers visited each household containing a respondent identified using the aforementioned process, requesting that the questionnaire be completed and arranging for its subsequent collection. Of the survey participants, 5,096 individuals gave valid responses to the questionnaire (response rate: 56.6%). Despite the relatively low response rate, the age, gender, and geographical distributions of these respondents were similar to those of initial survey invitees (N = 9,000). However, the response rate tended to be lower for participants who were 20 years old or older compared to those under 20 years old. With regard to geographical distribution, the response rates in the Kanto and Kinki areas tended to be lower than for those in other areas. The questionnaire that was administered to the 5,096 participants included all questions that were used for developing the GAMES test. Of these respondents, 2,953 gave consent for the additional survey and 766 of those living in the broader Tokyo metropolitan area were invited to participate in the study to develop a screening test. The interview for this investigation was carried out in July and August 2019, and a total of 281 individuals who had consented to participate in the interview subsequently attended and were enrolled in this study. Of the 766 invitees and 281 participants, Tokyo was overrepresented compared to those from adjacent prefectures, but no difference was found in relation to age and gender. Because the number of subjects diagnosed as having GD was too small for detailed statistical analysis, the same self-reported questionnaire survey was additionally conducted among the 44 outpatients who had been definitively diagnosed as having ICD-11 GD by addiction psychiatrists at the Specialized Internet Addiction Clinic of the Center (Fig. 1).

Fig. 1.
Fig. 1.

Flow diagram for identifying study participants in the development of the GAMES test and estimating the prevalence of GD. GAMES = Gaming Engagement Screener. GD = gaming disorder

Citation: Journal of Behavioral Addictions 10, 2; 10.1556/2006.2021.00041

Sources of questions in the screening test

The candidate questions for the screening test were prepared from the following sources: 1) Ten questions adapted for gaming on the basis of the questions related to 3 clinical manifestation criteria of GD, selected from the Internet Related Disorder-Clinical Assessment Tool (I-CAT) (Brandt et al., 2018; Besser, Loerbroks, Bischof, Bischof, & Rumpf, 2019); 2) six original questions related to the GD criteria, prepared utilizing the day-to-day clinical experience of psychiatrists and clinical psychologists in charge of GD treatment at the Center; 3) ten questions constituting the Japanese version of the IGDT-10 (Király et al., 2017); 4) three questions for the evaluation of functional impairment, designed to assess the influence of GD on social activities using the modified 11-grade visual analog scale (VAS) of Sheehan (Sheehan, Harnett-Sheehan, & Raj, 1996; Hodgins, 2013); and 5) fifteen independent candidate questions for the evaluation of functional impairment derived from clinical observation of GD patients at the Center.

Of these 44 questions, 5 questions not found in the ICD-11 definition of GD (related to preoccupation, tolerance, withdrawal symptoms, deception, and escapism in the IGDT-10) were excluded, and the validity of the remaining 39 potential candidate questions were analyzed. Each candidate question fell under one of the following categories found in the definition of ICD-11 GD: A) loss of control over gaming, B) increasing prioritization of gaming in life, C) functional impairment due to gaming, and D) continuation of gaming despite negative consequences.

Diagnosis of GD

GD in the study participants was definitively diagnosed based on face-to-face interviews by professionals at the Center with expertise in the diagnosis and treatment of GD. Because no validated tool for ICD-11 GD diagnosis had been published at the time of the interviews we developed the interview tool for diagnosis used in the present study (Supplementary material). To minimize differences in symptom evaluation arising between interviewers, the latter had been trained in advance using model cases, so that evaluations could be performed in a standardized manner. The interviews for the diagnosis were carried out individually and 18 individuals were identified with suspected GD. The results obtained in the questionnaire survey were totally blind to interviewers. All experts involved in the interviews later met and of the 18 individuals, seven were diagnosed to have GD. When combined with 44 outpatients diagnosed with GD by addiction psychiatrists and confirmed with the same diagnostic instrument, there was a total of 51 individuals with confirmed GD.

Screening test development

Reference group

The gaming disorder group comprised 51 persons definitively diagnosed, as above. The control group comprised 274 subjects, after the exclusion of the 7 individuals who had been diagnosed with GD.

Statistical evaluation of questions

First, the relationship of individual candidate questions to the presence/absence of GD was judged using WAIC (widely applicable information criterion) and WBIC (widely applicable Bayesian information criterion), which are evaluation indicators of the degree of fitness of a statistical model (Watanabe, 2010, 2013). Both WAIC and WBIC can be used, regardless of the data distribution, and the lower the values, the greater the fitness of the model.

To avoid the problem of multicollinearity, the variance inflation factor (VIF) was calculated. No combination of questions yielded VIF values exceeding 10, allowing us to rule out multicollinearity. When the Spearman correlation coefficient between two items was used for the evaluation of multicollinearity, a high correlation coefficient was noted sporadically. When the correlation coefficient between two items was 0.6 or higher, only one of the two was adopted in the multivariate statistical model.

Selection of variables for screening and setting their cutoff level

Next, the variables were selected in accordance with the definition of GD. For each of the 4 categories (A through D) mentioned above, 2 variables with a relatively low WAIC or WBIC were selected. Then, multiple logistic regression analysis was conducted, using the presence/absence of GD as the dependent variable with the gender, age, and presence/absence of candidate items as covariates for adjustment to evaluate the degree of fitness of the statistical model. The Hosmer-Lemeshow test was used to evaluate the applicability for logistic regression models. We determined in advance that an item which was met by at least two out of seven diagnosed individuals among the general young population would be included in the test, because an item positive for only one diagnosed participant was deemed inappropriate.

As the time spent gaming on weekdays has been shown to be a variable useful for the identification of GD, this was added to the 8 variables selected by the above-mentioned process to yield the Gaming Engagement Screener test (GAMES test). The sensitivity and specificity of this screening test were evaluated by separately conducting an analysis for optimal identification of the combined 51 subjects with GD. The optimum cutoff level was determined using the Youden’s Index (Sensitivity + Specificity - 1) (Bantis, Nakas, & Reiser, 2014).

Evaluation of the reliability and validity of the items constituting the GAMES test

Evaluation by factor analysis

An exploratory factor analysis was conducted to examine whether or not the items constituting the screening test possessed a factor structure conforming to the framework of the GD definition. Principal factor analysis for 325 subjects was used as the method for this factor analysis. The generalized least squares method was employed for factor extraction. Promax rotation was conducted for interpretation of the results of factor analysis. The Bartlett sphericity test and Kaiser-Meyer-Olkin measure of sampling adequacy were used to check the prerequisites for the factor analysis (Malhotra, 1999; Hair, Black, Babin, & Anderson, 2010).

Relationship to the IGDT-10 score

The degree of agreement between the total GAMES test score and the total IGDT-10 score and between the presence of GD (score 5 or higher) and the presence of IGD were examined. Because the Japanese version of the IGDT-10 has not been validated, the presence of IGD was determined using the originally suggested cutoff point (Király et al., 2017). The weighted κ coefficient and the intraclass correlation coefficient (ICC) served as the degree of agreement.

Evaluation of internal consistency and test-retest reliability

Internal consistency was evaluated using Cronbach’s alpha and the omega coefficients. For evaluation of the test-retest reproducibility, a second self-reported survey using the same questionnaire was conducted two weeks later involving a randomly selected half of the participants (N = 146) and the 44 outpatients with GD. Responses were collected from 109 survey participants and from all 44 outpatients. Excluding the 9 survey participants who failed to give valid answers to the primary questions, the responses of the remaining 144 subjects were included in the analysis. Reproducibility, an indicator of reliability, was evaluated with the weighted κ coefficient and ICC.

Estimation of the prevalence of GD among young Japanese

The GAMES test was applied to 5,096 respondents of the initial survey, representing young Japanese aged between 10 and 29 years. The data were adjusted by every one-year age group and gender for the estimated population in Japan as of October 1, 2018 (Statistics Bureau of Japan, 2018).

All statistical analyses of the data were conducted using SPSS Version 25.0 (IBM Corp, 2017).

Ethics

This study was approved by the Ethics Committee of the Center (Approval No. 332). Informed consent to participate in the study was obtained from all participants. If participants were younger than 18 years old informed consent was obtained from not only the participants but also their guardians. The study procedures were performed in accordance with the Declaration of Helsinki.

Results

Development of the GAMES test

Selection of 8 items representing 4 criteria of GD

Of the 39 items analyzed, 11 items received negative responses from all 7 individuals identified as having GD in the survey of the general population. Excluding these 11 items, the WAIC and WBIC values of the remaining 28 items are shown in Table 1. The items with low WAIC and WBIC values that received an affirmative answer from 2 or more of these 7 individuals were questions 4, 5, and 6 in Group A, questions 7, 8, 9, and 11 in Group B, questions 13, 14, 15, 16, 20, and 21 in Group C, and questions 24 and 26 in Group D.

Table 1.

WAIC and WBIC values and odds ratios of candidate items for the presence/absence of gaming disorder and the number of items affirmed by 7 individuals from the general population with gaming disordera,b)

No.Sourcec)Diagnostic categoryQuestion itemsWAICWBICWAIC+ WBICRank by WAICRank by WBICRank by WAIC+WBICYes by person without disorderNo by person without disorderYes by person with disorderNo by person with disorderOdds ratioYes by general population subject with disorder (n = 7)
1I-CATAHave you felt the continuing desire to stop or reduce gaming because you noticed to use games too much?279.37140.94420.312625267020429223.844
2I-CATAHave you repeatedly tried to stop or reduce gaming because you thought to use games too much and have been unsuccessful with this attempt?293.10147.89440.992828286520917341.611
3OriginalAHave you often played games under circumstances where you were not allowed to (e.g., during school or work hours)?278.62141.66420.282526251126313388.181
4OriginalAHave you often been unable to stop gaming when you should have?244.33124.60368.921819196920542913.864
5OriginalAHave you often played games for longer than intended before you started gaming?273.56139.11412.672424241551194657.066
6IGDT-10AHave you ever in the past 12 months unsuccessfully tried to reduce the time spent on games?239.11120.88359.9916151515259272419.433
7I-CATBHave you noticed that you have significantly lost interest in important activities, such as sports, hobbies, or meetings with friends or relatives because of gaming?209.24106.75316.005559265361570.673
8I-CATBHave you limited or given up important activities, such as sports, hobbies, or meetings with friends or relatives because of gaming?222.37113.66336.031110118266321956.003
9OriginalBIs gaming the most important part of your daily life?234.13118.11352.241413143271262593.954
10OriginalBDo you usually give first priority to gaming when making a plan or schedule?238.96121.22360.171516166268262546.451
11OriginalBIs gaming more important than study, work, fellowship with your friends, or family events?245.14125.60370.732020204270232855.453
12IGDT-10BHave you selected gaming over meeting friends or engaging in hobbies or sports activities that you had previously enjoyed during the last 12 months?216.82111.80328.62797027416351
13SheehanCTo what extent has gaming cast an unfavorable impact on your school or job performance during the past 12 months. Please encircle the matching point along the line given below.183.5193.38276.892122025442959.274
14SheehanCTo what extent has gaming cast an unfavorable impact on your social activities (meeting friends, hobbies, etc.) during the past 12 months. Please encircle the matching point along the line given below.220.22115.49335.701011106268321975.237
15SheehanCTo what extent has gaming cast an unfavorable impact on your family life and roles within the family during the past 12 months. Please encircle the matching point along the line given below.211.72108.82320.5466618256351631.116
16OriginalCReduction in school or job performance241.03122.37363.4017171737237361515.374
17OriginalCChange of school (e.g., from ordinary senior high school to correspondence high school)283.36144.26427.6127272702745461
18OriginalCDecrease in the number of real-world friends262.44133.22395.67222222027414371
19OriginalCWorsened relationships with family members229.67116.77346.441212124270282382.171
20OriginalCDifficulty in getting up in the morning (30 days or more during the past 12 months)219.52111.16330.6898923251381331.903
21OriginalCDay/night reversal or a tendency towards it (30 days or more during past 12 months)207.95105.56313.5244411263371463.193
22OriginalCSocial withdrawal (6 months or more in total during the past 12 months)232.43118.70351.1313141322722625141.441
23OriginalCFailure to eat regularly251.82130.51382.3321212111263242721.253
24I-CATDHave you continued gaming although you endangered your education or risked or lost your job because of gaming?181.2995.17276.461218266429155.173
25I-CATDHave you continued gaming although it has caused or aggravated significant physical problems or illnesses (e.g. back or eye pain, headache, joint problems, fainting, significant overweight or underweight)?266.13135.26401.392323233623827247.442
26I-CATDHave you continued gaming despite experiencing mental health problems caused by gaming, e.g. becoming depressed or anxious, or experiencing problems with sleeping?218.75111.12329.8787819255371435.473
27I-CATDHave you continued gaming although your gaming costs so much money that it has caused you serious problems (e.g. through the purchase of games, add-ons, games hardware, game currency, game items, subscriptions, apps or other things)?244.85123.94368.791918184270232855.453
28IGDT-10DHave you played a lot despite negative consequences (for instance losing sleep, not being able to do well in school or work, having arguments with your family or friends, and/or neglecting important duties)?202.74105.37308.1133332712922119.081

a) Eleven items affirmed (answered “Yes”) by no subject from the general population were deleted.

b) WAIC = widely applicable information criterion, WBIC = widely applicable Bayesian information criterion.

c) I-CAT = the Internet Related Disorder-Clinical Assessment Tool, Original = original questions, IGDT-10 = the Ten-Item Internet Gaming Disorder Test, Sheehan = the modified 11-grade visual analog scale of Sheehan.

Two items were selected from each of the 4 groups and multiple logistic regression analysis was repeatedly conducted. The highest R2 (Cox-Snell R2 = 0.548, Nagelkerke R2 = 0.943) was recorded by the model including questions 4 and 5 from Group A, questions 7 and 9 from Group B, questions 16 and 21 from Group C, and questions 24 and 26 from Group D. All of these items were met for at least 3 out of 7 diagnosed individuals from among the young population. When the presence/absence of GD was predicted using these 8 variables with a cutoff score of 0.5 (absence = 0/presence = 1), the accuracy of the prediction was 96.1%. The test of Hosmer and Lemeshow yielded a P value of 0.999. Thus, the null hypothesis was not rejected and the model fitness was judged as high.

The results of the multiple logistic regression analysis are shown in Table 2. Because the data from outpatients with GD were added to this analysis to enable multivariate analysis, the odds ratio changed as the number of added outpatients increased or decreased. The frequency of subjects providing affirmative answers for each item differed between the GD and control groups; the odds ratios were quite high, and confidence intervals were also wide. Therefore, we judged that the odds ratio value may not be important in deciding whether to include/exclude a given item from the screening test.

Table 2.

Results of multiple logistic regression analysis using presence/absence of gaming disorder serving as a dependent variablea)

Item No.IDb)Question itemsYes by person without disorderNo by person without disorderYes by person with disorderNo by person with disorderAdjusted odds ratio95% confidence interval
14-AHave you often been unable to stop gaming when you should have?6920542928.2(0.75–1,063.9)
25-AHave you often played games for longer than intended before you started gaming?1551194650.02(0.00–0.95)
37-BHave you noticed that you have significantly lost interest in important activities, such as sports, hobbies, or meetings with friends or relatives because of gaming?9265361517.0(1.00–287.4)
49-BIs gaming the most important part of your daily life?32712625667.2(7.71–57,767)
516-CReduction in school or job performance37237361547.3(1.46–1,533)
621-CDay/night reversal or a tendency towards day/night reversal (30 days or more during past 12 months)112633714111.0(2.28–5,402)
724-DHave you continued gaming although you endangered your education or risked or lost your job because of gaming?826642945.5(1.17–1771)
826-DHave you continued gaming despite experiencing mental health problems caused by gaming, e.g. becoming depressed or anxious, or experiencing problems with sleeping?19255371449.4(1.51–1,621)

a) Adjusted for gender, age and presence/absence of job.

b) ID is the combination of the original number and the diagnostic category in Table 1.

Cutoff score and sensitivity and specificity

Table 3 shows the data on the sensitivity and specificity of the 8-item screening test without the ninth item and the 9-item GAMES test. The table additionally gives the results of the analysis of the cutoff level for the screening test under two settings; combined 51 subjects with GD and only 7 subjects from the general population. Based on the Youden index values, the optimal cutoff score was ≥4 (full score = 8, score point of 1 for each item that elicited an affirmative response and 0 for each item that elicited a negative response among the 8 items) for the combined data. In this case, a sensitivity of 94% and specificity of 97% were obtained.

Table 3.

Analysis for setting the cutoff level of the draft screening test

Gaming disorder (+)Gaming disorder (−)SensitivitySpecificityPositive predictive valuePositive Likelihood ratioYouden Index
YesNoYesNo
8-item screening testa)
Analysis on combined subjectsc)≥2 points510981761.000.640.342.800.64
≥3 points501342400.980.880.607.900.86
≥4 points48392650.940.970.8428.650.91
≥5 points42922720.820.990.95112.820.82
≥6 points331802740.651.001.000.65
Analysis confined to the general population≥2 points70981761.000.640.072.800.64
≥3 points70342401.000.880.178.060.88
≥4 points6192650.860.970.4026.100.82
≥5 points2522720.290.990.5039.140.28
≥6 points0702740.001.000.000.000.00
9-item screening test (GAMES test)b)
Analysis on combined subjectsc)≥3 points510612131.000.780.464.490.78
≥4 points510222521.000.920.7012.450.92
≥5 points50152690.980.980.9153.730.96
≥6 points46522720.900.990.96123.570.89
≥7 points391202740.761.001.000.76
Analysis confined to the general population≥3 points70612131.000.780.104.490.78
≥4 points70222521.000.920.2412.450.92
≥5 points7052691.000.980.5854.800.98
≥6 points4322720.570.990.6778.290.56
≥7 points1602740.141.001.000.14

a) A screening test reflecting 4 key elements in the definition of gaming disorder.

b) A screening test including the time spent gaming on weekdays.

c) Combined subjects of the general population and treatment seekers.

When the average gaming time on a typical weekday was added to the items analyzed, both the sensitivity and specificity increased, accompanied by further improvement of the screening efficiency. When the cut-off score for a “positive” case was set at ≥ 5 (full score = 10, 9 items in total comprising the above-mentioned 8 items in addition to average gaming time, with the latter assigned a score of 0 if less than 2 h, 1 if more than 2 h but less than 6 h, and 2 if 6 h or more), the combined analysis on the general population and treatment seeking groups showed a sensitivity of 98%, specificity of 98%, and the area under the curve (AUC) was 0.981. When the analysis was confined to the general population it also showed significant screening accuracy (sensitivity 100%, specificity 98% and AUC 0.991). The positive predictive value was 91% for the combined data and 58% for the analysis confined to the general population.

Reliability and validity of the GAMES test

Evaluation of the constituent concept validity by factor analysis

When an exploratory factor analysis was conducted on the 9-item screening test, the Kaiser-Meyer-Olkin measure of sampling adequacy was as high as 0.91. The Bartlett spherical test endorsed the statistical significance (χ2 (36) = 1,122.1, P < 0.01). Two factors were extracted based on the Kaiser-Guttman Rule on the number of factors. Table 4 shows the factor pattern after Promax rotation (κ = 4) following extraction of factors using the generalized least squares method.

Table 4.

Result of explanatory factor analysis of the 9 items of GDST-9a)

Item No.Factor 1 loadingsFactor 2 loadingsCommunalities
10.1050.5440.39
2−0.1570.7390.43
30.793−0.0530.58
40.699−0.1020.44
50.1720.4850.42
60.759−0.0490.53
70.730.1590.73
80.6650.0420.52
90.6780.1160.59
Proportion variance0.440.06

a) The inter-factorial correlation between factor 1 and 2 was 0.68.

Regarding the first of the two extracted factors, high factor loads were obtained for Items 3, 4, 6, 7, 8, and 9, suggesting that these were items associated with “prioritization of gaming and negative consequences.” In regard to the second factor, high factor loads were obtained for Items 1, 2, and 5, suggesting that these were items associated with “loss of control.” Because the proportion variance of factor 1 was far higher than that of factor 2, the former factor dominates the factor structure of this instrument.

Relationship to the IGDT-10 data

The weighted κ coefficient for the degree of agreement between the GAMES test total score and the IGDT-10 total score was 0.40, with an ICC of 0.54. The κ coefficient for the degree of agreement between GAMES test scores of ≥5 and IGDT-10 scores of ≥5 was 0.25, with an ICC of 0.25, indicating a lower degree of agreement.

Internal consistency and test-retest reliability

The alpha and omega coefficients for the GAMES test were 0.98 and 0.86, respectively. In regard to the test-retest reproducibility, both the weighed κ coefficient for GAMES test score and ICC were quite high (0.86 and 0.96, respectively).

Estimated prevalence of ICD-11 GD in the general young population

Table 5 shows the estimated prevalence of ICD-11 GD among the young population in Japan by 5-year age group and gender, based on the result of the GAMES test. The prevalence tended to be higher among younger respondents and to decrease with the advancement of age for both genders. The estimated prevalence was 7.6% (95% confidence interval, 6.5–8.7%) for males, 2.5% (1.9–3.2%) for females and 5.1% (4.5–5.8%) in total.

Table 5.

The estimated prevalence of ICD-11 gaming disorder by age class and gendera)

Age class (years old)Males % (95%CI)Females % (95%CI)Total % (95%CI)
10–149.2 (8.3–10.1)3.1 (2.6–3.7)6.2 (5.7–6.8)
15–1912.0 (10.9–13.1)3.0 (2.4–3.6)7.6 (7.0–8.3)
20–246.2 (4.9–7.5)2.3 (1.5–3.0)4.1 (3.4–4.8)
25–294.0 (2.9–5.0)1.5 (0.9–2.2)2.7 (2.1–3.3)
Total7.6 (6.5–8.7)2.5 (1.9–3.2)5.1 (4.5–5.8)

a) CI = confidence interval.

Discussion

We developed a new screening test for ICD-11 GD, the GAMES test, using a reference group with GD comprising study participants from both the general population and treatment seekers. A diagnosis of GD was made based on face-to-face interviews with experts in the diagnosis and treatment of behavioral addictions using a diagnostic interview instrument newly developed for this study. The interviews and self-reported questionnaire surveys were conducted on the same day to avoid changes in target conditions (Reitsma, Rutjes, Khan, Coomarasamy, & ). The question items were selected based on appropriate statistical methods from a pool of questions derived from multiple sources (Besser et al., 2019; Brandt et al., 2018; Hodgins, 2013; Király et al., 2017; Sheehan et al., 1996).

In previous studies on the development of screening and assessment tools for both IGD and GD, other approaches have been employed (Bertens et al., 2013). Several studies used a latent class analysis to assess diagnostic accuracy because a reference group assessment of IGD or GD was unavailable (Pontes et al., 2014; Lemmens et al., 2015; Király et al., 2017; Paschke et al., 2020). However, this analysis may not have captured the real target group of subjects and if the model is incorrect, it is unclear whether the resulting estimates are meaningful (Albert & Dodd, 2004; Pepe & Janes, 2007). Other studies did not validate the cutoff scores, perhaps because question items in the instruments were matched to the diagnostic criteria of IGD (Pearcy et al., 2016; van Rooij, Schoenmakers, & van de Mheen, 2017). Other instruments, have been developed, primarily to assess the severity of the disorder and its consequences, rather than for the purpose of making a diagnosis (Pontes et al., 2019; Pontes & Griffiths, 2015).

The GAMES test showed extremely high sensitivity and specificity for subjects, who were representative of the general young population in Japan. Positive predictive values were also high, not only for the general population but also for the combined subject groups. The results suggest that if this screening test were applied to the general population, about 60% of the individuals judged as positive would have GD. The GAMES test showed high internal consistency with the alpha and omega coefficients standing at well over 0.8. In addition, the test-retest reliability of the test was high, demonstrating that the GAMES test evaluation was highly reproducible.

Exploratory factor analysis revealed a two-factor solution to best model the nine items of the GAMES test. The first factor was a major element, explaining approximately 44% of the portion variance. This factor represents the prioritization of gaming and negative consequences or functional impairment. The prioritization of gaming, which can be described as “salience” is one of the most important characteristics of addictions and is the core feature of the clinical manifestations of GD. Two out of four key elements of the GD definition are related to negative consequences. One of the unique features of the GD definition is the inclusion of a functional impairment item which must be met in order to make a diagnosis. It seeks to avoid the overdiagnosis of GD, which is common to polythetic approaches such as the IGD criteria (Billieux, Schimmenti, Khazaal, Maurage, & Heeren, 2015; Billieux et al., 2017). For these reasons, it is appropriate that these items constitute the main factor in the GAMES test. The second factor comprises three items mainly related to loss of control over gaming.

A recently-developed instrument, the GADIS-A, comprising ten items, also has a two-factor solution (Paschke et al., 2020). The main factor of GADIS-A is “impending or manifest consequences due to gaming behavior”, which may be construed as reflecting priority of gaming and impairment, and the second, is “pathological gaming behavior”. The values of portion variances for the two factors are almost equal in the case of GADIS-A (Paschke et al., 2020). The structure of another screening tool for GD - GDT – showed a one-factor solution with almost the same variance value as factor 1 of the GAMES test (Pontes et al., 2019).

Concurrent validity was examined using the Japanese version of IGDT-10 (Király et al., 2019). The reasons for using this instrument were two-fold: 1) assessment or screening instruments for ICD-11 GD were unavailable when this study was conducted, and 2) IGDT-10 has been widely used internationally as a screening instrument for IGD (Király et al., 2019). The IGDT-10 was translated into Japanese and the accuracy of translation confirmed by comparing the original English version and the English version that was retranslated from the Japanese version, but validation of the instrument has not been conducted using Japanese samples. Nonetheless, the Japanese version has already been used in previous studies (Kinjo & Osaki, 2019; Nakayama, Matsuzaki, Mihara, Kitayuguchi, & Higuchi, 2020). The results of this study showed that correlation between the GAMES test and IGDT-10 was too low. This finding should not be interpreted as a low concurrent validity of the GAMES test, rather that the diagnoses of GD and IGD may have different constructs. Another reason may be related to the use of the unvalidated Japanese version of IGDT-10 in the study. Low concordance of the two diagnoses in the same clinical samples has been reported. For example, the rates of subjects with IGD who were also diagnosed as having GD were 64% in a study in Taiwan and as low as 16% in a study in Korea (Jo et al., 2019; Ko et al., 2019).

To the authors’ knowledge, this study showed for the first time the estimated prevalence of ICD-11 GD among the general young population (Darvesh et al., 2020). As previously mentioned, the prevalence of IGD has been estimated in a number of studies in different countries. Findings were quite diverse, depending on the characteristics of study samples, screening instruments used, and method of data collection (Chia et al., 2020; Darvesh et al., 2020; Fam, 2018; Mihara & Higuchi, 2017; Saunders et al., 2017; ). The estimated prevalence of GD among representative Japanese young people in this study was 5.1%, which is comparable to a pooled estimate of problematic gaming among young people, derived from a meta-analysis of 16 studies conducted globally (Fam, 2018). The estimated prevalence of GD also showed a male preponderance and a higher tendency among adolescents compared to older generations, which is consistent with previous studies on the prevalence of IGD (Darvesh et al., 2020; Mihara & Higuchi, 2017). This study provides a reliable tool to estimate the prevalence of GD, which is expected to serve as a basis on which measures against the disorder will be implemented in different jurisdictions.

Methodological limitations

Finally, the methodological limitations of this study should be summarized. Firstly, the sample size of interviewees and the comparatively low prevalence of GD in the general population resulted in a small participant group diagnosed as having GD. To increase the statistical strength, we added treatment seekers with GD to the reference group. We separately analyzed the data of the seven participants that had GD and those of the combined 51 participants with GD in the process of developing the GAMES test. Secondly, because validated interview schedules for the diagnosis of GD are not available, we devised a schedule and administered it to interviewees. In addition, the interview was conducted by one interviewer, although each interviewer had clinically seen numerous patients with GD. Recently, there has been a debate in the field around whether GD/IGD is a formative construct (van Rooij, Looy, & Billieux, 2017). GD has a formative construct feature in which the four categories (A through D) are essential in order to define GD. On the contrary, the GAMES test is a reflective construct, not requiring all four categories to be met for screening purposes. Fourthly, with regard to the evaluation of concurrent validity, an additional study using appropriate instruments may be necessary in the future. Finally, the GAMES test was developed solely utilizing Japanese participants. In this context and considering the small sample size of the reference group, this study can be regarded as a pilot study, and the validity and reliability of this instrument needs to be examined in future studies using larger samples and other ethnic groups with different gaming cultures.

Conclusion

In this study, a new screening test for ICD-11 GD, consisting of 9 items (GAMES test) was developed utilizing a reference group with GD and a non-GD general young population. A diagnosis of GD was made based on face-to-face interviews with experts in behavioral addictions using a diagnostic interview instrument newly developed for this study. The question items for the GAMES test were selected employing appropriate statistical procedures from a pool of questions derived from multiple sources contained in a self-reported questionnaire conducted on the same day as the face-to-face interviews. The test showed high sensitivity, specificity, positive predictive values, and internal consistency, and is highly reproducible. It consists of one main factor representing the salience and negative consequences of GD and a highly correlated second factor related to loss of control.

The GAMES test allowed us, for the first time, to estimate the prevalence of GD among subjects representing the general young population in Japan. The estimated prevalence was 5.1% which is comparable to the pooled estimate of problematic gaming derived from a meta-analysis of studies conducted globally. Consistent with previous studies, it showed a male preponderance and higher tendency among adolescents compared to older generations.

Funding sources

This study was funded by a research grant on addiction given to the National Center for Addiction Services Administration and subsidized by the Ministry of Health, Labour and Welfare of Japan.

Authors’ contributions

SH, YO, AK, HJR and JBS developed the conception and design of the study. All authors contributed to the collection, analysis, and interpretation of data. SH and YO drafted the article. All authors contributed to the revision and approved the final version of the article. All authors had full access to the data in this study, and take responsibility for the integrity of the data and the accuracy of the data analysis.

Conflict of interest disclosure

All authors report no financial conflict or other relationship relevant to the subject of this article. SH, SM, MM, TK, TM, HN and JBS have been engaged in the treatment of disorders due to substance use and addictive behaviors including gaming disorder.

SH, SM, HJR and JBS are members of the World Health Organization (WHO) Work Group on “Behavioural Disorders Associated with Excessive Use of the Internet, Computers, Smartphones and Similar Electronic Devices”, which was established in 2014. They have been involved in developing and reviewing the diagnostic guidelines of gaming disorder in the ICD-11. The preliminary findings of the present study were presented to the Work Group in 2019. The views expressed in this publication are those of the authors and do not necessarily represent the decisions or policies of the WHO.

Acknowledgments

This study was funded by a research grant on addiction given to Kurihama Medical and Addiction Center and subsidized by the Ministry of Health, Labour and Welfare of Japan. The authors wish to thank Ms. Kiyoko Hidaka, Mr. Satoshi Furuno, Ms. Naoko Endo, Ms. Haruka Miyakoshi and Ms. Moemi Shibasaki for contributing to the data collection for this study.

Supplementary material

Supplementary data to this article can be found online at https://doi.org/10.1556/2006.2021.00041.

References

  • Albert, P. S., & Dodd, L. E. (2004). A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60, 427435. http://doi.org/10.1111/j.0006-341X.2004.00187.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, 5th ed. (DSM-5). Arlington, VA: American Psychiatric Association.

    • Search Google Scholar
    • Export Citation
  • Bantis, L. E., Nakas, C. T., & Reiser, B. (2014). Construction of confidence regions in the ROC space after the estimation of the optimal Youden index-based cut-off point. Biometrics, 70, 212223. http://doi.org/10.1111/biom.12107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bertens, L. C. M., Broekhuizen, B. D. L., Naaktgeboren, A., Rutten, F. H., Hoes, A. W., Van Mourik, Y., … Restsma, J. B. (2013). Use of expert panels to define the reference standard in diagnostic research: A systematic review of published methods and reporting. Plos Medicine, 10, e1011531. http://doi.org/10.1371/journal.pmed.1001531.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Besser, B., Loerbroks, L., Bischof, G., Bischof, A., & Rumpf, H. J. (2019). Performance of the DSM-5-based criteria for Internet addiction: A factor analytical examination of three samples. Journal of Behavioral Addiction, 8, 288294. http://doi.org/10.1556/2006.8.2019.19.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Billieux, J., King, D., Higuchi, S., Achab, S., Bowden-Jones, H., Hao, W., … Poznyak, V. (2017). Functional impairment matters in the screening and diagnosis of gaming disorder. Journal of Behavioral Addictions, 6, 285289. http://doi.org/10.1556/2006.6.2017.036.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Billieux, J., Schimmenti, A., Khazaal, Y., Maurage, P., & Heeren, A. (2015). Are we overpathologizing everyday life? A tenable blueprint for behavioral addiction research. Journal of Behavioral Addictions, 4, 119123. http://doi.org/10.1556/2006.4.2015.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandt, D., Glanert, S., Bischof, G., Besser, B., Bischof, A., & Rumpf H. J. (2018). Determination of the test-retest reliability of a computerized diagnostic interview for internet-related disorders. Journal of Behavioral Addiction, 7(Suppl. 1), 47.

    • Search Google Scholar
    • Export Citation
  • Chia, D. X. Y., Ng, C. W. L., Kandasami, G., Seow, M. Y. L., Choo, C. C., Chew, P. K. H., … Zhang, M. W. B. (2020). Prevalence of internet addiction and gaming disorder in Southeast Asia: A meta-analysis. International Journal of Environmental Research and Public Health, 17, 2582. http://doi.org/10.3390/ijerph17072582.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Darvesh, N., Radhakrishnan, A., Lachance, C. C., Ninic, V., Sharpe, J. P., Ghassemi. M., … Tricco, A. C. (2020). Exploring the prevalence of gaming disorder and internet gaming disorder: A rapid scoping review. Systematic Reviews, 9, 68. http://doi.org/10.1186/s13643-020-01329-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fam, J. Y. (2018). Prevalence of Internet gaming disorder in adolescents: A meta-analysis across three decades. Scandinavian Journal of Psychology, 59, 524531. http://doi.org/10.1111/sjop.12459.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective. New Jersey: Pearson Prentice Hall.

    • Search Google Scholar
    • Export Citation
  • Hodgins, D. C. (2013). Reliability and validity of the Sheehan disability scale modified for pathological gambling. BMC Psychiatry, 13, 177. http://doi.org/10.1186/1471-244X-13-177.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IBM Corp. (2017). IBM SPSS statistics for windows, version 25.0. NY: IBM Corp.

  • Jo, Y. S., Bhang, S. Y., Choi, J. S., Lee, H. K., Lee, S. Y., & Kweon, Y. S.(2019). Clinical characteristics of diagnosis of internet gaming disorder: Comparison of DSM-5 IGD and ICD-11 GD diagnosis. Journal of Clinical Medicine, 8, 945. http://doi.org/10.3390/jcm8070945.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jo, S.-J., Yim, H. W., Lee, H.-K., Lee, H.-C., Choi, J.-S., & Baek, K.-Y. (2018). The internet gaming use-elicited symptom screen proved to be a valid tool for adolescents aged 10–19 years. Acta Paediatrica, 107, 511516. http://doi.org/10.1111/apa.14087 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, D., Chamberlain, S. R., Carragher, N., Billieux, J., Stein, D., Muller, K., … Delfabbro, P. H. (2020). Screening and assessment tools for gaming disorder: A comprehensive systematic review. Clinical Psychology Review, 77, 101831. http://doi.org/10.1016/j.cpr.2020.101831.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, D., & Delfabbro, P. (2019). Internet gaming disorder: Theory, assessment, treatment, and prevention. London: Academic Press.

  • Kinjo, A., & Osaki, Y. (2019). Epidemiology related to gaming disorder. Journal of Clinical and Experimental Medicine (IGAKU NO AYUMI), 271, 567571. (In Japanese).

    • Search Google Scholar
    • Export Citation
  • Király, O., Bõthe, B., Ramos-Diaz, J., Rahimi-Movaghar, A., Lukavska, K., Hrabec, O., … Demetrovics, Z. (2019). Ten-Item Internet gaming disorder test (IGDT-10): Measurement invariance and cross-cultural validation across seven language-based samples. Psychology of Addictive Behaviors, 33, 91103. http://doi.org/10.1037/adb0000433.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Király, O., Sleczka, P., Pontes, H. M., Urbán, R., Griffiths, M. D., & Demetrovics, Z. (2017). Validation of the ten-item internet gaming disorder test (IGDT-10) and evaluation of the nine DSM-5 internet gaming disorder criteria. Addictive Behaviors, 64, 253260. http://doi.org/10.1016/j.addbeh.2015.11.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ko, C. H., Lin, H. C., Lin, P. C., & Yen, J. Y. (2019). Validity, functional impairment and complications related to Internet gaming disorder in the DSM-5 and gaming disorder in the ICD-11. Australian & New Zealand Journal of Psychiatry, 4867419881499. http://doi.org/10.1177/0004867419881499.

    • Search Google Scholar
    • Export Citation
  • Lemmens, J. S., Valkenburg, P. M., & Gentile, D. A. (2015). The internet gaming disorder scale. Psychological Assessment, 27, 567582. http://doi.org/10.1037/pas0000062.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Malhotra, N. K. (1999). Marketing research: An applied orientation. New Jersey: Prentice Hall.

  • Mihara, S., & Higuchi, S. (2017). Cross-sectional and longitudinal epidemiological studies of Internet gaming disorder: A systematic review of the literature. Psychiatry and Clinical Neurosciences, 71, 425444. http://doi.org/10.1111/pcn.12532.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nakayama, H., Matsuzaki, T., Mihara, S., Kitayuguchi, T., & Higuchi, S. (2020). Relationship between problematic gaming and age at the onset of habitual gaming. Pediatrics International. 65, 12751281. http://doi.org/10.1111/ped.14290.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paschke, K., Austermann, M. S., & Thomasius, R. (2020). Assessing ICD-11 gaming disorder in adolescent gamers: Development and validation of the gaming disorder scale for adolescents (GADIS-A). Journal of Clinical Medicine, 9, 993. http://doi.org/10.3390/jcm9040993.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pearcy, B. T. D., Roberts, L. D., & McEvoy, P. M. (2016). Psychometric testing of the personal internet gaming disorder evaluation-9: A new measure designed to assess internet gaming disorder. Cyberpsychology, Behavior, and Social Networking, 19, 335341. http://doi.org/10.1089/cyber.2015.0534.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pepe, M. S., & Janes, H. (2007). Insights into latent class analysis of diagnostic test performance. Biostatistics, 8, 474484. http://doi.org/10.1093/biostatistics/kxl038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pontes, H. M., & Griffiths, M. D. (2015). Measuring DSM-5 internet gaming disorder: Development and validation of a short psychometric scale. Computers in Human Behavior, 45, 137143. http://dx.doi.org/10.1016/j.chb.2014.12.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pontes, H. M., Schivinski, B., Sindermann, C., Li, M., Becker, B., Zhou, M., & Montag, C.(2019). Measurement and conceptualization of gaming disorder according to the world Health Organization framework: The development of the gaming disorder test. International Journal of Mental Health Addiction. http://dx.doi.org/10.1007/s11469.

    • Search Google Scholar
    • Export Citation
  • Reitsma, J. B., Rutjes, A. W., Khan, K. S., Coomarasamy, A., & Bossuuyt, P. M. (2009). A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of Clinical Epidemiology, 62, 797806. http://doi.org/10.1016/j.jclinepi.2009.02.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saunders, J. B., Hao, W., Long, J., King, D. L., Mann, K., Fauth-Büler, M., … Poznyak, V. (2017). Gaming disorder: Its delineation as an important condition for diagnosis, management, and prevention. Journal of Behavioral Addictions, 6, 271279. https://doi.org/10.1556/2006.6.2017.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sheehan, D. V., Harnett-Sheehan, K., & Raj, B. A. (1996). The measurement of disability. International Clinical Psychopharmacology, 11(Suppl. 3), 8995.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Statistics Bureau of Japan. Current population estimates as of October 1st, 2018. Tokyo: Statistics Bureau of Japan. Retrieved from https://www.stat.go.jp/english/data/jinsui/2.html.

    • Search Google Scholar
    • Export Citation
  • van Rooij, A. J., Looy, J. V., & Billieux, J. (2017). Internet gaming disorder as a formative construct: Implications for conceptualization and measurement. Psychiatry and Clinical Neurosciences, 71, 445458. https://doi.org/10.1111/pcn.12404.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Rooij, A. J., Schoenmakers, T. M., & van de Mheen, D. (2017). Clinical validation of the C-VAT 2.0 assessment tool for gaming disorder: A sensitivity analysis of the proposed DSM-5 criteria and the clinical characteristics of young patients with ‘video game addiction’. Addictive Behaviors, 64, 269274. https://doi.org/10.1016/j.addbeh.2015.10.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 35713594.

    • Search Google Scholar
    • Export Citation
  • Watanabe, S. (2013). A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 14, 867897.

  • World Health Organization. (2016). Behavioural disorders associated with excessive use of the Internet, computers, smartphones and similar electronic devices: Clinical descriptions, diagnostic guidelines and priorities for international research: Meeting report. Geneva, Switzerland: World Health Organization.

    • Search Google Scholar
    • Export Citation
  • World Health Organization. (2019). ICD-11 for mortality and morbidity statistics 2019. Retrieved from https://icd.who.int/browse11/l-m/en.

    • Search Google Scholar
    • Export Citation

Appendix The Gaming Engagement Screener (GAMES) test

Please read questions 1 to 8 and select an appropriate answer (Yes or No) for each question in relation to your gaming behavior over the last 12 months. For question 9, please select the most appropriate answer. Gaming means games played on smartphones, gaming devices, personal computers, or other similar devices.

No.Question itemsAnswer
YesNo
1Have you often been unable to stop gaming when you should have?10
2Have you often played games for longer than intended before you started gaming?10
3Have you noticed that you have significantly lost interest in important activities, such as sports, hobbies, or meeting with friends or relatives because of gaming?10
4Is gaming the most important part of your daily life?10
5Has your school or job performance deteriorated because of gaming?10
6Have you experienced day/night reversal or a tendency towards day/night reversal because of gaming (30 days or more during the past 12 months)?10
7Have you continued gaming although you endangered your education or risked or lost your job because of gaming?10
8Have you continued gaming despite experiencing mental health problems caused by gaming e.g. becoming depressed or anxious, or experiencing problems with sleeping?10
9Approximately how many hours do you spend gaming on a typical weekday?0: Less than 2 h

1: At least 2 h but less than 6 h

2: 6 h or longer

Evaluation method: A total score of 5 or more obtained by totaling the scores for all the items suggests the presence of the ICD-11 gaming disorder.

Supplementary Materials

  • Albert, P. S., & Dodd, L. E. (2004). A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60, 427435. http://doi.org/10.1111/j.0006-341X.2004.00187.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, 5th ed. (DSM-5). Arlington, VA: American Psychiatric Association.

    • Search Google Scholar
    • Export Citation
  • Bantis, L. E., Nakas, C. T., & Reiser, B. (2014). Construction of confidence regions in the ROC space after the estimation of the optimal Youden index-based cut-off point. Biometrics, 70, 212223. http://doi.org/10.1111/biom.12107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bertens, L. C. M., Broekhuizen, B. D. L., Naaktgeboren, A., Rutten, F. H., Hoes, A. W., Van Mourik, Y., … Restsma, J. B. (2013). Use of expert panels to define the reference standard in diagnostic research: A systematic review of published methods and reporting. Plos Medicine, 10, e1011531. http://doi.org/10.1371/journal.pmed.1001531.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Besser, B., Loerbroks, L., Bischof, G., Bischof, A., & Rumpf, H. J. (2019). Performance of the DSM-5-based criteria for Internet addiction: A factor analytical examination of three samples. Journal of Behavioral Addiction, 8, 288294. http://doi.org/10.1556/2006.8.2019.19.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Billieux, J., King, D., Higuchi, S., Achab, S., Bowden-Jones, H., Hao, W., … Poznyak, V. (2017). Functional impairment matters in the screening and diagnosis of gaming disorder. Journal of Behavioral Addictions, 6, 285289. http://doi.org/10.1556/2006.6.2017.036.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Billieux, J., Schimmenti, A., Khazaal, Y., Maurage, P., & Heeren, A. (2015). Are we overpathologizing everyday life? A tenable blueprint for behavioral addiction research. Journal of Behavioral Addictions, 4, 119123. http://doi.org/10.1556/2006.4.2015.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandt, D., Glanert, S., Bischof, G., Besser, B., Bischof, A., & Rumpf H. J. (2018). Determination of the test-retest reliability of a computerized diagnostic interview for internet-related disorders. Journal of Behavioral Addiction, 7(Suppl. 1), 47.

    • Search Google Scholar
    • Export Citation
  • Chia, D. X. Y., Ng, C. W. L., Kandasami, G., Seow, M. Y. L., Choo, C. C., Chew, P. K. H., … Zhang, M. W. B. (2020). Prevalence of internet addiction and gaming disorder in Southeast Asia: A meta-analysis. International Journal of Environmental Research and Public Health, 17, 2582. http://doi.org/10.3390/ijerph17072582.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Darvesh, N., Radhakrishnan, A., Lachance, C. C., Ninic, V., Sharpe, J. P., Ghassemi. M., … Tricco, A. C. (2020). Exploring the prevalence of gaming disorder and internet gaming disorder: A rapid scoping review. Systematic Reviews, 9, 68. http://doi.org/10.1186/s13643-020-01329-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fam, J. Y. (2018). Prevalence of Internet gaming disorder in adolescents: A meta-analysis across three decades. Scandinavian Journal of Psychology, 59, 524531. http://doi.org/10.1111/sjop.12459.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective. New Jersey: Pearson Prentice Hall.

    • Search Google Scholar
    • Export Citation
  • Hodgins, D. C. (2013). Reliability and validity of the Sheehan disability scale modified for pathological gambling. BMC Psychiatry, 13, 177. http://doi.org/10.1186/1471-244X-13-177.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • IBM Corp. (2017). IBM SPSS statistics for windows, version 25.0. NY: IBM Corp.

  • Jo, Y. S., Bhang, S. Y., Choi, J. S., Lee, H. K., Lee, S. Y., & Kweon, Y. S.(2019). Clinical characteristics of diagnosis of internet gaming disorder: Comparison of DSM-5 IGD and ICD-11 GD diagnosis. Journal of Clinical Medicine, 8, 945. http://doi.org/10.3390/jcm8070945.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jo, S.-J., Yim, H. W., Lee, H.-K., Lee, H.-C., Choi, J.-S., & Baek, K.-Y. (2018). The internet gaming use-elicited symptom screen proved to be a valid tool for adolescents aged 10–19 years. Acta Paediatrica, 107, 511516. http://doi.org/10.1111/apa.14087 .

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, D., Chamberlain, S. R., Carragher, N., Billieux, J., Stein, D., Muller, K., … Delfabbro, P. H. (2020). Screening and assessment tools for gaming disorder: A comprehensive systematic review. Clinical Psychology Review, 77, 101831. http://doi.org/10.1016/j.cpr.2020.101831.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, D., & Delfabbro, P. (2019). Internet gaming disorder: Theory, assessment, treatment, and prevention. London: Academic Press.

  • Kinjo, A., & Osaki, Y. (2019). Epidemiology related to gaming disorder. Journal of Clinical and Experimental Medicine (IGAKU NO AYUMI), 271, 567571. (In Japanese).

    • Search Google Scholar
    • Export Citation
  • Király, O., Bõthe, B., Ramos-Diaz, J., Rahimi-Movaghar, A., Lukavska, K., Hrabec, O., … Demetrovics, Z. (2019). Ten-Item Internet gaming disorder test (IGDT-10): Measurement invariance and cross-cultural validation across seven language-based samples. Psychology of Addictive Behaviors, 33, 91103. http://doi.org/10.1037/adb0000433.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Király, O., Sleczka, P., Pontes, H. M., Urbán, R., Griffiths, M. D., & Demetrovics, Z. (2017). Validation of the ten-item internet gaming disorder test (IGDT-10) and evaluation of the nine DSM-5 internet gaming disorder criteria. Addictive Behaviors, 64, 253260. http://doi.org/10.1016/j.addbeh.2015.11.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ko, C. H., Lin, H. C., Lin, P. C., & Yen, J. Y. (2019). Validity, functional impairment and complications related to Internet gaming disorder in the DSM-5 and gaming disorder in the ICD-11. Australian & New Zealand Journal of Psychiatry, 4867419881499. http://doi.org/10.1177/0004867419881499.

    • Search Google Scholar
    • Export Citation
  • Lemmens, J. S., Valkenburg, P. M., & Gentile, D. A. (2015). The internet gaming disorder scale. Psychological Assessment, 27, 567582. http://doi.org/10.1037/pas0000062.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Malhotra, N. K. (1999). Marketing research: An applied orientation. New Jersey: Prentice Hall.

  • Mihara, S., & Higuchi, S. (2017). Cross-sectional and longitudinal epidemiological studies of Internet gaming disorder: A systematic review of the literature. Psychiatry and Clinical Neurosciences, 71, 425444. http://doi.org/10.1111/pcn.12532.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nakayama, H., Matsuzaki, T., Mihara, S., Kitayuguchi, T., & Higuchi, S. (2020). Relationship between problematic gaming and age at the onset of habitual gaming. Pediatrics International. 65, 12751281. http://doi.org/10.1111/ped.14290.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paschke, K., Austermann, M. S., & Thomasius, R. (2020). Assessing ICD-11 gaming disorder in adolescent gamers: Development and validation of the gaming disorder scale for adolescents (GADIS-A). Journal of Clinical Medicine, 9, 993. http://doi.org/10.3390/jcm9040993.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pearcy, B. T. D., Roberts, L. D., & McEvoy, P. M. (2016). Psychometric testing of the personal internet gaming disorder evaluation-9: A new measure designed to assess internet gaming disorder. Cyberpsychology, Behavior, and Social Networking, 19, 335341. http://doi.org/10.1089/cyber.2015.0534.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pepe, M. S., & Janes, H. (2007). Insights into latent class analysis of diagnostic test performance. Biostatistics, 8, 474484. http://doi.org/10.1093/biostatistics/kxl038.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pontes, H. M., & Griffiths, M. D. (2015). Measuring DSM-5 internet gaming disorder: Development and validation of a short psychometric scale. Computers in Human Behavior, 45, 137143. http://dx.doi.org/10.1016/j.chb.2014.12.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pontes, H. M., Schivinski, B., Sindermann, C., Li, M., Becker, B., Zhou, M., & Montag, C.(2019). Measurement and conceptualization of gaming disorder according to the world Health Organization framework: The development of the gaming disorder test. International Journal of Mental Health Addiction. http://dx.doi.org/10.1007/s11469.

    • Search Google Scholar
    • Export Citation
  • Reitsma, J. B., Rutjes, A. W., Khan, K. S., Coomarasamy, A., & Bossuuyt, P. M. (2009). A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of Clinical Epidemiology, 62, 797806. http://doi.org/10.1016/j.jclinepi.2009.02.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saunders, J. B., Hao, W., Long, J., King, D. L., Mann, K., Fauth-Büler, M., … Poznyak, V. (2017). Gaming disorder: Its delineation as an important condition for diagnosis, management, and prevention. Journal of Behavioral Addictions, 6, 271279. https://doi.org/10.1556/2006.6.2017.039.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sheehan, D. V., Harnett-Sheehan, K., & Raj, B. A. (1996). The measurement of disability. International Clinical Psychopharmacology, 11(Suppl. 3), 8995.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Statistics Bureau of Japan. Current population estimates as of October 1st, 2018. Tokyo: Statistics Bureau of Japan. Retrieved from https://www.stat.go.jp/english/data/jinsui/2.html.

    • Search Google Scholar
    • Export Citation
  • van Rooij, A. J., Looy, J. V., & Billieux, J. (2017). Internet gaming disorder as a formative construct: Implications for conceptualization and measurement. Psychiatry and Clinical Neurosciences, 71, 445458. https://doi.org/10.1111/pcn.12404.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • van Rooij, A. J., Schoenmakers, T. M., & van de Mheen, D. (2017). Clinical validation of the C-VAT 2.0 assessment tool for gaming disorder: A sensitivity analysis of the proposed DSM-5 criteria and the clinical characteristics of young patients with ‘video game addiction’. Addictive Behaviors, 64, 269274. https://doi.org/10.1016/j.addbeh.2015.10.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 35713594.

    • Search Google Scholar
    • Export Citation
  • Watanabe, S. (2013). A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 14, 867897.

  • World Health Organization. (2016). Behavioural disorders associated with excessive use of the Internet, computers, smartphones and similar electronic devices: Clinical descriptions, diagnostic guidelines and priorities for international research: Meeting report. Geneva, Switzerland: World Health Organization.

    • Search Google Scholar
    • Export Citation
  • World Health Organization. (2019). ICD-11 for mortality and morbidity statistics 2019. Retrieved from https://icd.who.int/browse11/l-m/en.

    • Search Google Scholar
    • Export Citation
The author instruction is available in PDF.
Please, download the file from HERE

Dr. Zsolt Demetrovics
Institute of Psychology, ELTE Eötvös Loránd University
Address: Izabella u. 46. H-1064 Budapest, Hungary
Phone: +36-1-461-2681
E-mail: jba@ppk.elte.hu

Indexing and Abstracting Services:

  • Web of Science [Science Citation Index Expanded (also known as SciSearch®)
  • Journal Citation Reports/Science Edition
  • Social Sciences Citation Index®
  • Journal Citation Reports/ Social Sciences Edition
  • Current Contents®/Social and Behavioral Sciences
  • EBSCO
  • GoogleScholar
  • PsycINFO
  • PubMed Central
  • SCOPUS
  • Medline
  • CABI
2020  
Total Cites 4024
WoS
Journal
Impact Factor
6,756
Rank by Psychiatry (SSCI) 12/143 (Q1)
Impact Factor Psychiatry 19/156 (Q1)
Impact Factor 6,052
without
Journal Self Cites
5 Year 8,735
Impact Factor
Journal  1,48
Citation Indicator  
Rank by Journal  Psychiatry 24/250 (Q1)
Citation Indicator   
Citable 86
Items
Total 74
Articles
Total 12
Reviews
Scimago 47
H-index
Scimago 2,265
Journal Rank
Scimago Clinical Psychology Q1
Quartile Score Psychiatry and Mental Health Q1
  Medicine (miscellaneous) Q1
Scopus 3593/367=9,8
Scite Score  
Scopus Clinical Psychology 7/283 (Q1)
Scite Score Rank Psychiatry and Mental Health 22/502 (Q1)
Scopus 2,026
SNIP  
Days from  38
submission  
to 1st decision  
Days from  37
acceptance  
to publication  
Acceptance 31%
Rate  

2019  
Total Cites
WoS
2 184
Impact Factor 5,143
Impact Factor
without
Journal Self Cites
4,346
5 Year
Impact Factor
5,758
Immediacy
Index
0,587
Citable
Items
75
Total
Articles
67
Total
Reviews
8
Cited
Half-Life
3,3
Citing
Half-Life
6,8
Eigenfactor
Score
0,00597
Article Influence
Score
1,447
% Articles
in
Citable Items
89,33
Normalized
Eigenfactor
0,7294
Average
IF
Percentile
87,923
Scimago
H-index
37
Scimago
Journal Rank
1,767
Scopus
Scite Score
2540/376=6,8
Scopus
Scite Score Rank
Cllinical Psychology 16/275 (Q1)
Medicine (miscellenous) 31/219 (Q1)
Psychiatry and Mental Health 47/506 (Q1)
Scopus
SNIP
1,441
Acceptance
Rate
32%

 

Journal of Behavioral Addictions
Publication Model Gold Open Access
Submission Fee none
Article Processing Charge 850 EUR/article
Printed Color Illustrations 40 EUR (or 10 000 HUF) + VAT / piece
Regional discounts on country of the funding agency World Bank Lower-middle-income economies: 50%
World Bank Low-income economies: 100%
Further Discounts Editorial Board / Advisory Board members: 50%
Corresponding authors, affiliated to an EISZ member institution subscribing to the journal package of Akadémiai Kiadó: 100%
Subscription Information Gold Open Access
Purchase per Title  

Journal of Behavioral Addictions
Language English
Size A4
Year of
Foundation
2011
Publication
Programme
2021 Volume 10
Volumes
per Year
1
Issues
per Year
4
Founder Eötvös Loránd Tudományegyetem
Founder's
Address
H-1053 Budapest, Hungary Egyetem tér 1-3.
Publisher Akadémiai Kiadó
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 2062-5871 (Print)
ISSN 2063-5303 (Online)

Senior editors

Editor(s)-in-Chief: Zsolt DEMETROVICS

Assistant Editor(s): Csilla ÁGOSTON

Associate Editors

  • Judit BALÁZS (ELTE Eötvös Loránd University, Hungary)
  • Joel BILLIEUX (University of Lausanne, Switzerland)
  • Matthias BRAND (University of Duisburg-Essen, Germany)
  • Anneke GOUDRIAAN (University of Amsterdam, The Netherlands)
  • Daniel KING (Flinders University, Australia)
  • Ludwig KRAUS (IFT Institute for Therapy Research, Germany)
  • H. N. Alexander LOGEMANN (ELTE Eötvös Loránd University, Hungary)
  • Anikó MARÁZ (Humboldt University of Berlin, Germany)
  • Astrid MÜLLER (Hannover Medical School, Germany)
  • Marc N. POTENZA (Yale University, USA)
  • Hans-Jurgen RUMPF (University of Lübeck, Germany)
  • Attila SZABÓ (ELTE Eötvös Loránd University, Hungary)
  • Róbert URBÁN (ELTE Eötvös Loránd University, Hungary)
  • Aviv M. WEINSTEIN (Ariel University, Israel)

Editorial Board

  • Max W. ABBOTT (Auckland University of Technology, New Zealand)
  • Elias N. ABOUJAOUDE (Stanford University School of Medicine, USA)
  • Hojjat ADELI (Ohio State University, USA)
  • Alex BALDACCHINO (University of Dundee, United Kingdom)
  • Alex BLASZCZYNSKI (University of Sidney, Australia)
  • Kenneth BLUM (University of Florida, USA)
  • Henrietta BOWDEN-JONES (Imperial College, United Kingdom)
  • Beáta BÖTHE (University of Montreal, Canada)
  • Wim VAN DEN BRINK (University of Amsterdam, The Netherlands)
  • Gerhard BÜHRINGER (Technische Universität Dresden, Germany)
  • Sam-Wook CHOI (Eulji University, Republic of Korea)
  • Damiaan DENYS (University of Amsterdam, The Netherlands)
  • Jeffrey L. DEREVENSKY (McGill University, Canada)
  • Naomi FINEBERG (University of Hertfordshire, United Kingdom)
  • Marie GRALL-BRONNEC (University Hospital of Nantes, France)
  • Jon E. GRANT (University of Minnesota, USA)
  • Mark GRIFFITHS (Nottingham Trent University, United Kingdom)
  • Heather HAUSENBLAS (Jacksonville University, USA)
  • Tobias HAYER (University of Bremen, Germany)
  • Susumu HIGUCHI (National Hospital Organization Kurihama Medical and Addiction Center, Japan)
  • David HODGINS (University of Calgary, Canada)
  • Eric HOLLANDER (Albert Einstein College of Medicine, USA)
  • Jaeseung JEONG (Korea Advanced Institute of Science and Technology, Republic of Korea)
  • Yasser KHAZAAL (Geneva University Hospital, Switzerland)
  • Orsolya KIRÁLY (Eötvös Loránd University, Hungary)
  • Emmanuel KUNTSCHE (La Trobe University, Australia)
  • Hae Kook LEE (The Catholic University of Korea, Republic of Korea)
  • Michel LEJOXEUX (Paris University, France)
  • Anikó MARÁZ (Eötvös Loránd University, Hungary)
  • Giovanni MARTINOTTI (‘Gabriele d’Annunzio’ University of Chieti-Pescara, Italy)
  • Frederick GERARD MOELLER (University of Texas, USA)
  • Daniel Thor OLASON (University of Iceland, Iceland)
  • Nancy PETRY (University of Connecticut, USA)
  • Bettina PIKÓ (University of Szeged, Hungary)
  • Afarin RAHIMI-MOVAGHAR (Teheran University of Medical Sciences, Iran)
  • József RÁCZ (Hungarian Academy of Sciences, Hungary)
  • Rory C. REID (University of California Los Angeles, USA)
  • Marcantanio M. SPADA (London South Bank University, United Kingdom)
  • Daniel SPRITZER (Study Group on Technological Addictions, Brazil)
  • Dan J. STEIN (University of Cape Town, South Africa)
  • Sherry H. STEWART (Dalhousie University, Canada)
  • Attila SZABÓ (Eötvös Loránd University, Hungary)
  • Ferenc TÚRY (Semmelweis University, Hungary)
  • Alfred UHL (Austrian Federal Health Institute, Austria)
  • Johan VANDERLINDEN (University Psychiatric Center K.U.Leuven, Belgium)
  • Alexander E. VOISKOUNSKY (Moscow State University, Russia)
  • Kimberly YOUNG (Center for Internet Addiction, USA)

 

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Jun 2021 0 0 0
Jul 2021 0 134 109
Aug 2021 0 547 415
Sep 2021 0 303 300
Oct 2021 0 245 181
Nov 2021 0 199 180
Dec 2021 0 0 0