View More View Less
  • 1 Department of Psychology, University of Calgary, Calgary, AB, , Canada
Open access

Background and aims

To date, no research has examined the viability of using behavioral tasks typical of cognitive and neuropsychology within addiction populations through online recruitment methods. Therefore, we examined the reliability and validity of three behavioral tasks of impulsivity common in addiction research in a sample of individuals with a current or past history of problem gambling recruited online.

Methods

Using a two-stage recruitment process, a final sample of 110 participants with a history of problem or disordered gambling were recruited through MTurk and completed self-report questionnaires of gambling involvement symptomology, a Delay Discounting Task (DDT), Balloon Analogue Risk Task (BART), Cued Go/No-Go Task, and the UPPS-P.

Results

Participants demonstrated logically consistent responding on the DDT. The area under the empirical discounting curve (AUC) ranged from 0.02 to 0.88 (M = 0.23). The BART demonstrated good split-third reliability (ρs = 0.67 to 0.78). The tasks generally showed small correlations with each other (ρs = ±0.06 to 0.19) and with UPPS-P subscales (ρs = ±0.01 to 0.20).

Discussion and conclusions

The behavioral tasks demonstrated good divergent validity. Correlation magnitudes between behavioral tasks and UPPS-P scales and mean scores on these measures were generally consistent with the existing literature. Behavioral tasks of impulsivity appear to have utility for use with problem and disordered gambling samples collected online, allowing researchers a cost efficient and rapid avenue for conducting behavioral research with gamblers. We conclude with best-practice recommendations for using behavioral tasks using crowdsourcing samples.

Abstract

Background and aims

To date, no research has examined the viability of using behavioral tasks typical of cognitive and neuropsychology within addiction populations through online recruitment methods. Therefore, we examined the reliability and validity of three behavioral tasks of impulsivity common in addiction research in a sample of individuals with a current or past history of problem gambling recruited online.

Methods

Using a two-stage recruitment process, a final sample of 110 participants with a history of problem or disordered gambling were recruited through MTurk and completed self-report questionnaires of gambling involvement symptomology, a Delay Discounting Task (DDT), Balloon Analogue Risk Task (BART), Cued Go/No-Go Task, and the UPPS-P.

Results

Participants demonstrated logically consistent responding on the DDT. The area under the empirical discounting curve (AUC) ranged from 0.02 to 0.88 (M = 0.23). The BART demonstrated good split-third reliability (ρs = 0.67 to 0.78). The tasks generally showed small correlations with each other (ρs = ±0.06 to 0.19) and with UPPS-P subscales (ρs = ±0.01 to 0.20).

Discussion and conclusions

The behavioral tasks demonstrated good divergent validity. Correlation magnitudes between behavioral tasks and UPPS-P scales and mean scores on these measures were generally consistent with the existing literature. Behavioral tasks of impulsivity appear to have utility for use with problem and disordered gambling samples collected online, allowing researchers a cost efficient and rapid avenue for conducting behavioral research with gamblers. We conclude with best-practice recommendations for using behavioral tasks using crowdsourcing samples.

Introduction

Over the past several years, the use of Amazon’s Mechanical Turk (MTurk; www.mturk.com) for experimental- and survey-based psychological research has surged, including recruitment of clinical samples (Chandler & Shapiro, 2016). MTurk is an online crowdsourcing platform that allows large groups of individuals (MTurkers) to complete tasks for small monetary payment. As the use of MTurk for psychological research has grown in popularity, questions have been raised regarding the reliability of data collected and the validity of inferences made from MTurk samples (Huff & Tingley, 2015; Paolacci & Chandler, 2014). Although the extant literature provides some confidence for the reliability and validity of self-report data on MTurk (Chandler & Shapiro, 2016; Kim & Hodgins, 2017; Mishra & Carleton, 2017), whether this extends to data that require sustained attention (e.g., neuropsychological tasks) remains unknown. In this study, we aimed to address questions regarding the quality of MTurk data using a clinical sample of individuals with current or past problem gambling behavior.

One reason for MTurk’s rising popularity in psychological research is due to the advantages it provides as a recruitment tool. Better-quality data can be collected in less time than alternative methods of recruiting convenience samples (e.g., through Facebook; Shao et al., 2015), and workers can be kept completely anonymous, reducing the risk of experimenter influence on the results (Crump, McDonnell, & Gureckis, 2013). Samples recruited from MTurk also tend to be more demographically diverse than other convenience samples, such as undergraduate students (Chandler & Shapiro, 2016). Specifically, the mean age of MTurkers tends to be higher than undergraduate samples and lower than other adult convenience samples and the racial/ethnicity composition tends to be less non-Hispanic white than undergraduate samples (Berinsky, Huber, & Lenz, 2012). Workers also tend to be younger, less religious, more liberal, and have a higher mean education than the overall population and other non-probability samples (Chandler & Shapiro, 2016).

Furthermore, data collected through MTurk have demonstrated high convergent and concurrent validity (Chandler & Shapiro, 2016), scale reliability (Buhrmester, Kwang, & Gosling, 2011), test–retest reliability (Chandler & Shapiro, 2016; Kim & Hodgins, 2017), and comparable effect sizes to those seen in the existing literature on a variety of psychological measures (Shapiro, Chandler, & Mueller, 2013). In addition, populations that are traditionally difficult to recruit in large numbers or who are underrepresented in traditional recruitment techniques can be reached through MTurk (Blumberg & Luke, 2007; Chandler & Shapiro, 2016). Importantly for this paper, MTurk can be used to recruit populations with substance abuse and gambling addiction (Kim & Hodgins, 2017). However, issues of inattention and inaccurate reporting of symptoms may be particularly concerning among addiction populations, where issues of truthful responding are especially prominent (Godinho, Kushnir, & Cunningham, 2016).

Several studies have directly addressed issues of data quality among addiction populations. Kim and Hodgins (2017) examined the reliability and validity of self-report MTurk data collected from gambling, alcohol, and cannabis users. Overall, data from the gambling and alcohol samples showed good internal consistency, 1-week test–retest reliability, and external validity. Moreover, the reliability indices were like those found in the existing literature, with the exception of the cannabis sample, suggesting caution when collecting data from this population (Kim & Hodgins, 2017). In a related vein, Mishra and Carleton (2017) investigated the usage of crowdsourcing for gambling research. In a series of studies, they demonstrated good convergent validity and adequate test–retest reliability across several measures of problem gambling and general gambling involvement, associations with reports of personality, impulsivity, and behavioral risk-taking with magnitudes consistent with existing literature. Similar to the previous research suggesting higher rates of psychopathology in crowdsourcing samples (Arditte, Çek, Shaw, & Timpano, 2016), Mishra and Carleton (2017) found rates of problematic gambling higher than the general population. On the whole, these studies suggest that although crowdsourcing samples are not representative of the general population, MTurk may be used to efficiently recruit addiction samples and collect high-quality data.

This study extends this examination of the utility of MTurk by investigating its use in behavioral tasks typical of cognitive or neuropsychology within addiction samples. Research using behavioral tasks has consistently shown aberrant impulsive choice (MacKillop et al., 2011), response inhibition (De Wit, 2009), and risk-taking behaviors in addiction populations (De Wit, 2009; MacKillop et al., 2011). The ability to collect large and diverse samples quickly using these tasks would be of benefit to the field. Such tasks typically involve recording of response times, sustained attention, and often complex instructions, which may be problematic for web-based administration, especially among participants recruited through crowdsourcing. That said, several behavioral tasks have been validated with MTurk among non-clinical samples (Crump et al., 2013). However, whether these results translate to clinical samples, including problem gamblers, have yet to been tested.

Overview of the present research

In sum, MTurk is increasingly being used in psychological research with clinical samples (Chandler & Shapiro, 2016; Shapiro et al., 2013) and while issues of data quality have been addressed with regard to self-report measures (Kim & Hodgins, 2017; Mishra & Carleton, 2017), this has not been done with behavioral tasks common to addiction research. Therefore, the aim of the present research was to investigate the reliability and validity of three common behavioral tasks [Delay Discounting Task (DDT), Balloon Analogue Risk Task (BART), and Cued Go/No-Go Task] in individuals with a lifetime history of problem gambling recruited through MTurk. Logical consistency of responding on the DDT and split-third reliability of the BART were examined as indicators of task reliability. Given the multifaceted nature of impulsivity, the UPPS-P Impulsive Behavior Scale was used to provide a measure of convergent and divergent validity. A comparison of the behavioral tasks disordered gambling symptoms provided a measure of external validity. The results of these tasks (including means and standard deviations) and correlations with self-reported impulsivity were compared to previous research to further assess the utility of using behavioral tasks of impulsivity on MTurk with clinical samples.

Methods

Participants

A total of 304 individuals residing in the United States (55% male; Mage = 37.57, SDage = 11.42) participated in this study. Participation was limited to individuals aged 18 years and older who felt that they had experienced a problem with gambling behavior at some point in their life. Of those who gave permission to be recontacted for Part 2 (99%), 268 (89%) were eligible and 116 completed Part 2.

Procedure

The study was hosted through TurkPrime, a platform designed for social science research that integrates with the MTurk platform and recruits participants through MTurk. A recruitment notice was posted on Amazon’s MTurk to recruit our sample of interest, “For this study, we are recruiting people who may feel they currently have a problem with gambling activities or have had a problem with gambling in the past. Examples of gambling would include: card games (e.g., poker, blackjack), slot machines, sports betting (including horse racing), lotteries and other type of Casino games (e.g., roulette).” Participants were further informed about the follow-up study. MTurkers were compensated approximately $0.10 per minute for their participation, a rate that has been suggested as an ethical remuneration for MTurk studies (Chandler & Shapiro, 2016). Consistent with previously employed methods, participation was limited to MTurkers with an approval rating of 95% (i.e., MTurkers who have been approved 95% of the time or greater for the tasks they have previously completed) and duplicate IP addresses were blocked to prevent multiple responding (Kim & Hodgins, 2017).

Participants were redirected to a survey hosted by Qualtrics where they provided informed consent. Given concerns that MTurk workers may misreport symptoms in order to be eligible for studies with explicit inclusion criteria, best research practices recommend a two-stage screening process, and use of subtle measures of reliability (Chandler & Paolacci, 2017; Chandler & Shapiro, 2016). Therefore, to ensure that our sample consisted of our target population, Part 1 included three screening items to assess for lifetime problem gambling behavior [NORC DSM-IV Screen for Gambling Problems questions pertaining to loss of Control, Lying, and Preoccupation (NODS-CLiP); Toce-Gerstein, Gerstein, & Volberg, 2009]. Participants who met the eligibility criteria completed a battery of self-report measures and were then directed to a consent to be recontacted to complete Part 2. Participants who were interested and eligible were invited through TurkPrime to complete Part 2. Participants who demonstrated inconsistent responding, characterized by responses on the NODS-CLiP that differed from those provided on the identical questions presented later were not invited to complete Part 2.

In Part 2, participants were redirected to Inquisit, a study platform hosted by Millisecond (www.millisecond.com) that administers behavioral tasks online. As part of informed consent, they were notified that they would be asked to install the Inquisit launcher, and that they would be directed to a page that provided instructions on how to uninstall the program upon completion. Participants were also given information on how to exit the behavioral tasks if they would like to stop. Then, they completed three behavioral tasks and a questionnaire.

Measures

NODS-CLiP (Toce-Gerstein & Volberg, 2004)

Participants first completed the NODS-CLiP, rapid screen for disordered and problem gambling in adults. The three questions, derived from the National Opinion Research Center Diagnostic Screen (NORC) DSM-IV Screen for Gambling Problems (NODS) pertain to loss of control, lying, and preoccupation diagnostic criteria in the DSM. A threshold of one item endorsed captures 96.2% of NODS problem and disordered gamblers (Toce-Gerstein et al., 2009). Consistent with previous research (e.g., Kim, Wohl, Salmon, & Santesso, 2017; Suurvali, Hodgins, Toneatto, & Cunningham, 2008), participants who endorsed at least one item went on to complete the remaining questionnaires.

Demographics and gambling involvement

Participants completed a questionnaire assessing demographic and gambling involvement in the previous 12 months. Those who had not gambled in the preceding 12 months did not complete the gambling involvement questions.

NORC DSM Screen for Gambling Problems (NODS; Gerstein et al., 1999)

The NODS is a self-report screening measure of both current and lifetime problem gambling based on DSM-IV criteria. Seventeen questions for lifetime and 17 corresponding past-year items scored yes or no and measure 10 disordered gambling criteria. Consistent with DSM-5 criteria (American Psychiatric Association, 2013), the criterion of engaging in illegal activity was excluded from analysis.

Delay Discounting Task (DDT)

According to a modified version of the Richards, Zhang, Mitchell, and de Wit (1999), DDT was used. In this task, participants are asked to make choices between smaller-immediate rewards and larger-later rewards. The task was adjusted to provide suitable variability in possible scores using hypothetical rewards. Consistent with other tasks using hypothetical rewards, the standard delayed reward was increased to $1,000, and seven temporal delays were presented: 1 day, 1 week, 1 month, 3 months, 1 year, 5 years, and 25 years (Koffarnus & Bickel, 2014). The magnitude of the rewards was systematically varied using a random-adjustment algorithm to estimate indifference points for each temporal delay, where the subjective value of the immediate and delayed reward is equal (Myerson, Green, & Warusawitharana, 2001). A random-adjustment task is preferred over procedures with a set order of presentation, because participants are not able to anticipate the next question. The area under the empirical discounting curve (AUC) was used as an indicator of impulsive choice (Myerson et al., 2001). Given significant debate regarding different discounting equations, it has been recommended that the AUC should be used, as it is a theoretically neutral parameter (Madden & Bickel, 2010). Smaller AUCs reflect steeper discounting and greater impulsive choice.

Balloon Analogue Risk Task (BART; Lejuez et al., 2002)

As a measure of risk-taking, participants completed the BART. In this task, participants are presented an image of a deflated balloon, which they are asked to pump up virtually. For each successful pump, they can earn hypothetical money. They are informed that the balloon will explode at some point and that no money will be earned from exploded balloons. Participants are informed that the point at which the balloon pops is random, and pumps can range from 1 to 128 for each individual trial. The task involves 30 trials (i.e., 30 balloons). The average pump count for unexploded balloons is used as an indicator of risk-taking propensity (Lejuez et al., 2002).

Cued Go/No-Go Task (Fillmore, 2003)

As a measure of motor response inhibition, participants completed the Cued Go/No-Go Task. Participants are provided green and blue rectangles presented vertically (125 trials) or horizontally (125 trials). They are told to press the space bar when they see a green rectangle but to refrain from pressing when they see a blue rectangle. The vertical rectangles have a higher probability of being green (80%), and the horizontal rectangles have a high probability of being blue (80%). Participants are given a cue about the orientation of the rectangle (a silhouette) shortly before the color of the rectangle is revealed. As such, response-appropriate expectancies influence behavioral control mechanisms (Vogel-Sprott & Fillmore, 2011). Inhibition errors were used as a measure of motor response inhibition, with higher errors indicating greater trouble with response inhibition. The number of inhibition errors can range from 0 to 200. Due to different response-appropriate expectancies on horizontal and vertical cues, inhibition errors were reported individually for each orientation.

UPPS-P Impulsive Behavior Scale (Lynam, Smith, Whiteside, & Cyders, 2006)

The UPPS-P is a 59-item self-report measure of personality traits that lead to impulsive behaviors. Each item is measured on a 4-point Likert scale from 1 to 4. Total scores can range from 59 to 236, with higher scores indicating greater impulsivity. It assesses five impulsivity subscales; negative urgency measures the tendency to act rashly in response to strong negative emotions (12 items), (lack of) premeditation assesses the ability to think through possible consequences before acting (11 items), (lack of) perseverance assesses the ability to persist in completing tasks (10 items), sensation seeking assesses the preference for excitement and stimulation (12 items), and finally positive urgency measures the tendency to act rashly in response to strong positive emotions (14 items).

Data analytic plan

All analyses were conducted in R (R Core Team, 2008) software. A variety of measures of reliability and validity were calculated. Part 1 completion time was examined as a subtle measure of validity. Outliers were examined using visual plots and Z scores of +3.29 (Cohen, Cohen, West, & Aiken, 2003). To address concerns with symptom misreporting in MTurk samples (Chandler & Paolacci, 2017), we compared NODS-CLiP responses to the identical question presented later in the lifetime NODS. As a subtle measure of reliability, the logical consistency of responding in the DDT was also examined. Split-third reliability (comparing performance on the first, second, and third blocks of balloons) was examined using Spearman’s correlation coefficients. In the absence of an established measure of response consistency in the Go/No-Go Task, only the relationship of this task to other measures and the magnitude of the correlations with UPPS-P scored compared to the preexisting literature were examined.

Spearman’s correlation coefficients were used to examine concurrent and divergent validity of the behavioral tasks of impulsivity. External validity was examined by calculating Spearman’s correlations between the behavioral task measures and NODS scores for both current and lifetime modules. The magnitude of the associations to previous results in the literature was compared.

Ethics

The study procedures were carried out in accordance with Conjoint Faculties Research Ethics Board at the University of Calgary. The institutional review board of the University of Calgary approved the study. All subjects were informed about the study and all provided consent.

Results

Preliminary analysis

Demographic characteristics of our sample for Parts 1 and 2 are included in Table 1. In Part 1, 204 participants reported gambling problems in the past year, among whom 122 met NODS screening criteria for disordered gambling; 244 met lifetime screening criteria for disordered gambling. In Part 2, 45 participants reported gambling problems in the past year, and 73 participants met lifetime screening criteria for disordered gambling. T-tests were conducted to assess demographic differences between eligible participants who completed Part 2 versus those who did not. No significant differences were found, ps ≥.54. In addition, a χ2 test examined whether individuals with a history of disordered gambling (current vs. past) were more or less likely to complete Part 2. The χ2 test was not significant, χ2(1) < 0.001, p = 1.

Table 1.

Demographic information of participants who completed Part 1 and Part 2

Part 1Part 2
N%N%
Gender
 Male16654.605852.73
 Female13343.805045.45
 Other20.6021.82
 Missing30.9000.00
Income
 Under $10,000123.9054.55
 $10,000–$39,0009932.504137.27
 $40,000–$69,0009631.603128.18
 $70,000–$99,0009731.902421.82
Education
 High-school diploma or less6220.392320.91
 Trades or apprenticeship123.9565.45
 College4213.821513.64
 University below bachelors3611.841614.55
 Bachelors11638.163733.64
 Graduate degree3611.841311.82
Ethnicity
 Caucasian22480.308779.09
 South Asian51.7000.00
 Black3411.201110.00
 Filipino/Pacific Islander20.7000.00
 Latin American237.601412.73
 Chinese62.0021.82
 Southeast Asian10.3010.91
 Arab10.3010.91
 Japanese10.3000.00
 Korean20.7010.91
 Aboriginal10.3000.00
Employment
 Full-time24881.588980.91
 Unemployed185.9276.36
 Student82.6310.91
 Part-time258.2287.27
 Retired51.6443.64
 Other72.3032.73

In addition, we compared this sample on gambling-specific variables to previous research. In the present sample, we observed some similarities to community-based samples regarding associations between problematic gambling and frequency and types of gambling involvement. Males reported higher hours engaged in their most frequent activity (M = 39.06) compared to females (M =16.15), consistent with prior research demonstrating that males gamble more frequently than females (Potenza, Maciejewski, & Mazure, 2006; Wong, Zane, Saw, & Chan, 2013). In addition, lottery tickets were the form of gambling most frequently engaged in by this sample, which is consistently reported in population surveys (2–3 times/month; Gerstein et al., 1999; Tu, 2013; Welte, Barnes, Tidwell, Hoffman, & Wieczorek, 2015). However, while video lottery terminal (VLT) engagement is consistently reported as the most problematic form of gambling (Currie et al., 2006; LaPlante, Nelson, LaBrie, & Shaffer, 2011; MacLaren, 2016), casino slots games showed a similarly high association with problem gambling symptomology (r = .40) as that between problem gambling and VLT engagement (r = .37).

Reliable responding

No participants completed the survey unusually quickly compared to other participants, and four took unusually long to complete (0.01%). Eighty participants (26%) showed inconsistency in their endorsement of disordered gambling symptoms, as demonstrated by different scores on at least one NODS-CLiP item and its corresponding NODS item. Most inconsistent responding was characterized by endorsement of items on the NODS-CLiP and not on the NODS. Individuals who consistently responded demonstrated significantly higher lifetime NODS (M = 5.91, t = 10.58, p < .001) and past-year scores (M = 4.48, t = 7.83, p < .001) compared to those who inconsistently responded (Mlifetime =2.88, Mpast-year = 1.75). These latter participants were not invited to complete Part 2, resulting in a total of 93 participants who completed the behavioral tasks.

Logical consistency and task reliability

Delay Discounting Task

The two criteria used to examine logically consistent responding in the DDT have been used in discounting tasks with a standard delay value of $1,000 and 7 delay intervals, as was the case in this study (Koffarnus & Bickel, 2014). First, the indifferent point at the first delay must be at least $100 greater than the indifference point at the longest delay. Second, no more than 1 indifference point can be $200 greater than the point preceding it. These criteria were employed to detect and exclude cases where the value of the delayed reward haphazardly fluctuated across delays, which might indicate that the participant did not understand the task, or that they did not attend to the questions (see Johnson & Bickel, 2008 for a discussion on identifying non-systematic discounting data). Eight participants were removed from analysis due to failure to meet the first criteria. No additional participants failed to meet the second criteria. AUCs ranged from 0.02 to 0.88 (M = 0.232, SD = 0.21; Figure 1).

Figure 1.
Figure 1.

Overall delay discounting curve with 95% confidence intervals (CI) across delays. Area under the curve (AUC) = 0.23, 95% CI of AUC [0.18, 0.28]. Discounting curve created by plotting the average indifference point across the sample. Temporal delays are represented as a proportion of the maximum temporal delay. The shaded area illustrates the 95% CIs across all delay points: 1 day, 95% CI [977.64, 981.42]; 1 week, 95% CI [875.13, 965.34]; 1 month, 95% CI [710.23, 857.77]; 3 months, 95% CI [552.77, 730.53]; 1 year, 95% CI [292.14, 468.80]; 5 years, 95% CI [107.32, 221.15]; and 25 years, 95% CI [73.45, 162.78]

Citation: Journal of Behavioral Addictions 7, 4; 10.1556/2006.7.2018.117

Balloon Analogue Risk Task (BART)

Adjusted average pump count (APC) ranged from 4.17 to 84.11 (M = 53.56, SD = 14.77). Interitem reliability was examined by comparing APC across trials. APCs across blocks of trials demonstrated moderate to high correlations (ρs = 0.67–0.78; Table 2), indicating good interitem reliability.

Table 2.

Spearman’s ρ correlation coefficients for the adjusted average pump count (APC) on the Balloon Analogue Risk Task

Overall APCTrials 1–10Trials 11–20Trials 21–30
Trials 1–100.87
Trials 11–200.920.73
Trials 21–300.900.670.78
M53.5455.8651.0855.62
SD4.7717.1315.9816.24

Note. SD: standard deviation.

Discussion

Research consistently demonstrates differences in various facets of behavioral impulsivity in disordered gambling samples (e.g., Kovacs, Richman, Janka, Maraz, & Ando, 2017). As such, behavioral tasks of impulsivity are frequently used, although their utility with online clinical samples has not been investigated. The purpose of this study was to examine the utility of using three common behavioral tasks with an online sample of individuals with current and lifetime problem and disordered gambling. Overall, this study supports the use of behavioral tasks with gambling samples recruited through MTurk. However, the results of the present research also suggest some caution and the need to act appropriately to minimize the effect of inattention and symptom misreporting.

This study screened for logically consistent responding in the DDT, which is frequently unaddressed in research using discounting tasks. Only eight participants were excluded from analyses due to failing to meet the criteria for logically consistent responding, suggesting that participants tended to respond consistently. Similarly, the study showed strong within task reliability on the BART, indicating that participants responded consistently.

Within the delay discounting literature, we found AUCs ranging between 0.21 and 0.32 among problem and disordered gambling samples (Krmpotich et al., 2015; Ledgerwood, Alessi, Phoenix, & Petry, 2009), similar to our reported mean AUC in this study (0.23). Regarding the BART, previous research has reported an APC of 37–45 in community samples (Hunt, Hopko, Bare, Lejuez, & Robinson, 2005; Krmpotich et al., 2015; White, Lejuez, & de Wit, 2008). Although addiction groups often display greater mean APC than control groups (Lauriola, Panno, Levin, & Lejuez, 2014), mean APCs have been as low as 21.3 in disordered gambling samples (Ledgerwood et al., 2009). However, the APC on the BART was slightly higher than expected compared with previous research. Next, we compared the within-session reliability of the BART. Specifically, previous research has reported strong split-third reliability (>.7; White et al., 2008) across the blocks of trials, which was found in this study (.67–.78). On the whole, the results suggest that participants on MTurk demonstrate sustained attention and respond reliably to behavioral tasks as compared to individuals with disordered gambling recruited through traditional avenues.

Correlation magnitudes observed in this study between Delay Discounting AUC, BART APC, and Go/No-Go inhibition errors to each other and to UPPS-P subscale scores (Table 3). In general, the mean scores on the variables of interest in the behavioral tasks and their correlations with the UPPS-P scale scores were consistent to those seen in previous research. As impulsivity is multifaceted construct, the magnitude of these correlations is within the range that would be expected. The use of behavioral measures has revealed that the facets of impulsivity tapped into by these tasks reflect separate underlying processes in addictive disorders (De Wit, 2009). Therefore, they tend to demonstrate only modest correlations with each other. However, the strongest relationships are observed between measures of response inhibition (e.g., Go/No-Go tasks) and UPPS-P dimensions (r = .10–.13; Cyders & Coskunpinar, 2011). The magnitude of the relationship between inhibition errors on the horizontal target in this study demonstrated similar, if somewhat higher, correlations with UPPS-P scales. Delayed discounting of monetary rewards appear most strongly correlated with lack of planning (r = .13) and positive urgency (.13; Cyders & Coskunpinar, 2011). This study found a similar correlation between AUC and lack of premeditation (r = .16) but was unrelated to positive urgency. The BART correlates most strongly with the sensation-seeking subscale (r = .08–.12; Cyders & Coskunpinar, 2011), consistent with that observed in this study (r = .08).

Table 3.

Spearman’s ρ correlation coefficients between behavioral task parameter, UPPS-P subscale and total scores, past-year NODS scores, and lifetime NODS scores

AUCAPCInhibition errors – vertical targetInhibition errors – horizontal targetUPPS-P total scoreNegative urgency(Lack of) PremeditationPerseverationSensation seekingPositive urgencyNODS past yearNODS lifetime
APC0.161.00
Inhibition errors – vertical target−0.190.081.00
Inhibition errors – horizontal target−0.060.110.501.00
UPPS-P total score0.050.020.000.191.00
Negative urgency−0.01−0.15−0.020.030.821.00
(Lack of) Premeditation0.160.13−0.040.140.710.441.00
(Lack of) Perseveration0.100.03−0.020.150.660.550.581.00
Sensation seeking0.030.080.050.200.660.350.350.161.00
Positive urgency−0.030.020.020.170.880.750.470.490.501.00
NODS past year−0.150.050.040.090.250.260.170.210.130.291.00
NODS lifetime−0.050.140.02−0.050.150.240.080.070.050.160.841.00
M0.2353.540.030.03115.8723.4220.2916.9128.9526.304.385.71
SD0.2214.770.120.1131.188.716.415.248.5510.842.162.11

Note. Bold values indicate correlation is significant at p ≤ .05. Note that an exact p value was not able to be calculated due to rank ties. Therefore, all p values were estimates and should be interpreted with caution. NODS past-year scores included only participants who had gambled in the prior 12 months. SD: standard deviation; AUC: area under the empirical discounting curve for the delay discounting task; APC: adjusted average pump count for the Balloon Analogue Risk Task; NODS: NORC DSM-IV Screen for Gambling Problems.

Finally, the average scores on UPPS-P subscales (Table 3) was similar to that observed in other literature. For individuals with problem and disordered gambling, the highest subscale scores are typically found on sensation seeking, negative urgency, and positive urgency subscales (Clark et al., 2012; Haw, 2017; Michalczuk, Bowden-Jones, Verdejo-Garcia, & Clark, 2011). This same pattern was observed in the present project; individuals showed the highest scores on the sensation-seeking subscale (M = 28.95, SD = 8.55), followed by positive urgency (M = 26.3, SD = 10.84) and negative urgency (M = 23.42, SD = 8.71). Mean lack of premeditation was slightly lower (M = 20.29, SD = 6.41), and lack of perseveration had the lowest score (M = 16.91, SD = 5.24). Moreover, these scores were within the range of scores observed in previous research with problem and disordered gambling populations (Haw, 2017; Michalczuk et al., 2011). For example, Haw’s (2017) reported mean UPPS-P subscale scores for men and women are similar to the scores we observed [Ms = 15.61 (SD = 5.50; perseverance) to 28.78 (SD = 8.98; sensation seeking)]. The sample in the study of Michalczuk et al. (2011) was composed of individuals with disordered gambling and average UPPS-P subscale scores were higher than we observed, as we would expect. However, sensation seeking (M = 33.4, SD = 5.9), positive urgency (M = 35.9, SD = 8.3), and negative urgency (M = 35.6, SD = 6.4) continued to demonstrate the highest scores, with lack of premeditation (M = 26.6, SD = 5.9) and lack of perseverance (M = 23.0, SD = 5.6) showing the lowest. Altogether, the measures of behavioral impulsivity showed associations with each other and self-report impulsivity that are consistent with research conducted in lab settings.

Limitations

Although the study found promising findings for the utility of these behavioral assays with online clinical samples, several limitations should be considered. First, the statement of inclusion criteria in the recruitment notice may have encouraged symptom overreporting. As has been noted by other researchers (Kim & Hodgins, 2017), it is difficult (or impossible) to verify self-report data obtained online. The high proportion of inconsistent symptom responding highlights concerns with careless responding or symptom misreporting. Most of these inconsistencies arose from endorsement of symptoms on the NODS-CLiP screening items, followed by lack of endorsement on the identical NODS item later in the questionnaire. Participants were notified in the recruitment notice of the target population and therefore they may have been primed to endorse gambling symptoms in the screening items to quality for the study. Chandler and Paolacci (2017) recently demonstrated that many participants failed to meet eligibility criteria for MTurk studies and recommended that multiple-step screening processes be used to exclude ineligible participants. Thus, the two-step process of recruitment and screening used in this study allowed us to assess for inconsistent responding and exclude these participants, lending confidence to our results. In addition, this study also highlights the importance of multistep screening when it is not possible or ethical to completely blind participants to the purpose of the research. Second, this study did not include a group of individuals who completed the questionnaires and tasks in-person. Therefore, we cannot speak to the influence of method variance on the results we observed. However, the data obtained in this study generally showed similar psychometric properties when compared to the existing literature, lending some confidence to the validity of our findings. Third, these tasks lack a clear gold standard with which to evaluate responses. A consistent concern raised is their poor correlations with trait measures of impulsivity (Sharma, Markon, & Clark, 2014), highlighting potential issues with their construct validity. We recognize that lack of such a gold standard is a significant limitation in any research using behavioral tasks. However, impulsivity is increasingly recognized as multifaceted construct, with behavioral tasks tapping in to distinct facets compared to self-report measures of “trait” impulsivity (De Wit, 2009). Thus, the relationships observed between the UPPS-P and behavioral tasks were consistent with previous literature and our current understanding of impulsivity. In consideration of this limitation, only widely used behavioral tasks were selected, which generally show high reliability. In addition, the tasks examined in this study have demonstrated predictive validity on a variety of important outcomes, including future drug use (De Wit, 2009; Perry & Carroll, 2008), problem gambling, (Verdejo-García, Lawrence, & Clark, 2008; Vitaro, Arseneault, & Tremblay, 1999), and substance-use treatment outcomes (Stevens et al., 2014). Finally, MTurkers are a non-representative sample. In the present sample, we observed differences, as well as similarities, on associations between gambling-related variables compared to those seen in community-based samples. Of note, although VLT and casino slots are considered to be measures of separate types of gambling, they share significant similarities in play. This may contribute to the similarly high associations with problem gambling symptomology that we observed. Overall, while MTurk samples are not representative of the general gambling population, it appears to be a useful tool for recruiting samples characterized by problematic gambling.

Recommendations

Given our findings, we make the following recommendations for obtaining quality data from gambling samples on MTurk:

  1. 1.Researchers should consider multistep screening when recruiting clinical populations and examine the consistency of symptom reporting specifically.
  2. 2.Analyses should include examination of response consistency on reward-related decision-making tasks (e.g., probabilistic or delayed reward discounting).
  3. 3.Studies utilizing behavioral tasks that have not been directly examined in an online clinical population should pilot the tasks and recruitment strategy.
  4. 4.Continue to uphold best research practices with MTurk. Such recommendations can be found in the study of Chandler and Shapiro (2016) or Chandler and Paolacci (2017).
  5. 5.Be cognizant that MTurkers are non-representative samples. In regard to gambling characteristic, the proportion of problem gamblers tends to be greater than that of traditional avenues (Kim & Hodgins, 2017).

Conclusions

Overall, the random-adjusting DDT and Cued Go/No-Go Tasks appear to be viable for use with addiction samples collected through MTurk. The BART that may be more susceptible to inattention and caution is warranted when using hypothetical rewards. Further research should investigate the potential impact of real versus hypothetical rewards with online clinical samples before adopting it for web-based use. In conclusion, this is the first study to highlight the potential of using behavioral tasks with clinical MTurk samples. As we continue to push the boundaries of knowledge and how we study addiction, the use of behavioral tasks through MTurk might be a promising avenue to advance this goal.

Authors’ contribution

MGS: study concept and design, analysis and interpretation of data, and statistical analysis. HSK: study concept and design and analysis and interpretation of data. DCH: study concept and design and study supervision. All authors had full access to all data and take responsibility for the integrity of the data and the accuracy of the data analysis.

Conflict of interest

The authors declare no conflict of interest.

References

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.

    • Search Google Scholar
    • Export Citation
  • Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28(6), 684. doi:10.1037/pas0000217

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon. com’s Mechanical Turk. Political Analysis, 20(3), 351368. doi:10.1093/pan/mpr057

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blumberg, S. J., & Luke, J. V. (2007). Coverage bias in traditional telephone surveys of low-income and young adults. Public Opinion Quarterly, 71(5), 734749. doi:10.1093/poq/nfm047

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 35. doi:10.1177/1745691610393980

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chandler, J., & Paolacci, G. (2017). Lie for a Dime: When most prescreening responses are honest but most study participants are impostors. Social Psychological and Personality Science, 8(5), 500508. doi:10.1177/1948550617698203

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12(1), 5381. doi:10.1146/annurev-clinpsy-021815-093623

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clark, L., Stokes, P. R., Wu, K., Michalczuk, R., Benecke, A., Watson, B. J., Egerton, A., Piccini, P., Nutt, D. J., Bowden-Jones, H., & Lingford-Hughes, A. R. (2012). Striatal dopamine D2/D3 receptor binding in pathological gambling is correlated with mood-related impulsivity. Neuroimage, 63(1), 4046. doi:10.1016/j.neuroimage.2012.06.067

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple correlation/regression analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum Associates.

    • Search Google Scholar
    • Export Citation
  • Crump, M. J., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One, 8(3), e57410. doi:10.1371/journal.pone.0057410

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Currie, S. R., Hodgins, D. C., Wang, J., El-Guebaly, N., Wynne, H., & Chen, S. (2006). Risk of harm among gamblers in the general population as a function of level of participation in gambling activities. Addiction, 101(4), 570580. doi:10.1111/j.1360-0443.2006.01392.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cyders, M. A., & Coskunpinar, A. (2011). Measurement of constructs using self-report and behavioral lab tasks: Is there overlap in nomothetic span and construct representation for impulsivity? Clinical Psychology Review, 31(6), 965982. doi:10.1016/j.cpr.2011.06.001

    • Crossref
    • Search Google Scholar
    • Export Citation
  • De Wit, H. (2009). Impulsivity as a determinant and consequence of drug use: A review of underlying processes. Addiction Biology, 14(1), 2231. doi:10.1111/j.1369-1600.2008.00129.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fillmore, M. T. (2003). Drug abuse as a problem of impaired control: Current approaches and findings. Behavioral and Cognitive Neuroscience Reviews, 2(3), 179197. doi:10.1177/1534582303257007

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gerstein, D., Volberg, R. A., Toce, M. T., Harwood, H., Johnson, R. A., Buie, T., Christiansen, E., Chuchro, L., Cummings, W., Engelman, L., & Hill, M. A. (1999). Gambling impact and behavior study: Report to the national gambling impact study commission. Chicago, IL: National Opinion Research Center.

    • Search Google Scholar
    • Export Citation
  • Godinho, A., Kushnir, V., & Cunningham, J. A. (2016). Unfaithful findings: Identifying careless responding in addictions research. Addiction, 111(6), 955956. doi:10.1111/add.13221

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haw, J. (2017). Impulsivity predictors of problem gambling and impaired control. International Journal of Mental Health and Addiction, 15(1), 154165. doi:10.1007/s11469-015-9603-9

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huff, C., & Tingley, D. (2015). Who are these people? Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Research & Politics, 2(3), 112. doi:10.1177/2053168015604648

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hunt, M. K., Hopko, D. R., Bare, R., Lejuez, C., & Robinson, E. (2005). Construct validity of the Balloon Analog Risk Task (BART) associations with psychopathy and impulsivity. Assessment, 12(4), 416428. doi:10.1177/1073191105278740

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, M. W., & Bickel, W. K. (2008). An algorithm for identifying nonsystematic delay-discounting data. Experimental and Clinical Psychopharmacology, 16(3), 264274. doi:10.1037/1064-1297.16.3.264

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kim, H. S., & Hodgins, D. C. (2017). Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon’s Mechanical Turk. Psychology of Addictive Behaviors, 31(1), 8594. doi:10.1037/adb0000219

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kim, H. S., Wohl, M. J. A., Salmon, M., & Santesso, D. (2017). When do gamblers help themselves? Self-discontinuity increases self-directed change over time. Addictive Behaviors, 64, 148153. doi:10.1016/j.addbeh.2016.08.037

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koffarnus, M. N., & Bickel, W. K. (2014). A 5-trial adjusting delay discounting task: Accurate discount rates in less than one minute. Experimental and Clinical Psychopharmacology, 22(3), 222228. doi:10.1037/a0035973

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kovacs, I., Richman, M. J., Janka, Z., Maraz, A., & Ando, B. (2017). Decision making measured by the Iowa Gambling Task in alcohol use disorder and gambling disorder: A systematic review and meta-analysis. Drug Alcohol Depend, 181, 152161. doi:10.1016/j.drugalcdep.2017.09.023

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krmpotich, T., Mikulich-Gilbertson, S., Sakai, J., Thompson, L., Banich, M. T., & Tanabe, J. (2015). Impaired decision-making, higher impulsivity, and drug severity in substance dependence and pathological gambling. Journal of Addiction Medicine, 9(4), 273280. doi:10.1097/ADM.0000000000000129

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LaPlante, D. A., Nelson, S. E., LaBrie, R. A., & Shaffer, H. J. (2011). Disordered gambling, type of gambling and gambling involvement in the British Gambling Prevalence Survey 2007. European Journal of Public Health, 21(4), 532537. doi:10.1093/eurpub/ckp177

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauriola, M., Panno, A., Levin, I. P., & Lejuez, C. W. (2014). Individual differences in risky decision making: A meta-analysis of sensation seeking and impulsivity with the Balloon Analogue Risk Task. Journal of Behavioral Decision Making, 27(1), 2036. doi:10.1002/bdm.1784

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ledgerwood, D. M., Alessi, S. M., Phoenix, N., & Petry, N. M. (2009). Behavioral assessment of impulsivity in pathological gamblers with and without substance use disorder histories versus healthy controls. Drug and Alcohol Dependence, 105(1), 8996. doi:10.1016/j.drugalcdep.2009.06.011

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D. R., & Brown, R. A. (2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8(2), 75. doi:10.1037//1076-898X.8.2.75

    • Search Google Scholar
    • Export Citation
  • Lynam, D. R., Smith, G. T., Whiteside, S. P., & Cyders, M. A. (2006). The UPPS-P: Assessing five personality pathways to impulsive behavior. West Lafayette, IN: Purdue University.

    • Search Google Scholar
    • Export Citation
  • MacKillop, J., Amlung, M. T., Few, L. R., Ray, L. A., Sweet, L. H., & Munafò, M. R. (2011). Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology, 216(3), 305321. doi:10.1007/s00213-011-2229-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • MacLaren, V. V. (2016). Video lottery is the most harmful form of gambling in Canada. Journal of Gambling Studies, 32(2), 459485. doi:10.1007/s10899-015-9560-z

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Madden, G. J., & Bickel, W. K. (2010). Introduction. In G. J. Madden & W. K. Bickel (Eds.), Impulsivity: The behavioral and neurological science of discounting (pp. 38). Washington, DC: American Psychological Association.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michalczuk, R., Bowden-Jones, H., Verdejo-Garcia, A., & Clark, L. (2011). Impulsivity and cognitive distortions in pathological gamblers attending the UK National Problem Gambling Clinic: A preliminary report. Psychological Medicine, 41(12), 26252635. doi:10.1017/S003329171100095X

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mishra, S., & Carleton, R. N. (2017). Use of online crowdsourcing platforms for gambling research. International Gambling Studies, 17(1), 125143. doi:10.1080/14459795.2017.1284250

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76(2), 235243. doi:10.1901/jeab.2001.76-235

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184188. doi:10.1177/0963721414531598

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Perry, J. L., & Carroll, M. E. (2008). The role of impulsive behavior in drug abuse. Psychopharmacology, 200(1), 126. doi:10.1007/s00213-008-1173-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Potenza, M. N., Maciejewski, P. K., & Mazure, C. M. (2006). A gender-based examination of past-year recreational gamblers. Journal of Gambling Studies, 22(1), 4164. doi:10.1007/s10899-005-9002-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • R Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org

    • Search Google Scholar
    • Export Citation
  • Richards, J. B., Zhang, L., Mitchell, S. H., & de Wit, H. (1999). Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. Journal of the Experimental Analysis of Behavior, 71(2), 121143. doi:10.1901/jeab.1999.71-121

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shao, W., Guan, W., Clark, M. A., Liu, T., Santelices, C., Cortes, D. E., & Merchant, R. C. (2015). Variations in recruitment yield, costs, speed and participant diversity across internet platforms in a global study examining the efficacy of an HIV/AIDS and HIV testing animated and live-action video among English-or Spanish-speaking Internet or social media users. Digital Culture & Education, 7(1), 4086. Retrieved from http://europepmc.org/abstract/MED/27330570

    • Search Google Scholar
    • Export Citation
  • Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1(2), 213220. doi:10.1177/2167702612469015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sharma, L., Markon, K. E., & Clark, L. A. (2014). Toward a theory of distinct types of impulsive behaviors: A meta-analysis of self-report and behavioral measures. Psychological Bulletin, 140(2), 374408. doi:10.1037/a0034418

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stevens, L., Verdejo-García, A., Goudriaan, A. E., Roeyers, H., Dom, G., & Vanderplasschen, W. (2014). Impulsivity as a vulnerability factor for poor addiction treatment outcomes: A review of neurocognitive findings among individuals with substance use disorders. Journal of Substance Abuse Treatment, 47(1), 5872. doi:10.1016/j.jsat.2014.01.008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Suurvali, H., Hodgins, D., Toneatto, T., & Cunningham, J. (2008). Treatment seeking among Ontario problem gamblers: Results of a population survey. Psychiatric Services, 59(11), 13431346. doi:10.1176/ps.2008.59.11.1343

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toce-Gerstein, M., Gerstein, D., & Volberg, R. (2009). The NODS-CLiP: A rapid screen for adult pathological and problem gambling. Journal of Gambling Studies, 25(4), 541555. doi:10.1007/s10899-009-9135-y

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toce-Gerstein, M., & Volberg, R. (2004). The NODS-CLiP: A new brief screen for pathological gambling. Paper presented at the International Symposium on Problem Gambling and Co-Occurring Disorders, Mystic, CT.

    • Search Google Scholar
    • Export Citation
  • Tu, D. (2013). New Zealanders’ participation in gambling: Results from the 2012 health and lifestyles survey. Wellington, New Zealand: Health Promotion Agency.

    • Search Google Scholar
    • Export Citation
  • Verdejo-García, A., Lawrence, A. J., & Clark, L. (2008). Impulsivity as a vulnerability marker for substance-use disorders: Review of findings from high-risk research, problem gamblers and genetic association studies. Neuroscience and Biobehavioral Reviews, 32(4), 777810. doi:10.1016/j.neubiorev.2007.11.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vitaro, F., Arseneault, L., & Tremblay, R. E. (1999). Impulsivity predicts problem gambling in low SES adolescent males. Addiction, 94(4), 565575. doi:10.1046/j.1360-0443.1999.94456511.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vogel-Sprott, M., & Fillmore, M. T. (2011). Learning, expectancy, and behavioral control implications for drug abuse. In T. R. Schachtman & S. S. Reilly (Eds.), Associative learning and conditioning theory: Human and non-human applications (pp. 213234). New York, NY: Oxford University Press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Welte, J. W., Barnes, G. M., Tidwell, M.-C. O., Hoffman, J. H., & Wieczorek, W. F. (2015). Gambling and problem gambling in the United States: Changes between 1999 and 2013. Journal of Gambling Studies, 31(3), 695715. doi:10.1007/s10899-014-9471-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, T. L., Lejuez, C. W., & de Wit, H. (2008). Test-retest characteristics of the Balloon Analogue Risk Task (BART). Experimental and Clinical Psychopharmacology, 16(6), 565570. doi:10.1037/a0014083

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wong, G., Zane, N., Saw, A., & Chan, A. K. K. (2013). Examining gender differences for gambling engagement and gambling problems among emerging adults. Journal of gambling studies, 29(2), 171189. doi:10.1007/s10899-012-9305-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.

    • Search Google Scholar
    • Export Citation
  • Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28(6), 684. doi:10.1037/pas0000217

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon. com’s Mechanical Turk. Political Analysis, 20(3), 351368. doi:10.1093/pan/mpr057

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blumberg, S. J., & Luke, J. V. (2007). Coverage bias in traditional telephone surveys of low-income and young adults. Public Opinion Quarterly, 71(5), 734749. doi:10.1093/poq/nfm047

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 35. doi:10.1177/1745691610393980

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chandler, J., & Paolacci, G. (2017). Lie for a Dime: When most prescreening responses are honest but most study participants are impostors. Social Psychological and Personality Science, 8(5), 500508. doi:10.1177/1948550617698203

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12(1), 5381. doi:10.1146/annurev-clinpsy-021815-093623

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clark, L., Stokes, P. R., Wu, K., Michalczuk, R., Benecke, A., Watson, B. J., Egerton, A., Piccini, P., Nutt, D. J., Bowden-Jones, H., & Lingford-Hughes, A. R. (2012). Striatal dopamine D2/D3 receptor binding in pathological gambling is correlated with mood-related impulsivity. Neuroimage, 63(1), 4046. doi:10.1016/j.neuroimage.2012.06.067

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple correlation/regression analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum Associates.

    • Search Google Scholar
    • Export Citation
  • Crump, M. J., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One, 8(3), e57410. doi:10.1371/journal.pone.0057410

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Currie, S. R., Hodgins, D. C., Wang, J., El-Guebaly, N., Wynne, H., & Chen, S. (2006). Risk of harm among gamblers in the general population as a function of level of participation in gambling activities. Addiction, 101(4), 570580. doi:10.1111/j.1360-0443.2006.01392.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cyders, M. A., & Coskunpinar, A. (2011). Measurement of constructs using self-report and behavioral lab tasks: Is there overlap in nomothetic span and construct representation for impulsivity? Clinical Psychology Review, 31(6), 965982. doi:10.1016/j.cpr.2011.06.001

    • Crossref
    • Search Google Scholar
    • Export Citation
  • De Wit, H. (2009). Impulsivity as a determinant and consequence of drug use: A review of underlying processes. Addiction Biology, 14(1), 2231. doi:10.1111/j.1369-1600.2008.00129.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fillmore, M. T. (2003). Drug abuse as a problem of impaired control: Current approaches and findings. Behavioral and Cognitive Neuroscience Reviews, 2(3), 179197. doi:10.1177/1534582303257007

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gerstein, D., Volberg, R. A., Toce, M. T., Harwood, H., Johnson, R. A., Buie, T., Christiansen, E., Chuchro, L., Cummings, W., Engelman, L., & Hill, M. A. (1999). Gambling impact and behavior study: Report to the national gambling impact study commission. Chicago, IL: National Opinion Research Center.

    • Search Google Scholar
    • Export Citation
  • Godinho, A., Kushnir, V., & Cunningham, J. A. (2016). Unfaithful findings: Identifying careless responding in addictions research. Addiction, 111(6), 955956. doi:10.1111/add.13221

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Haw, J. (2017). Impulsivity predictors of problem gambling and impaired control. International Journal of Mental Health and Addiction, 15(1), 154165. doi:10.1007/s11469-015-9603-9

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Huff, C., & Tingley, D. (2015). Who are these people? Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Research & Politics, 2(3), 112. doi:10.1177/2053168015604648

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hunt, M. K., Hopko, D. R., Bare, R., Lejuez, C., & Robinson, E. (2005). Construct validity of the Balloon Analog Risk Task (BART) associations with psychopathy and impulsivity. Assessment, 12(4), 416428. doi:10.1177/1073191105278740

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Johnson, M. W., & Bickel, W. K. (2008). An algorithm for identifying nonsystematic delay-discounting data. Experimental and Clinical Psychopharmacology, 16(3), 264274. doi:10.1037/1064-1297.16.3.264

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kim, H. S., & Hodgins, D. C. (2017). Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon’s Mechanical Turk. Psychology of Addictive Behaviors, 31(1), 8594. doi:10.1037/adb0000219

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kim, H. S., Wohl, M. J. A., Salmon, M., & Santesso, D. (2017). When do gamblers help themselves? Self-discontinuity increases self-directed change over time. Addictive Behaviors, 64, 148153. doi:10.1016/j.addbeh.2016.08.037

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koffarnus, M. N., & Bickel, W. K. (2014). A 5-trial adjusting delay discounting task: Accurate discount rates in less than one minute. Experimental and Clinical Psychopharmacology, 22(3), 222228. doi:10.1037/a0035973

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kovacs, I., Richman, M. J., Janka, Z., Maraz, A., & Ando, B. (2017). Decision making measured by the Iowa Gambling Task in alcohol use disorder and gambling disorder: A systematic review and meta-analysis. Drug Alcohol Depend, 181, 152161. doi:10.1016/j.drugalcdep.2017.09.023

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krmpotich, T., Mikulich-Gilbertson, S., Sakai, J., Thompson, L., Banich, M. T., & Tanabe, J. (2015). Impaired decision-making, higher impulsivity, and drug severity in substance dependence and pathological gambling. Journal of Addiction Medicine, 9(4), 273280. doi:10.1097/ADM.0000000000000129

    • Crossref
    • Search Google Scholar
    • Export Citation
  • LaPlante, D. A., Nelson, S. E., LaBrie, R. A., & Shaffer, H. J. (2011). Disordered gambling, type of gambling and gambling involvement in the British Gambling Prevalence Survey 2007. European Journal of Public Health, 21(4), 532537. doi:10.1093/eurpub/ckp177

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauriola, M., Panno, A., Levin, I. P., & Lejuez, C. W. (2014). Individual differences in risky decision making: A meta-analysis of sensation seeking and impulsivity with the Balloon Analogue Risk Task. Journal of Behavioral Decision Making, 27(1), 2036. doi:10.1002/bdm.1784

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ledgerwood, D. M., Alessi, S. M., Phoenix, N., & Petry, N. M. (2009). Behavioral assessment of impulsivity in pathological gamblers with and without substance use disorder histories versus healthy controls. Drug and Alcohol Dependence, 105(1), 8996. doi:10.1016/j.drugalcdep.2009.06.011

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D. R., & Brown, R. A. (2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8(2), 75. doi:10.1037//1076-898X.8.2.75

    • Search Google Scholar
    • Export Citation
  • Lynam, D. R., Smith, G. T., Whiteside, S. P., & Cyders, M. A. (2006). The UPPS-P: Assessing five personality pathways to impulsive behavior. West Lafayette, IN: Purdue University.

    • Search Google Scholar
    • Export Citation
  • MacKillop, J., Amlung, M. T., Few, L. R., Ray, L. A., Sweet, L. H., & Munafò, M. R. (2011). Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology, 216(3), 305321. doi:10.1007/s00213-011-2229-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • MacLaren, V. V. (2016). Video lottery is the most harmful form of gambling in Canada. Journal of Gambling Studies, 32(2), 459485. doi:10.1007/s10899-015-9560-z

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Madden, G. J., & Bickel, W. K. (2010). Introduction. In G. J. Madden & W. K. Bickel (Eds.), Impulsivity: The behavioral and neurological science of discounting (pp. 38). Washington, DC: American Psychological Association.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Michalczuk, R., Bowden-Jones, H., Verdejo-Garcia, A., & Clark, L. (2011). Impulsivity and cognitive distortions in pathological gamblers attending the UK National Problem Gambling Clinic: A preliminary report. Psychological Medicine, 41(12), 26252635. doi:10.1017/S003329171100095X

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mishra, S., & Carleton, R. N. (2017). Use of online crowdsourcing platforms for gambling research. International Gambling Studies, 17(1), 125143. doi:10.1080/14459795.2017.1284250

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76(2), 235243. doi:10.1901/jeab.2001.76-235

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184188. doi:10.1177/0963721414531598

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Perry, J. L., & Carroll, M. E. (2008). The role of impulsive behavior in drug abuse. Psychopharmacology, 200(1), 126. doi:10.1007/s00213-008-1173-0

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Potenza, M. N., Maciejewski, P. K., & Mazure, C. M. (2006). A gender-based examination of past-year recreational gamblers. Journal of Gambling Studies, 22(1), 4164. doi:10.1007/s10899-005-9002-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • R Core Team. (2008). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org

    • Search Google Scholar
    • Export Citation
  • Richards, J. B., Zhang, L., Mitchell, S. H., & de Wit, H. (1999). Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. Journal of the Experimental Analysis of Behavior, 71(2), 121143. doi:10.1901/jeab.1999.71-121

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shao, W., Guan, W., Clark, M. A., Liu, T., Santelices, C., Cortes, D. E., & Merchant, R. C. (2015). Variations in recruitment yield, costs, speed and participant diversity across internet platforms in a global study examining the efficacy of an HIV/AIDS and HIV testing animated and live-action video among English-or Spanish-speaking Internet or social media users. Digital Culture & Education, 7(1), 4086. Retrieved from http://europepmc.org/abstract/MED/27330570

    • Search Google Scholar
    • Export Citation
  • Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1(2), 213220. doi:10.1177/2167702612469015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sharma, L., Markon, K. E., & Clark, L. A. (2014). Toward a theory of distinct types of impulsive behaviors: A meta-analysis of self-report and behavioral measures. Psychological Bulletin, 140(2), 374408. doi:10.1037/a0034418

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stevens, L., Verdejo-García, A., Goudriaan, A. E., Roeyers, H., Dom, G., & Vanderplasschen, W. (2014). Impulsivity as a vulnerability factor for poor addiction treatment outcomes: A review of neurocognitive findings among individuals with substance use disorders. Journal of Substance Abuse Treatment, 47(1), 5872. doi:10.1016/j.jsat.2014.01.008

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Suurvali, H., Hodgins, D., Toneatto, T., & Cunningham, J. (2008). Treatment seeking among Ontario problem gamblers: Results of a population survey. Psychiatric Services, 59(11), 13431346. doi:10.1176/ps.2008.59.11.1343

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toce-Gerstein, M., Gerstein, D., & Volberg, R. (2009). The NODS-CLiP: A rapid screen for adult pathological and problem gambling. Journal of Gambling Studies, 25(4), 541555. doi:10.1007/s10899-009-9135-y

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Toce-Gerstein, M., & Volberg, R. (2004). The NODS-CLiP: A new brief screen for pathological gambling. Paper presented at the International Symposium on Problem Gambling and Co-Occurring Disorders, Mystic, CT.

    • Search Google Scholar
    • Export Citation
  • Tu, D. (2013). New Zealanders’ participation in gambling: Results from the 2012 health and lifestyles survey. Wellington, New Zealand: Health Promotion Agency.

    • Search Google Scholar
    • Export Citation
  • Verdejo-García, A., Lawrence, A. J., & Clark, L. (2008). Impulsivity as a vulnerability marker for substance-use disorders: Review of findings from high-risk research, problem gamblers and genetic association studies. Neuroscience and Biobehavioral Reviews, 32(4), 777810. doi:10.1016/j.neubiorev.2007.11.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vitaro, F., Arseneault, L., & Tremblay, R. E. (1999). Impulsivity predicts problem gambling in low SES adolescent males. Addiction, 94(4), 565575. doi:10.1046/j.1360-0443.1999.94456511.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vogel-Sprott, M., & Fillmore, M. T. (2011). Learning, expectancy, and behavioral control implications for drug abuse. In T. R. Schachtman & S. S. Reilly (Eds.), Associative learning and conditioning theory: Human and non-human applications (pp. 213234). New York, NY: Oxford University Press.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Welte, J. W., Barnes, G. M., Tidwell, M.-C. O., Hoffman, J. H., & Wieczorek, W. F. (2015). Gambling and problem gambling in the United States: Changes between 1999 and 2013. Journal of Gambling Studies, 31(3), 695715. doi:10.1007/s10899-014-9471-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • White, T. L., Lejuez, C. W., & de Wit, H. (2008). Test-retest characteristics of the Balloon Analogue Risk Task (BART). Experimental and Clinical Psychopharmacology, 16(6), 565570. doi:10.1037/a0014083

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wong, G., Zane, N., Saw, A., & Chan, A. K. K. (2013). Examining gender differences for gambling engagement and gambling problems among emerging adults. Journal of gambling studies, 29(2), 171189. doi:10.1007/s10899-012-9305-1

    • Crossref
    • Search Google Scholar
    • Export Citation
The author instruction is available in PDF.
Please, download the file from HERE

Dr. Zsolt Demetrovics
Institute of Psychology, ELTE Eötvös Loránd University
Address: Izabella u. 46. H-1064 Budapest, Hungary
Phone: +36-1-461-2681
E-mail: jba@ppk.elte.hu

Indexing and Abstracting Services:

  • Web of Science [Science Citation Index Expanded (also known as SciSearch®)
  • Journal Citation Reports/Science Edition
  • Social Sciences Citation Index®
  • Journal Citation Reports/ Social Sciences Edition
  • Current Contents®/Social and Behavioral Sciences
  • EBSCO
  • GoogleScholar
  • PsycINFO
  • PubMed Central
  • SCOPUS
  • Medline
  • CABI
2020  
Total Cites 4024
WoS
Journal
Impact Factor
6,756
Rank by Psychiatry (SSCI) 12/143 (Q1)
Impact Factor Psychiatry 19/156 (Q1)
Impact Factor 6,052
without
Journal Self Cites
5 Year 8,735
Impact Factor
Journal  1,48
Citation Indicator  
Rank by Journal  Psychiatry 24/250 (Q1)
Citation Indicator   
Citable 86
Items
Total 74
Articles
Total 12
Reviews
Scimago 47
H-index
Scimago 2,265
Journal Rank
Scimago Clinical Psychology Q1
Quartile Score Psychiatry and Mental Health Q1
  Medicine (miscellaneous) Q1
Scopus 3593/367=9,8
Scite Score  
Scopus Clinical Psychology 7/283 (Q1)
Scite Score Rank Psychiatry and Mental Health 22/502 (Q1)
Scopus 2,026
SNIP  
Days from  38
submission  
to 1st decision  
Days from  37
acceptance  
to publication  
Acceptance 31%
Rate  

2019  
Total Cites
WoS
2 184
Impact Factor 5,143
Impact Factor
without
Journal Self Cites
4,346
5 Year
Impact Factor
5,758
Immediacy
Index
0,587
Citable
Items
75
Total
Articles
67
Total
Reviews
8
Cited
Half-Life
3,3
Citing
Half-Life
6,8
Eigenfactor
Score
0,00597
Article Influence
Score
1,447
% Articles
in
Citable Items
89,33
Normalized
Eigenfactor
0,7294
Average
IF
Percentile
87,923
Scimago
H-index
37
Scimago
Journal Rank
1,767
Scopus
Scite Score
2540/376=6,8
Scopus
Scite Score Rank
Cllinical Psychology 16/275 (Q1)
Medicine (miscellenous) 31/219 (Q1)
Psychiatry and Mental Health 47/506 (Q1)
Scopus
SNIP
1,441
Acceptance
Rate
32%

 

Journal of Behavioral Addictions
Publication Model Gold Open Access
Submission Fee none
Article Processing Charge 850 EUR/article
Printed Color Illustrations 40 EUR (or 10 000 HUF) + VAT / piece
Regional discounts on country of the funding agency World Bank Lower-middle-income economies: 50%
World Bank Low-income economies: 100%
Further Discounts Editorial Board / Advisory Board members: 50%
Corresponding authors, affiliated to an EISZ member institution subscribing to the journal package of Akadémiai Kiadó: 100%
Subscription Information Gold Open Access
Purchase per Title  

Journal of Behavioral Addictions
Language English
Size A4
Year of
Foundation
2011
Publication
Programme
2021 Volume 10
Volumes
per Year
1
Issues
per Year
4
Founder Eötvös Loránd Tudományegyetem
Founder's
Address
H-1053 Budapest, Hungary Egyetem tér 1-3.
Publisher Akadémiai Kiadó
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 2062-5871 (Print)
ISSN 2063-5303 (Online)

Senior editors

Editor(s)-in-Chief: Zsolt DEMETROVICS

Assistant Editor(s): Csilla ÁGOSTON

Associate Editors

  • Judit BALÁZS (ELTE Eötvös Loránd University, Hungary)
  • Joel BILLIEUX (University of Lausanne, Switzerland)
  • Matthias BRAND (University of Duisburg-Essen, Germany)
  • Anneke GOUDRIAAN (University of Amsterdam, The Netherlands)
  • Daniel KING (Flinders University, Australia)
  • Ludwig KRAUS (IFT Institute for Therapy Research, Germany)
  • H. N. Alexander LOGEMANN (ELTE Eötvös Loránd University, Hungary)
  • Anikó MARÁZ (Humboldt University of Berlin, Germany)
  • Astrid MÜLLER (Hannover Medical School, Germany)
  • Marc N. POTENZA (Yale University, USA)
  • Hans-Jurgen RUMPF (University of Lübeck, Germany)
  • Attila SZABÓ (ELTE Eötvös Loránd University, Hungary)
  • Róbert URBÁN (ELTE Eötvös Loránd University, Hungary)
  • Aviv M. WEINSTEIN (Ariel University, Israel)

Editorial Board

  • Max W. ABBOTT (Auckland University of Technology, New Zealand)
  • Elias N. ABOUJAOUDE (Stanford University School of Medicine, USA)
  • Hojjat ADELI (Ohio State University, USA)
  • Alex BALDACCHINO (University of Dundee, United Kingdom)
  • Alex BLASZCZYNSKI (University of Sidney, Australia)
  • Kenneth BLUM (University of Florida, USA)
  • Henrietta BOWDEN-JONES (Imperial College, United Kingdom)
  • Beáta BÖTHE (University of Montreal, Canada)
  • Wim VAN DEN BRINK (University of Amsterdam, The Netherlands)
  • Gerhard BÜHRINGER (Technische Universität Dresden, Germany)
  • Sam-Wook CHOI (Eulji University, Republic of Korea)
  • Damiaan DENYS (University of Amsterdam, The Netherlands)
  • Jeffrey L. DEREVENSKY (McGill University, Canada)
  • Naomi FINEBERG (University of Hertfordshire, United Kingdom)
  • Marie GRALL-BRONNEC (University Hospital of Nantes, France)
  • Jon E. GRANT (University of Minnesota, USA)
  • Mark GRIFFITHS (Nottingham Trent University, United Kingdom)
  • Heather HAUSENBLAS (Jacksonville University, USA)
  • Tobias HAYER (University of Bremen, Germany)
  • Susumu HIGUCHI (National Hospital Organization Kurihama Medical and Addiction Center, Japan)
  • David HODGINS (University of Calgary, Canada)
  • Eric HOLLANDER (Albert Einstein College of Medicine, USA)
  • Jaeseung JEONG (Korea Advanced Institute of Science and Technology, Republic of Korea)
  • Yasser KHAZAAL (Geneva University Hospital, Switzerland)
  • Orsolya KIRÁLY (Eötvös Loránd University, Hungary)
  • Emmanuel KUNTSCHE (La Trobe University, Australia)
  • Hae Kook LEE (The Catholic University of Korea, Republic of Korea)
  • Michel LEJOXEUX (Paris University, France)
  • Anikó MARÁZ (Eötvös Loránd University, Hungary)
  • Giovanni MARTINOTTI (‘Gabriele d’Annunzio’ University of Chieti-Pescara, Italy)
  • Frederick GERARD MOELLER (University of Texas, USA)
  • Daniel Thor OLASON (University of Iceland, Iceland)
  • Nancy PETRY (University of Connecticut, USA)
  • Bettina PIKÓ (University of Szeged, Hungary)
  • Afarin RAHIMI-MOVAGHAR (Teheran University of Medical Sciences, Iran)
  • József RÁCZ (Hungarian Academy of Sciences, Hungary)
  • Rory C. REID (University of California Los Angeles, USA)
  • Marcantanio M. SPADA (London South Bank University, United Kingdom)
  • Daniel SPRITZER (Study Group on Technological Addictions, Brazil)
  • Dan J. STEIN (University of Cape Town, South Africa)
  • Sherry H. STEWART (Dalhousie University, Canada)
  • Attila SZABÓ (Eötvös Loránd University, Hungary)
  • Ferenc TÚRY (Semmelweis University, Hungary)
  • Alfred UHL (Austrian Federal Health Institute, Austria)
  • Johan VANDERLINDEN (University Psychiatric Center K.U.Leuven, Belgium)
  • Alexander E. VOISKOUNSKY (Moscow State University, Russia)
  • Kimberly YOUNG (Center for Internet Addiction, USA)

 

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Jun 2021 0 24 13
Jul 2021 0 16 16
Aug 2021 0 28 14
Sep 2021 0 12 10
Oct 2021 0 24 30
Nov 2021 0 26 35
Dec 2021 0 1 1