ICT inclusion research

It has been argued that ICT inclusion is a complex issue for several reasons. Four driving forces for ICT in learning have been proposed by ten Brummelhuis and Kuiper (2008), these are (1) the teacher, (2) the learner, (3) the learning content, and (4) the learning material (p. 107). Because ICT integration is greatly advocated, institutions tend to make considerable investments in equipping their classrooms with various devices; however, sometimes neither sufficient teacher-training, nor adequate testing of the technological device for student learning purposes takes place prior to the investment (McKenzie, 2001; ten Brummelhuis & Kuiper, 2008). ICT inclusion is meaningful only if it facilitates learning in a way that it prepares students for the necessary skills of today's information society. These skills are partly pedagogical: the aim of education is to teach students, but they are also technical in a sense that students need to learn how to operate these devices in order to be independent users of them and regard them as learning tools.

Implementing ICT learning and teaching is inseparable from local variables. Although the EU promotes ICT inclusive teaching, implementation takes place on micro-levels (Caena, 2014), and inclusion is often researched on a national level, focussing on a specific stage of education (BECTA, 2013; MDOS, 2016; Öveges & Csizér, 2018). Additionally, the success of inclusion depends on teachers, learners and the educational context of implementation (Lim, 2002). Success rests on local variables: teachers' general methodology and ICT-specific methodology, as well as students' openness and willingness towards learning with technology. As such, due to a lack of centralised directive, it is advisable to conduct ICT inclusion research focussing on a relatively homogenous group of learners, which was selected to be tertiary adult learners in this study.

The Hungarian context

Hungarian ICT inclusion research mainly focuses on facilitation of inclusion (teacher competences, available technology in institutions) and the digital competences of learners. However convenient it would be to claim that students enrolled in education are all autonomous users of technology for learning purposes, the issue is not that simple. Papp-Danka (2013) challenged the oversimplified categorisation of digital native and digital immigrant learners, because using ICT devices is rather a choice nowadays than an age-specific phenomenon. Individuals either generally have a positive attitude towards ICT devices or not, regardless of their age; therefore, those who favour ICT devices should be regarded as digital citizens (Papp-Danka, 2013). Although technology use for learning purposes is not evident, people own more and more ICT devices. In 2001 a fifth of Hungarian households owned computers, whereas a year later a third of them owned personal computers (Kárpáti, 2012). Targeting tertiary learners, it has been reported that almost every student owned a laptop in 2016 (MDOS, 2016), and background variables collected for this research have also confirmed this trend, since only one of the informants did not own a laptop, but all of the informants owned a smartphone. Evidently, technology has become ubiquitous.

It is also argued that several sub-competences have become part of basic ICT competences apart from learners being able to use the devices and the software that runs on them. In a comprehensive review, Tongori (2012) concluded that ICT literacy also includes legal and ethical aspects of using ICT devices responsibly and technological knowledge. ICT inclusive education is favoured because ICT use prepares learners for modern workplaces and contributes to the economic growth of the country; what is more, for modern generations, using ICT devices has become most natural (Tongori, 2012). In a study conducted into the digital competences of secondary learners, Tóth-Mózer (2014), among other aspects, concluded by urging for more research targeting the learners, because much is supposed of their reasons, abilities and willingness for ICT use, but little is confirmed by empirical research. It seems that students need specific training in using ICT devices for learning purposes, because based on the population examined, informants cannot be claimed digitally competent (Tóth-Mózer, 2014). EU initiatives also target digital literacy, as the EU (2015) published a Digital competences self-assessment grid as part of Europass CVs in which individuals can rate their own digital competences. The grid consists of five elements; (1) information processing, (2) communication, (3) content creation, (4) safety, and (5) problem solving (EU, 2015). People are welcome to assess their own competences through “can-do” statements and claiming to be basic, independent or proficient users of technology (EU, 2015) regarding each concept.

In their pilot study targeting the digital competences of Hungarian teacher majors, Dringó-Horváth and Gonda (2018) confirmed that future teachers' ICT abilities are mainly formed throughout their teaching practice. It was also suggested that more specific training modules are needed for a paradigm shift that would result in more competent teachers. Respondents in the study mainly claimed that ICT specific training should be an integrated part of teacher education programmes and should not only be offered in the form of workshops (Dringó-Horváth & Gonda, 2018) although workshops might be ideal for in-service teachers. Therefore, the present pilot study aims at contributing to ICT research among adult learners by designing a measurement tool for their current competences that might serve as the basis of designing more specific training modules and programmes.

Participants

The non-probability convenience sample consisted of 45 university students from Hungary. All respondents were Hungarian English majors, 15 of them were English Studies BA students and 30 of them were enrolled in the Teaching MA programme. Respondents were between the ages of 19 and 27, their mean age was 22.27 years (SD = 1.95; N = 45), therefore the sample featured Hungarian adult learners. The gender distribution of the sample was 37 females and 8 males that arguably well-represents the target population. Participants were between their first and sixth year of studies, the mean was 3.16 (SD = 1.52; N = 45). Filling in the questionnaire was voluntary, and the respondents were granted anonymity.

Besides gender, age and years of studies, some other background variables were collected to discover what ICT devices informants owned. There were 9 ICT devices listed in the questionnaire, respondents could select multiple items, and they were welcome to add any other devices they owned. It was important that they selected the devices only if they owned them. Altogether the 45 informants owned 163 ICT devices; each informant owned a smartphone, 44 owned a laptop (that keys in with the findings of MDOS, 2016), 29 a television, 12 a personal computer, 11 a smart television, 11 an e-book reader, 8 a tablet or iPad, 2 a Play Station 4, 1 a smartwatch and none of the informants owned a (non-smart) mobile phone.

Methods of data collection and analysis

For the validation of the instrument, a paper and pencil questionnaire was administered at two Hungarian universities. The dataset was then carefully input into SPSS version 22. To check construct validity and for preliminary results descriptive and inferential statistical calculations were performed.

Data collection instrument

The data collection instrument was a questionnaire which was designed for the purpose of this study. Constructs and items of the questionnaire were based on the literature and expert judgement. Before the pilot phase, a think-aloud pre-pilot was administered on two volunteers that resulted in the final wording of the items. Besides the background variables, the piloted questionnaire consisted of 9 constructs that altogether included 54 items. The questionnaire was administered in Hungarian, which was the native tongue of the participants. The questionnaire started with a brief message to the participants asking them to answer the questions honestly; they were reassured of their anonymity and they were offered a brief definition of ICT. The definition was adapted and translated from Hew and Brush (2007) who defined ICT as “computing devices such as desktop computers, laptops, handheld computers, software, or internet” (p. 225). Respondents were asked to rate the questionnaire items on 5-point Likert-scales (ranging from 1 to 5) depending on the extent to which they felt that the item described them (1 meant not true for me at all, 5 meant absolutely true for me). Informants were asked to rate 7 out of the 9 constructs from two perspectives: focussing on their free time use of the devices and specifically regarding their studies. It took approximately 15 minutes to fill in the questionnaire.

To describe the constructs of the questionnaire, a list of them follows with the number of items they consisted of. It is also indicated if respondents were asked to rate the items from two perspectives: free time use and learning use; or if the construct only targeted one of the perspectives: free time or learning use only. A sample item of each construct is also provided.

1. 1.Acceptance of ICT devices (free time and learning use; 4 items): the extent participants accept the use of ICT devices. Sample item: I think using ICT devices is advantageous.
2. 2.Availability of ICT devices (free time and learning use; 4 items): the extent participants have access to ICT devices. Sample item: An ICT device is usually available for me to use.
3. 3.Reasons for using ICT devices (free time and learning use; 4 items): it has been claimed that a major reason for using ICT devices in general and for learning purposes is to prepare students for the world of work in today's technology-driven world (Tongori, 2012). Therefore, it was important to find out the extent to which this claim is true for the sample. Sample item: I think in today's world anyone can be expected to put together a presentation using an ICT device.
4. 4.Willingness for ICT skills development (free time and learning use; 4 items): the extent respondents feel they are up to date users of ICT devices and how willing they are to develop constantly. Sample item: I feel like I devote enough time to develop my knowledge about ICT devices.
5. 5.Opportunities for ICT skills development (free time and learning use; 4 items): the extent respondents feel that it is possible for them to continuously develop their ICT skills. Sample item: I feel that I have every opportunity to learn how to use new ICT devices.
6. 6.Perceived ability to use ICT devices (free time and learning use; 6 items: 2 general items to be rated from both perspectives and an extra 2–2 items regarding free time use and learning use only): to what extent respondents believed they are competent users of ICT devices. Sample item: I can confidently differentiate between reliable and unreliable online content.
7. 7.ICT use over personal contact (free time and learning use; 5 items): how much respondents think that using ICT devices can substitute personal interaction. Sample item: I think ICT devices can substitute for face-to-face meetings because all kinds of information can be shared digitally.
8. 8.Using ICT devices for language learning (learning use only; 5 items) why and to what extent respondents use ICT devices for their learning purposes. Sample item: ICT devices make learning stress-free.
9. 9.Digital competencies (general use, selected items adapted from the Digital competences self-assessment grid (EU, 2015); 18 items): although the sixth construct of the piloted questionnaire aims to measure respondents' perceived ability to use ICT devices, selected items from the EU's (2015) competence list was adapted for the purposes of this construct because they provide more comprehensive insights into what is understood by modern ICT literacy. Out of the original five competence areas, items from three were partly adapted (information processing, communication, and problem solving). Most adapted items were slightly rephrased because the original document provides a self-assessment grid instead of a self-assessment questionnaire. Sample item: I can perform routine tasks on ICT devices such as restarting programs or the device and checking the internet connection.

The questionnaire consisted of five main parts: (1) items to be rated from both free time and learning perspectives (constructs 1–7), (2) construct 8 to be rated from the perspective of learning only and 2 learning-focused items from construct 6, (3) 2 free-time only items of construct 6, (4) construct 9 items to be rated from a general perspective and (5) collecting background variables.

The internal reliability of the latent constructs (RQ 1)

To confirm whether the questionnaire produced reliable scales, randomized questionnaire items were reorganised into constructs, and as the first method of reliability check, Cronbach's alphas of the constructs were calculated (consult Table 1, unreliable constructs italicised). It is suggested that constructs below a 0.6 Cronbach's alpha should be regarded as unreliable, but 0.7 is the desired value in order to consider the constructs reliable (Dörnyei & Csizér, 2012). Because informants were sometimes asked to rate items from two perspectives (free time use and learning use), for such constructs, Cronbach's alphas were calculated separately.

As the first round of reliability analysis showed, three constructs proved to be unreliable (Availability of ICT devices, Perceived ability to use ICT devices and ICT use over personal contact), and the learning use of Willingness for ICT skills development did not meet the reliability criteria. Furthermore, dimension reduction was applied to see if any of the items should be reversed or removed.

Literature suggests treating three constructs, Acceptance of ICT devices, Availability of ICT devices and Perceived ability to use ICT devices as joint constructs (Tongori, 2012; Tóth-Mózer, 2014), because arguably free time and learning use of devices cannot be separated from each other. Additionally, the “Alpha if item deleted” analysis of the Availability of ICT devices, Willingness for ICT skills development and ICT use over personal contact concepts confirmed that there was a problem with the wording of one of the items in each construct. In the Perceived ability to use ICT devices construct, two items had to be deleted, because principal component analysis confirmed that they measure different components. Dimension reduction also showed that one item in the ICT use over personal contact construct needed to be reversed. Fortunately, in the second round of data analysis that followed (consult Table 2), the constructs that were proposed for item recovery proved to be reliable.

After two rounds of reliability analysis, it could be concluded that the questionnaire proved to be reliable. Although constructs with 3 items (Availability of ICT devices and Willingness for ICT skills development) and constructs below a 0.7 Cronbach's alpha (Perceived ability to use ICT devices and ICT use over personal contact) are less desirable (Dörnyei & Csizér, 2012), before the administration of the questionnaire on a larger sample, some additional modifications could be made. As for the constructs with 3 items, additional items should be added that would most likely raise the reliability of each construct. As far as Perceived ability to use ICT devices is concerned, it would be worth considering excluding the construct altogether from the future administration of the questionnaire, because the adapted construct of Digital competencies proved to be much more reliable and the two constructs targeted to measure roughly the same issue. It would also be advisable to rephrase items of the ICT use over personal contact construct, because one of the items had to be deleted and another one had to be reversed to reach a 0.666 Cronbach's alpha, which is still under the desired 0.7 value. This construct would need further testing; therefore, it might be disregarded from the administration of the validated questionnaire.

Preliminary results and discussion (RQs 2–5)

The validated questionnaire gave grounds for some preliminary data analysis. Because only 45 informants participated in the research, these results should indeed be regarded as preliminary, and they are far from being generalizable (Dörnyei & Csizér, 2012). Research questions 2 to 4 enquired into the extent to which Hungarian English majors are willing and able to use ICT devices, as well as whether learners use them for some apparent future advantages their usage provides. To answer these questions, descriptive statistics of the scales were calculated (consult Table 3).

As Hungarian trends suggested (MDOS, 2016), availability of ICT devices for learners is relatively high (M = 4.63; SD = 0.44); however, neither the acceptance of ICT devices (M = 3.88; SD = 0.75) nor the willingness to take part in ICT skills development (free time: M = 3.57; SD = 0.81; learning use: M = 3.60; SD = 0.86) live up to the high availability value. It has been argued that ICT devices are educational tools for the learners rather than magic solutions to motivate and transform them into autonomous users of ICT devices for learning purposes. Learners are relatively competent users of ICT devices (M = 4.31; SD = 0.46), they do not unquestionably prefer device use over personal contact (free time: M = 3.01; SD = 0.59; learning use: M = 3.13; SD = 0.63), especially in their free time. This result keys in with the literature reviewed (Papp-Danka, 2013; Tóth-Mózer, 2014). Paired sample t-test results also shed light on the fact that learners have more positive attitudes towards using ICT devices for the apparent advantages their usage provides in the educational context (M = 4.48; SD = 0.66) over free time (M = 4.20; SD = 0.69), and as such, devices are indeed learning tools for them (t = 4.44; Sig. (2-tailed): P < 0.001).

Research question 5 enquired about whether there are any statistically significant correlations between the constructs of Digital competences, Availability of ICT devices and Acceptance of ICT devices among Hungarian English majors because literature suggested that availability and acceptance are high (MDOS, 2016; Tóth-Mózer, 2014), but there is room for improvement in learners' digital competence development (Tóth-Mózer, 2014). For this, bivariate Pearson correlation analyses confirmed a moderate correlation between Digital competences and Availability of ICT devices (r = 0.455; Sig. (2-tailed): P = 0.002), while there are weak correlations between Digital competences and Acceptance of ICT devices (r = 0.326; Sig. (2-tailed): P = 0.029), and Acceptance of ICT devices and Availability of ICT devices (r = 0.336; Sig. (2-tailed): P = 0.024). The moderate correlation between Digital competences and Availability of ICT devices suggests that operating modern technological devices requires a set of specific skills to be mastered and is indeed more complex than just being able to use some software (Tongori, 2012), because they involve competences such as conscious digital information processing and problem solving (EU, 2015). It is somewhat surprising that there are only weak correlations between acceptance and digital competences, and acceptance and availability, but these are perhaps consequences of how much technology has become part of everyday life.

When running Pearson correlations on all the scales, a strong correlation was discovered in the dataset. There appears to be a strong significant correlation between Acceptance of ICT devices and Willingness for ICT skills development – free time use (r = 0.674; Sig. (2-tailed): P < 0.001); however, between Acceptance of ICT devices and Willingness for ICT skills development – learning use there correlation is weak (r = 0.355; Sig. (2-tailed): P = 0.017), but it still proved to be significant. Because there are not any statistically significant causative relationships (regression) between the scales, it can only be hypothesized that ICT devices do not automatically trigger the need for wanting students to develop in their use for learning purposes. As Tóth-Mózer (2014) explained, the ubiquitous presence of ICT devices and learners' relatively good ability to use them do not necessarily trigger applying them for learning purposes; therefore, more emphasis should be put on their meaningful classroom inclusion.