(ICT) literacy of Hungarian English majors: A validation study

Information and communications technology (ICT) inclusion has long been at the forefront of professional language pedagogy discourse. It has been argued that ICT implementation is globally advocated but depends on local variables. ICT literacy nowadays does not only include owning and operating devices, but also the ability to create content, solve problems and minding digital safety. The aims of this study were to validate a questionnaire mapping the ICT literacy of one particular group of adult learners: Hungarian English majors ( N 5 45) and to offer some preliminary results. After two rounds of reliability analysis, all nine constructs of the questionnaire proved to be reliable, each above a minimum Cronbach ’ s alpha value of 0.60. Based on the questionnaire results, it can be said that Hungarian English majors have good digital competences, ICT devices are generally available for them, but their ICT acceptance is lower than hypothesised, and devices emerge as learning tools for students rather than substitutes for face-to-face interaction. Since the questionnaire was piloted on a small sample size ( N 5 45), results are only preliminary; therefore, this article outlines plans for future administration of the questionnaire.


INTRODUCTION
In the educational context, using Information and Communications Technology (ICT) devices for learning has long been part of professional discourse. Mishra and Koehler (2006) laid the theoretical foundations of effective ICT inclusive teaching by proposing the Technological Pedagogical Content Knowledge Framework (TPACK), arguing that meaningful ICT inclusion rests on the synthesis of teachers' pedagogical and content knowledge both in terms of subject-specific methodology and technical knowledge. In an overview of TPACKbased research, Koehler, Shin, and Mishra (2012) noted that several studies concluded by measuring moderate and high correlations between the TPACK subscales, suggesting that the elements of the framework might not be separate after all (Koehler et al., 2012), especially in terms of the reported high correlations between the pedagogical knowledge and the technological knowledge components of the framework, which suggests that pedagogical and technological knowledge are closely related. This correlation suggests that teachers generally need better subject-specific ICT methodological training, preferably throughout their preservice years (Caena, 2014;Cox, Preston, & Cox, 2009;Koehler, Mishra, Kereluik, Shin, & Graham, 2014;Russell, Bebell, O'Dwyer, & O'Connor, 2003).
Research has also challenged some of the most frequent beliefs surrounding ICT integration. Caena (2014) registered that the European Union's (EU) directives support ICT inclusion in the classrooms all around the EU; however, implementation is a local variable, member countries have total autonomy in designing how they would include technology in their teacher training programmes. It has also been confirmed that investing into technological devices does not automatically lead to meaningful inclusion in the classrooms (McKenzie, 2001), and apart from devices being present in an institution, there is a demand for methodologically welltrained teachers and a supportive school leadership (BECTA, 2003) in order to facilitate meaningful inclusion. Additionally, the complexity of teaching and teacher-learner relationships should not be over-simplified either. Since methodology and teacher-learner relationships are complex issues, it is disadvantageous to put older and younger teachers in the classic juxtaposition of the old is the past and the new is the future (Bayne & Ross, 2011, p. 161). Also, teachers should maintain a critical and professional approach to technology inclusion, however well-supported ICT inclusive education is in the media (Bayne & Ross, 2011). As it has been argued, teachers' ICT methodology is tightly linked to their subject-specific methodology, and as such, inclusion should rest on the expert judgement of educational professionals rather than following a fashion supported and pushed by the media.
There are many advantages of meaningful ICT integration. It has been reported that ICT use in education prepares learners for modern workplaces (Yelland, 2001), makes them more autonomous learners (Iding, Crosby, & Speitel, 2002), and technological devices emerge as lifelong learning tools for the learners (Inan & Lowther, 2010). On the other hand, it is the teacher who is responsible for the implementation of the integration, because owning devices neither suggests that their inclusion is meaningful (MDOS, 2016), nor can be linked to students' willingness to use them for learning purposes (Fekete, 2017).
The present study aimed to design and validate a questionnaire measuring the ICT literacy of one group of adult learners: Hungarian English majors. A number of factors support the idea of developing a measurement tool for Hungarian adult learners, such as the fact that the integration of ICT in the classrooms is a local variable (Caena, 2014); integration of devices is not evidently meaningful in Hungarian schools (MDOS, 2016); learners do not inherently like using technology for learning (Fekete, 2017); and teachers are urged to enrol in ICT training courses to receive training on modern, computer-assisted teaching methods ( € Oveges & Csiz er, 2018) in order to facilitate meaningful inclusion and a professional learning environment for students. Because little is known about tertiary students' ICT use in Hungary, the goal was to identify constructs that can help to measure the ICT literacy of Hungarian English majors, discover relationships between the constructs and describe how able and willing learners are to use ICT both in their free-time and in their learning processes.

REVIEW OF LITERATURE ICT inclusion research
It has been argued that ICT inclusion is a complex issue for several reasons. Four driving forces for ICT in learning have been proposed by ten Brummelhuis and Kuiper (2008), these are (1) the teacher, (2) the learner, (3) the learning content, and (4) the learning material (p. 107). Because ICT integration is greatly advocated, institutions tend to make considerable investments in equipping their classrooms with various devices; however, sometimes neither sufficient teacher-training, nor adequate testing of the technological device for student learning purposes takes place prior to the investment (McKenzie, 2001;ten Brummelhuis & Kuiper, 2008). ICT inclusion is meaningful only if it facilitates learning in a way that it prepares students for the necessary skills of today's information society. These skills are partly pedagogical: the aim of education is to teach students, but they are also technical in a sense that students need to learn how to operate these devices in order to be independent users of them and regard them as learning tools.
Implementing ICT learning and teaching is inseparable from local variables. Although the EU promotes ICT inclusive teaching, implementation takes place on micro-levels (Caena, 2014), and inclusion is often researched on a national level, focussing on a specific stage of education (BECTA, 2013;MDOS, 2016;€ Oveges & Csiz er, 2018). Additionally, the success of inclusion depends on teachers, learners and the educational context of implementation (Lim, 2002). Success rests on local variables: teachers' general methodology and ICT-specific methodology, as well as students' openness and willingness towards learning with technology. As such, due to a lack of centralised directive, it is advisable to conduct ICT inclusion research focussing on a relatively homogenous group of learners, which was selected to be tertiary adult learners in this study.

The Hungarian context
Hungarian ICT inclusion research mainly focuses on facilitation of inclusion (teacher competences, available technology in institutions) and the digital competences of learners. However convenient it would be to claim that students enrolled in education are all autonomous users of technology for learning purposes, the issue is not that simple. Papp-Danka (2013) challenged the oversimplified categorisation of digital native and digital immigrant learners, because using ICT devices is rather a choice nowadays than an age-specific phenomenon. Individuals either generally have a positive attitude towards ICT devices or not, regardless of their age; therefore, those who favour ICT devices should be regarded as digital citizens (Papp-Danka, 2013). Although technology use for learning purposes is not evident, people own more and more ICT devices. In 2001 a fifth of Hungarian households owned computers, whereas a year later a third of them owned personal computers (K arp ati, 2012). Targeting tertiary learners, it has been reported that almost every student owned a laptop in 2016 (MDOS, 2016), and background variables collected for this research have also confirmed this trend, since only one of the informants did not own a laptop, but all of the informants owned a smartphone. Evidently, technology has become ubiquitous.
It is also argued that several sub-competences have become part of basic ICT competences apart from learners being able to use the devices and the software that runs on them. In a comprehensive review, Tongori (2012) concluded that ICT literacy also includes legal and ethical aspects of using ICT devices responsibly and technological knowledge. ICT inclusive education is favoured because ICT use prepares learners for modern workplaces and contributes to the economic growth of the country; what is more, for modern generations, using ICT devices has become most natural (Tongori, 2012). In a study conducted into the digital competences of secondary learners, T oth-M ozer (2014), among other aspects, concluded by urging for more research targeting the learners, because much is supposed of their reasons, abilities and willingness for ICT use, but little is confirmed by empirical research. It seems that students need specific training in using ICT devices for learning purposes, because based on the population examined, informants cannot be claimed digitally competent (T oth-M ozer, 2014). EU initiatives also target digital literacy, as the EU (2015) published a Digital competences self-assessment grid as part of Europass CVs in which individuals can rate their own digital competences. The grid consists of five elements; (1) information processing, (2) communication, (3) content creation, (4) safety, and (5) problem solving (EU, 2015). People are welcome to assess their own competences through "can-do" statements and claiming to be basic, independent or proficient users of technology (EU, 2015) regarding each concept.
In their pilot study targeting the digital competences of Hungarian teacher majors, Dring o-Horv ath and Gonda (2018) confirmed that future teachers' ICT abilities are mainly formed throughout their teaching practice. It was also suggested that more specific training modules are needed for a paradigm shift that would result in more competent teachers. Respondents in the study mainly claimed that ICT specific training should be an integrated part of teacher education programmes and should not only be offered in the form of workshops (Dring o-Horv ath & Gonda, 2018) although workshops might be ideal for inservice teachers. Therefore, the present pilot study aims at contributing to ICT research among adult learners by designing a measurement tool for their current competences that might serve as the basis of designing more specific training modules and programmes.

THE CURRENT STUDY
As the literature suggests, there is need for more local and comprehensive research into the ICT literacy of Hungarian adult learners. The present study was conducted to answer the following set of research questions (RQs):

RESEARCH DESIGN
This quantitative questionnaire study was designed, administered and validated following the proposed protocol of D€ ornyei and Csiz er (2012)

Participants
The non-probability convenience sample consisted of 45 university students from Hungary. All respondents were Hungarian English majors, 15 of them were English Studies BA students and 30 of them were enrolled in the Teaching MA programme. Respondents were between the ages of 19 and 27, their mean age was 22.27 years (SD 5 1.95; N 5 45), therefore the sample featured Hungarian adult learners. The gender distribution of the sample was 37 females and 8 males that arguably well-represents the target population. Participants were between their first and sixth year of studies, the mean was 3.16 (SD 5 1.52; N 5 45). Filling in the questionnaire was voluntary, and the respondents were granted anonymity. Besides gender, age and years of studies, some other background variables were collected to discover what ICT devices informants owned. There were 9 ICT devices listed in the questionnaire, respondents could select multiple items, and they were welcome to add any other devices they owned. It was important that they selected the devices only if they owned them. Altogether the 45 informants owned 163 ICT devices; each informant owned a smartphone, 44 owned a laptop (that keys in with the findings of MDOS, 2016), 29 a television, 12 a personal computer, 11 a smart television, 11 an e-book reader, 8 a tablet or iPad, 2 a Play Station 4, 1 a smartwatch and none of the informants owned a (nonsmart) mobile phone.

Methods of data collection and analysis
For the validation of the instrument, a paper and pencil questionnaire was administered at two Hungarian universities. The dataset was then carefully input into SPSS version 22. To check construct validity and for preliminary results descriptive and inferential statistical calculations were performed.

Data collection instrument
The data collection instrument was a questionnaire which was designed for the purpose of this study. Constructs and items of the questionnaire were based on the literature and expert judgement. Before the pilot phase, a think-aloud prepilot was administered on two volunteers that resulted in the final wording of the items. Besides the background variables, the piloted questionnaire consisted of 9 constructs that altogether included 54 items. The questionnaire was administered in Hungarian, which was the native tongue of the participants. The questionnaire started with a brief message to the participants asking them to answer the questions honestly; they were reassured of their anonymity and they were offered a brief definition of ICT. The definition was adapted and translated from Hew and Brush (2007) who defined ICT as "computing devices such as desktop computers, laptops, handheld computers, software, or internet" (p. 225). Respondents were asked to rate the questionnaire items on 5-point Likert-scales (ranging from 1 to 5) depending on the extent to which they felt that the item described them (1 meant not true for me at all, 5 meant absolutely true for me). Informants were asked to rate 7 out of the 9 constructs from two perspectives: focussing on their free time use of the devices and specifically regarding their studies. It took approximately 15 minutes to fill in the questionnaire.
To describe the constructs of the questionnaire, a list of them follows with the number of items they consisted of. It is also indicated if respondents were asked to rate the items from two perspectives: free time use and learning use; or if the construct only targeted one of the perspectives: free time or learning use only. A sample item of each construct is also provided.
1. Acceptance of ICT devices (free time and learning use; 4 items): the extent participants accept the use of ICT devices. Sample item: I think using ICT devices is advantageous. 2. Availability of ICT devices (free time and learning use; 4 items): the extent participants have access to ICT devices. Sample item: An ICT device is usually available for me to use. 3. Reasons for using ICT devices (free time and learning use; 4 items): it has been claimed that a major reason for using ICT devices in general and for learning purposes is to prepare students for the world of work in today's technology-driven world (Tongori, 2012). Therefore, it was important to find out the extent to which this claim is true for the sample. ; 18 items): although the sixth construct of the piloted questionnaire aims to measure respondents' perceived ability to use ICT devices, selected items from the EU's (2015) competence list was adapted for the purposes of this construct because they provide more comprehensive insights into what is understood by modern ICT literacy. Out of the original five competence areas, items from three were partly adapted (information processing, communication, and problem solving). Most adapted items were slightly rephrased because the original document provides a selfassessment grid instead of a self-assessment questionnaire. Sample item: I can perform routine tasks on ICT devices such as restarting programs or the device and checking the internet connection.
The questionnaire consisted of five main parts: (1) items to be rated from both free time and learning perspectives (constructs 1-7), (2) construct 8 to be rated from the perspective of learning only and 2 learning-focused items from construct 6, (3) 2 free-time only items of construct 6, (4) construct 9 items to be rated from a general perspective and (5) collecting background variables.

RESULTS AND DISCUSSION
The internal reliability of the latent constructs (RQ 1) To confirm whether the questionnaire produced reliable scales, randomized questionnaire items were reorganised into constructs, and as the first method of reliability check, Cronbach's alphas of the constructs were calculated (consult Table 1, unreliable constructs italicised). It is suggested that constructs below a 0.6 Cronbach's alpha should be regarded as unreliable, but 0.7 is the desired value in order to consider the constructs reliable (D€ ornyei & Csiz er, 2012). Because informants were sometimes asked to rate items from two perspectives (free time use and learning use), for such constructs, Cronbach's alphas were calculated separately.
As the first round of reliability analysis showed, three constructs proved to be unreliable (Availability of ICT devices, Perceived ability to use ICT devices and ICT use over personal contact), and the learning use of Willingness for ICT skills development did not meet the reliability criteria. Furthermore, dimension reduction was applied to see if any of the items should be reversed or removed.
Literature suggests treating three constructs, Acceptance of ICT devices, Availability of ICT devices and Perceived ability to use ICT devices as joint constructs (Tongori, 2012;T oth-M ozer, 2014), because arguably free time and learning use of devices cannot be separated from each other. Additionally, the "Alpha if item deleted" analysis of the Availability of ICT devices, Willingness for ICT skills development and ICT use over personal contact concepts confirmed that there was a problem with the wording of one of the items in each construct. In the Perceived ability to use ICT devices construct, two items had to be deleted, because principal component analysis confirmed that they measure different components. Dimension reduction also showed that one item in the ICT use over personal contact construct needed to be reversed. Fortunately, in the second round of data analysis that followed (consult Table 2), the constructs that were proposed for item recovery proved to be reliable.
After two rounds of reliability analysis, it could be concluded that the questionnaire proved to be reliable. Although constructs with 3 items (Availability of ICT devices and Willingness for ICT skills development) and constructs below a 0.7 Cronbach's alpha (Perceived ability to use ICT devices and ICT use over personal contact) are less desirable (D€ ornyei & Csiz er, 2012), before the administration of the questionnaire on a larger sample, some additional modifications could be made. As for the constructs with 3 items, additional items should be added that would most likely raise the reliability of each construct. As far as Perceived ability to use ICT devices is concerned, it would be worth considering excluding the construct altogether from the future administration of the questionnaire, because the adapted construct of Digital competencies proved to be much more reliable and the two constructs targeted to measure roughly the same issue. It would also be advisable to rephrase items of the ICT use over personal contact construct, because one of the items had to be deleted and another one had to be reversed to reach a 0.666 Cronbach's alpha, which is still under the desired 0.7 value. This construct would need further testing; therefore, it might be disregarded from the administration of the validated questionnaire.

Preliminary results and discussion (RQs 2-5)
The validated questionnaire gave grounds for some preliminary data analysis. Because only 45 informants  Table 3). As Hungarian trends suggested (MDOS, 2016), availability of ICT devices for learners is relatively high (M 5 4.63; SD 5 0.44); however, neither the acceptance of ICT devices (M 5 3.88; SD 5 0.75) nor the willingness to take part in ICT skills development (free time: M 5 3.57; SD 5 0.81; learning use: M 5 3.60; SD 5 0.86) live up to the high availability value. It has been argued that ICT devices are educational tools for the learners rather than magic solutions to motivate and transform them into autonomous users of ICT devices for learning purposes. Learners are relatively competent users of ICT devices (M 5 4.31; SD 5 0.46), they do not unquestionably prefer device use over personal contact (free time: M 5 3.01; SD 5 0.59; learning use: M 5 3.13; SD 5 0.63), especially in their free time. This result keys in with the literature reviewed (Papp-Danka, 2013;T oth-M ozer, 2014). Paired sample t-test results also shed light on the fact that learners have more positive attitudes towards using ICT devices for the apparent advantages their usage provides in the educational context (M 5 4.48; SD 5 0.66) over free time (M 5 4.20; SD 5 0.69), and as such, devices are indeed learning tools for them (t 5 4.44; Sig. (2tailed): P < 0.001).
Research question 5 enquired about whether there are any statistically significant correlations between the constructs of Digital competences, Availability of ICT devices and Acceptance of ICT devices among Hungarian English majors because literature suggested that availability and acceptance are high (MDOS, 2016;T oth-M ozer, 2014), but there is room for improvement in learners' digital competence development (T oth-M ozer, 2014). For this, bivariate Pearson correlation analyses confirmed a moderate correlation between Digital competences and Availability of ICT devices (r 5 0.455; Sig. (2-tailed): P 5 0.002), while there are weak correlations between Digital competences and Acceptance of ICT devices (r 5 0.326; Sig. (2-tailed): P 5 0.029), and Acceptance of ICT devices and Availability of ICT devices (r 5 0.336; Sig. (2-tailed): P 5 0.024). The moderate correlation between Digital competences and Availability of ICT devices suggests that operating modern technological devices requires a set of specific skills to be mastered and is indeed more complex than just being able to use some software (Tongori, 2012), because they involve competences such as conscious digital information processing and problem solving (EU, 2015). It is somewhat surprising that there are only weak correlations between acceptance and digital competences, and acceptance and availability, but these are perhaps consequences of how much technology has become part of everyday life.
When running Pearson correlations on all the scales, a strong correlation was discovered in the dataset. There appears to be a strong significant correlation between Acceptance of ICT devices and Willingness for ICT skills developmentfree time use (r 5 0.674; Sig. (2-tailed): P < 0.001); however, between Acceptance of ICT devices and Willingness for ICT skills developmentlearning use there correlation is weak (r 5 0.355; Sig. (2-tailed): P 5 0.017), but it still proved to be significant. Because there are not any statistically significant causative relationships (regression) between the scales, it can only be hypothesized that ICT devices do not automatically trigger the need for wanting students to develop in their use for learning purposes. As T oth-M ozer (2014) explained, the ubiquitous presence of ICT devices and learners' relatively good ability to use them do not necessarily trigger applying them for learning purposes; therefore, more emphasis should be put on their meaningful classroom inclusion.

CONCLUSIONS
The proposed data collection instrument proved to be reliable after two rounds of reliability analysis (RQ 1). Because the questionnaire was administered on a sample ideal for piloting only (D€ ornyei & Csiz er, 2012), the scales and the preliminary results should be interpreted accordingly. Because of the pilot phase, no full factor analysis was carried out on all scales simultaneously; therefore, future studies are needed to establish the discriminant validity of the instrument. Although the sample size was ideal for validating the instrument, it was not enough to be the basis of generalizable answers to the non-technical research questions (RQs 2-5). Preliminary data analysis confirmed that in terms of availability and acceptance of ICT devices among Hungarian English majors, there is no distinction between free time and learning use. Paired sample t-tests proved that ICT devices are generally available (M 5 4.63; SD 5 0.44) for learners, but contrary to what could be expected, acceptance (M 5 3.88; SD 5 0.75) is lower (t 5 6.853; Sig. (2-tailed): P < 0.001) and devices are rather learning tools (M 5 4.48; SD 5 0.66) for students than substitute (M 5 3.01; SD 5 0.59) for face-to-face contact (t 5 8.624; Sig. (2-tailed): P < 0.001). Learners' digital competences are relatively high (M 5 4.31; SD 5 0.46); therefore, it is confirmed that ICT use nowadays also includes information processing and problem-solving competences on top of using the devices for communication. Statistically significant correlations between availability and acceptance, and acceptance and digital competences are weak, while between availability and digital competences, there is a moderate relationship between the scales. There is a statistically significant strong relationship between acceptance and the willingness to develop in ICT skills for free time use, but between acceptance and the willingness to develop in ICT skills for learning, the correlation is weak.
Acknowledging the limitations of the results which originate from a smaller sample size, the instrument proved to be valid and ready to be administered on a larger sample. A larger sample size (N ≥ 250) will allow for more statistical tests to be run on the sample and presumably result in unearthing further links between the scales.