Author:
Imre Fekete Budapest Business School, Budapest, Hungary
PhD Programme in Language Pedagogy, Doctoral School of Education, Eötvös Loránd University, Budapest, Hungary

Search for other papers by Imre Fekete in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-3651-5984
Open access

Abstract

Covid-19-triggered emergency remote teaching shed light on the discrepancies of the long-desired digital transformation of education. To learn more about Hungarian K12 (primary and secondary) teachers' techno-pedagogical skills, this study aimed to measure how they rate the components of the Technological Pedagogical Content Knowledge (TPACK) framework. The observed mean values gave grounds for clustering Hungarian K12 teachers based on their existing techno-pedagogical skills as well as proposing possible directions for development. It was found that among teachers who participated in the study (N = 216), 20% belong to the group of Beginners, 40% are Independent, and 40% are Advanced users of techno-pedagogical tools and methods. The groups are rather homogenous as gender, age, qualification and teaching experience are not predictors of techno-pedagogical knowledge. Beginners need help on the very operational levels of technology, Independent users transform traditional teaching methods in the online space, while Advanced users plan face-to-face and online classes differently but also include techno-pedagogy in their everyday classroom teaching practices. It was further observed that teachers in all groups are generally motivated in preparing for their online lessons, but the perceived motivation of their learners is much lower, and teachers do not generally consider online teaching effective.

Abstract

Covid-19-triggered emergency remote teaching shed light on the discrepancies of the long-desired digital transformation of education. To learn more about Hungarian K12 (primary and secondary) teachers' techno-pedagogical skills, this study aimed to measure how they rate the components of the Technological Pedagogical Content Knowledge (TPACK) framework. The observed mean values gave grounds for clustering Hungarian K12 teachers based on their existing techno-pedagogical skills as well as proposing possible directions for development. It was found that among teachers who participated in the study (N = 216), 20% belong to the group of Beginners, 40% are Independent, and 40% are Advanced users of techno-pedagogical tools and methods. The groups are rather homogenous as gender, age, qualification and teaching experience are not predictors of techno-pedagogical knowledge. Beginners need help on the very operational levels of technology, Independent users transform traditional teaching methods in the online space, while Advanced users plan face-to-face and online classes differently but also include techno-pedagogy in their everyday classroom teaching practices. It was further observed that teachers in all groups are generally motivated in preparing for their online lessons, but the perceived motivation of their learners is much lower, and teachers do not generally consider online teaching effective.

Introduction

The COVID-19 pandemic unquestionably brought sudden and unprecedented changes in all walks of life, out of which institutional teaching and learning are no exceptions. Not only did the pandemic provide an opportunity for teachers to explore new digital teaching methods, it also shed light on some of the contradictions between how ready teachers, learners, and institutions have been all over the world for the abrupt shift to online education (Peters et al., 2020). As has been pointed out, despite the fact that many universities worldwide already offer distant (fully online) or blended (i.e., a combination of face-to-face and online) courses, the sudden immediate shift has caused problems for all parties of education (Peters et al., 2020) because it would go against years of educational research to claim that online education is as simple as putting ordinary classroom practices in the online space (Niess, 2011; Rienties et al., 2020).

The digital transformation of education is a European Union (EU) directive (EU, 2018) with the goal to provide equal opportunities for all EU citizens to access knowledge. According to the EU's country reports (EU, 2019), Hungarian primary and secondary schools meet the standard of most European countries as far as students' digital device ownership is concerned (EU, 2019; MDOS, 2016), yet it has been pointed out many times that digital device ownership does not automatically trigger either meaningful student engagement in how to use the devices for effective learning purposes (Fekete 2017, 2020b; Pintér 2019; Tóth-Mózer, 2017), or teachers being ready to teach via technology (Dringó-Horváth & Gonda, 2018; Öveges & Csizér, 2018).

Following up the EU directive that aims at transforming each level of educational institutions to be ready for digital teaching and learning (EU, 2018), Caena (2014) reviewed the educational policies and curricula of EU member countries and concluded that the implementation of the digital transformation is the responsibility of the member states. More specifically, change can only be achieved from a bottom-up approach; by offering techno-pedagogical methodology courses for teachers enrolled in teacher education (Dringó-Horváth & Gonda, 2018; Fekete, 2020b), providing teacher training possibilities for practicing teachers (Öveges & Csizér, 2018) that would contribute to their technological pedagogical content knowledge development.

As it has been concluded by Főző and Racsko (2020), the first step towards a paradigm shift could be utilizing one of the many available frameworks to map schoolteachers' and learners' digital readiness on micro-levels, then following up on their development stages when it has been established in which areas they need help. Therefore, the aims of this study were to 1) identify skills groups among Hungarian K12 teachers based on their existing techno-pedagogical knowledge on the basis of a Technological Pedagogical Content Knowledge (TPACK) questionnaire (Schmidt et al., 2009) as well as 2) to profile the established groups on the basis of several background variables in order to 3) find viable development possibilities for teachers in each skills group that could be best tailored to individual needs.

Review of the literature

Dimensions of using digital devices

A recent report published on the level of Hungarian K12 schools' digital equipment and level of connectedness found that Hungarian schools on all three surveyed levels (primary, lower- and upper secondary) only meet half of the EU's mean scores, whereas device ownership of learners roughly follows the EU's total average (EU, 2019). It is also reported that the most frequently used digital devices for learning purposes are smartphones, which are used by a quarter of lower-secondary and half of upper-secondary learners. Although the report is mainly concerned with ownership, it is progressive in a way by reporting learners' confidence in their digital competences following up on a self-assessment tool published by the EU (2015), which also meets the EU's general average.

The EU's (2015) Digital literacy self-assessment grid was designed to cover a number of factors that pay a pivotal role in meaningful digital device use. The five factors are 1) safety, 2) communication and collaboration, 3) information and data processing, 4) problem solving, and 5) digital content creation. Users can categorize their competence levels by following certain descriptors to classify if they are basic, independent or advanced technology users of digital devices. As it has been concluded by Tongori (2012), an inclusive approach towards integrating digital devices into teaching and learning is generally favored as the ubiquitous presence of digital technologies prerequisites individuals to be familiar with digital customs and ethics.

Evidently, teachers are not the only sources of students to learn about new technologies, but it has been argued that they have the means to teach them how to use certain devices and technologies for learning purposes specifically (Pintér, 2019; Tóth-Mózer, 2017). Arguably, the best way of developing teachers' techno-pedagogical skills would be the inclusion of digital methodological training courses in teacher education (Dringó-Horváth & Gonda, 2018), but training practicing teachers should not be neglected either (Öveges & Csizér, 2018). Becker (1999) reported that the teachers most likely to include digital technologies in their everyday teaching practice are “1) younger teachers, 2) teachers who are leaders in their profession, and 3) teachers with constructivist pedagogies” (p. 30). Becker (1999) also stated that, despite the classic and convenient juxtaposition of digital natives and digital immigrants, the age factor might lose its importance over time as a predicting factor of digital device use in teaching. According to Bayne and Ross (2011), age has become a less important factor in competent Internet and digital device use, as using technologies is rather a choice than something linked to age; thus it is suggested that individuals using technologies should be called digital citizens (Bayne & Ross, 2011; Papp-Danka, 2013).

Teachers and technology

The Technological Pedagogical Content Knowledge (TPACK) framework was designed by Mishra and Koehler (2006). The authors proposed seven major knowledge components that together ensure meaningful and professionally grounded technology inclusive instruction in education. These components (Koehler et al., 2014) and their interpretations are as follows (Chai, Koh, Tsai, & Tan, 2011):

  1. content knowledge (CK): knowing the contents of the specific subject(s) one teaches such as mathematics, foreign languages, or history

  2. pedagogical knowledge (PK): knowing about how to plan a lesson, manage learners and handle pedagogical challenges such as heterogenous learning groups

  3. technological knowledge (TK): knowing how to use a computer or other digital devices and the programs or applications that run on them

  4. technological content knowledge (TCK): knowing how to find or create content to illustrate what is being taught such as creating a presentation or finding a demonstration video

  5. pedagogical content knowledge (PCK): knowing the teaching methods required to teach one's specific subject(s) so that they best facilitate learning, such as how to teach mathematical formulas or vocabulary learning methods

  6. technological pedagogical knowledge (TPK): knowing how to facilitate learning or implement lessons using technology such as using learning management software

  7. technological pedagogical content knowledge (TPACK): knowing how to implement lessons and facilitate learning by selecting the most appropriate technology for the very purposes of teaching one's specific subject

In a comprehensive literature review summarizing the most important findings of years of TPACK-research, Koehler et al. (2012) concluded that the activation of all TPACK framework components together result in meaningful technology inclusion; that is, it is not enough to have a positive attitude towards technology in general and/or being a good teacher of one's subject. Similar to a general positive attitude towards technology, classic offline methodological knowledge does not translate automatically in a digital environment either.

Voogt et al. (2013) argued that teachers' technology integration is based on a number of factors, such as their knowledge of pedagogy, their teaching experience, their beliefs and customs. This suggests that singling out specific elements of TPACK would only result in false claims about what reasoning is behind certain teachers' willingness towards technology inclusion or non-inclusion (Voogt et al., 2013). Reflecting on the complexity of reasoning behind technology use, better and more reliable instruments should be designed (Voogt et al., 2013) to understand about teachers' technology use in its complexity rather than by its single elements stripped from various other factors that influence the choice of technology use. Measurement instruments should be more specific in terms of subjects taught by teachers (Schmidt et al., 2009), local contexts where the integration is taking place (Főző & Racsko, 2020) and the dynamic, rapidly changing nature of what each TPACK element conceptually stands for (Főző & Racsko, 2020; Voogt et al., 2013).

The fact that beliefs overrule training or a seemingly positive attitude towards technology inclusion was observed by many researchers (Chen, 2008; Hennessy et al., 2007; Webb & Cox, 2004). Respondents in these studies were asked to elaborate on their professional reasoning for ICT inclusion in their classrooms. Most of them agreed with the principles of meaningful technology use and were in alignment with TPACK theory (Koehler et al., 2014), stating that ICT inclusion is the extension of a teacher's array of methodology. However, based on the post-interview lesson observations, the same teachers were reported to resort to their beliefs and use more traditional methods in the classroom than they had claimed (Heitink et al., 2016). This is important for two reasons; on the one hand, it confirms that teachers' beliefs are deep-rooted and the lack of experience with theoretically grounded use of technology results in them resorting to more traditional teaching methods. On the other hand, these findings also confirm that there is a positive attitude towards the use of technology in the classroom and teachers' beliefs might be altered through positive experiences and technology-focused training opportunities requiring longitudinal engagement.

A major takeaway of years of TPACK research is that teachers should receive subject-specific techno-pedagogical training. The fact that only 27% of Hungarian upper-secondary schoolteachers received such training as part of their teacher education compared to the 48% EU average (EU, 2019) signals a slow digital transformation of Hungarian teacher education despite repeated calls for action (Dringó-Horváth & Gonda, 2018; EU, 2018; MDOS, 2016; Öveges & Csizér, 2018).

The current study

The aim of this study was to identify and profile groups among Hungarian K12 teachers based on their techno-pedagogical skills to find out more about the nature of teachers' technology use as well as to find viable development areas respective of each group. Because emergency remote education affected all Hungarian K12 schoolteachers, the present study aims to reflect on online education as well. In order to fulfil this aim, a questionnaire was developed, which 1) adapted Schmidt et al.’s (2009) TPACK questionnaire, 2) proposed ten additional constructs on various aspects of emergency online teaching and 3) collected several background variables because to profile teachers belonging to the clusters found to discover similarities and differences between the groups in order to offer some possible ways of how teachers could develop depending on their corresponding clusters. Therefore, this study sought answers for the following research questions:

  1. How do Hungarian K12 teachers rate their Technological Pedagogical Content Knowledge (TPACK) components?

  2. How many clusters can be observed among Hungarian K12 teachers based on their techno-pedagogical skills level?

  3. Are there any statistically significant differences based on age, gender, teaching experience, qualification, place of work or career model among Hungarian K12 teachers in relation to their techno-pedagogical skills groups?

  4. What are some development possibilities of Hungarian K12 teachers' techno-pedagogical skills?

  5. Are there any statistically significant differences in how Hungarian K12 teachers perceived emergency remote teaching depending on which techno-pedagogical skills group they belong to?

Research design

Designing and validating the data collection instrument

Developing the online questionnaire that became the measurement instrument used in this study began in April 2020 by adapting an existing TPACK questionnaire (Schmidt et al., 2009) and conducting three informal interviews with K12 teachers about their experiences and challenges of the COVID-19 triggered emergency remote teaching. The original TPACK questionnaire contains 37 items, and informants are asked to rate them on a 5-point Likert scale depending on to what extent they feel that the statements are true for them. The items are organized into seven constructs that are the elements of the TPACK framework. As requested by the authors of the original measurement instrument, consent was asked and received from the corresponding author to validate the Hungarian translation and adaptation of the instrument.

The original questionnaire (Schmidt et al., 2009) was translated into Hungarian with some minor modifications. First, the items of the original questionnaire construct, content knowledge (CK) referred to specific content areas such as mathematics or social studies, but for the purposes of this study, they were translated in a way that the items did not include the names of any specific subjects, rather the wording ‘the subject(s) I teach’ was used. For example, the original item I have sufficient knowledge about mathematics was translated as I have sufficient knowledge about the subject(s) I teach. The other modification was that the item pool was mixed up, because in the original questionnaire, items appear one after the other with construct labels, whereas a mixed-up item pool is preferred “to create a sense of variety and to prevent respondents from simply repeating previous answers” (Dörnyei & Csizér, 2012, p. 78).

For the additional constructs, informal interviews were conducted with three volunteering teachers to find common themes about the challenges and difficulties surrounding emergency remote teaching that could later be related to the TPACK scales in March 2020. These themes were turned into questionnaire constructs 8 to 17, summarized in Table 1 (see the Appendix for item lists). All volunteering teachers were women, one of them worked in a secondary grammar school of the capital city, one in an elementary school of the county surrounding the capital city and one in an elementary school in Western Hungary. The development and validation process of the additional questionnaire constructs followed the protocol of Dörnyei and Csizér (2012) as well as some recent Hungarian questionnaire validation studies (Dringó-Horváth & Gonda, 2018; Fekete, 2020a).

Table 1.

The Reliability of the Measurement Instrument Following the Pilot and the Final phase

Construct's name Pilot phase (N = 50) Problem and solution Final Questionnaire (N = 216)
No. of items Cronbach's alpha Number of dimensions No. of items Cronbach's alpha Number of dimensions
1. Technological knowledge 6 0.878 1 6 0.894 1
2. Content knowledge 3 0.568 1 Not reliable with 3, but reliable with two new; altogether 5 items 5 0.835 1
3. Pedagogical knowledge 7 0.903 1 7 0.887 1
4. Pedagogical content knowledge 4 0.926 1 4 0.895 1
5. Technological content knowledge 4 0.784 1 4 0.854 1
6. Technological pedagogical knowledge 9 0.917 1 9 0.910 1
7. Technological pedagogical content knowledge 4 0.898 1 4 0.904 1
8. Teachers' fatigue 5 0.851 1 5 0.868 1
9. Teachers' motivation 5 0.840 1 5 0.791 1
10. Perceived motivation of learners 5 0.821 1 5 0.789 1
11. Perceived fatigue of learners 5 0.705 1 5 0.762 1
12. Giving online feedback 5 0.595 2 1 item opens another dimension, exclusion from large-scale analysis 4 0.753 1
13. Online learning management 5 0.747 1 5 0.769 1
14. Digital skills improvement 5 0.677 2 1 item opens another dimension, but does not with 216 informants 5 0.762 1
15. Perceived quality of students' online learning 7 0.823 1 7 0.851 1
16. Disadvantages of online teaching 7 0.665 2 2 items open another dimension, exclusion from final analysis 5 0.726 1
17. Offline vs. online teacherly self 5 0.667 1 5 0.779 1

Note: For the number of dimensions, the extraction method was principal component analysis.

Before large-scale data collection was launched, the questionnaire was pre-piloted during two think-aloud sessions with two Hungarian K12 teachers, then the online questionnaire was piloted with 50 participants in the first days of April 2020. To be accepted as valid, the questionnaire constructs had to meet two different criteria; 1) for each construct, Cronbach's alpha had to reach a minimum 0.60 cut-off value, and 2) items of the constructs had to load to the same dimension using principal component analysis (Dörnyei & Csizér, 2012). Table 1 summarizes the reliability analyses of the piloted questionnaire, the emerging problems with certain constructs, the proposed solutions to these problems and the reliability analyses of the final questionnaire constructs.

Participants

A total number of 216 Hungarian K12 teachers participated in this study. The sample consisted of 184 females (85%) and 31 males (15%) and one participant preferred not to disclose their gender. This gender ratio keys in with the 90–10% female-male ratio in Illés and Csizér’s (2018, p. 162) representative sample, which involved 1118 foreign language teachers from Hungarian primary and secondary schools. The age of the participants ranged from 24 to 65 years with a mean of 49.20 years (SD = 8.96). The teaching experience of the informants ranged from 1 to 43 years with a mean of 22.97 years (SD = 10.90).

According to the progressive Hungarian career model of teachers (Eurydice, 2020), 2.3% (N = 5) of the informants were in their compulsory traineeship period, 37% (N = 80) were in Teacher 1 status, 42.1% (N = 91) in Teacher 2, 17.6% (N = 38) were Master Teachers and 0.9% (N = 2) were Researcher Teachers. Teachers must submit a written portfolio and take an evaluating exam to be reclassified into their successive categories, but the last two categories are optional. The categories represent the teachers' qualifications as well as teaching experience.

Geographically, 43% (N = 93) of the informants worked in institutions of the capital city or the county surrounding the capital city, 30% (N = 65) worked in Western and 27% (N = 58) in Eastern Hungary. Hungary's most developed region is its central one, that is, the capital city and the surrounding county, followed by the Western and the Eastern counties (Kiss, 2018). Although this three-way categorization overlooks several regional differences, the tendencies might be relevant from the perspective of this study since the general level of regional development might affect teachers' level of technological knowledge by having access to different means of technology.

To gain a general understanding about the technological knowledge level of the participants, they were asked to rate their perceived level of technological knowledge before and after the curfew measures on a 5-point scale. The mean pre-lockdown average was 3.69 (SD = 0.85), while the post-lockdown mean score was 4.20 (SD = 0.65). A paired sample t-test confirmed that there was a statistically significant difference among how much informants knew about technology before and after the lockdown (t = 12.43; Sig. (2-tailed) < 0.001).

Methods of data collection and analysis

Data was collected through an online Google Forms questionnaire. Because of the online nature of data collection, informants remained completely anonymous even for the researcher. Data collection started in April and ended in May 2020. As a result of nationwide school closures in March 2020, the joining rate to digital and general K12 teaching methodology Facebook groups skyrocketed; therefore, the questionnaire was distributed on these platforms with a short message to teachers. The downloaded Excel data pool was coded and then loaded into Statistical Package for the Social Sciences (SPSS) version 22. SPSS was also used for running reliability analyses as well as descriptive and inferential statistical tests.

Results and discussion

How do Hungarian K12 teachers rate their technological pedagogical content knowledge (TPACK) components?

To observe how Hungarian K12 teachers rated the questionnaire constructs, Table 2 was created. It was important not only to include the mean values and standard deviations but the minimum and maximum values informants rated each construct, because these values give hints about the individual differences in the sample. It was hypothesized that there are going to be observable differences prompted by standard deviation and the minimum and maximum values.

Table 2.

Descriptive Statistics of the Questionnaire Constructs (N = 216)

Construct'’s name Minimum Maximum Mean Standard deviation
1. Technological knowledge 1.83 5.00 3.98 0.775
2. Content knowledge 2.00 5.00 4.65 0.448
3. Pedagogical knowledge 1.71 5.00 4.30 0.630
4. Pedagogical content knowledge 1.75 5.00 4.28 0.626
5. Technological content knowledge 2.00 5.00 4.22 0.685
6. Technological pedagogical knowledge 1.89 5.00 4.01 0.715
7. Technological pedagogical content knowledge 1.75 5.00 4.11 0.762
8. Teachers' fatigue 1.00 5.00 2.72 1.050
9. Teachers' motivation 2.20 5.00 4.37 0.596
10. Perceived motivation of learners 1.00 4.80 3.07 0.701
11. Perceived fatigue of learners 1.00 4.80 2.70 0.791
12. Giving online feedback 1.00 5.00 3.52 0.900
13. Online learning management 1.00 5.00 3.33 0.857
14. Digital skills improvement 1.00 5.00 3.31 0.839
15. Perceived quality of students' online learning 1.00 5.00 2.61 0.742
16. Disadvantages of online teaching 1.40 5.00 3.55 0.853
17. Offline vs. online teacherly self 1.40 5.00 3.54 0.911

In average, Hungarian K12 teachers rated their TPACK skills (constructs 1–7) rather high, their content knowledge (M = 4.65; SD = 0.448) being the highest, and technological knowledge (M = 3.98; SD = 0.775) the lowest observed average, yet technological knowledge's higher observed standard deviation, which is true to every other TPACK construct except for content knowledge, suggests many individual differences. The same tendency can be observed among the rest of the constructs. The lowest non-TPACK average is observed for Perceived quality of students' online learning (M = 2.61; SD = 0.742), the highest standard deviation value is observed for Teachers' fatigue (M = 2.72; SD = 1.050), while the highest non-TPACK mean is for Teachers' motivation (M = 4.37; SD = 0.596). These values, although informative, do not tell much about systematic individual differences; therefore, it would be advisable to take these into consideration to allow for drawing more reliable conclusions.

How many clusters can be observed among Hungarian K12 teachers based on their techno-pedagogical skills level?

It was hypothesized that there should be three or four sub-groups among the participants on the basis of how they rated their technological (TK), technological content (TCK), technological pedagogical (TPK) and technological pedagogical content knowledge (TPACK) skills. The hypothesis was supported by observing the minimum and maximum mean values and the frequency of the values participants selected regarding the mentioned constructs (Table 2). The hypothesis was further confirmed by previous research reporting on teachers' very different individual techno-pedagogical skills levels and beliefs on technology inclusion (Chen, 2008; Heitink et al., 2016; Hennessy et al., 2007; MDOS, 2016; Webb & Cox, 2004; Voogt et al., 2013).

To explore if the supposed groups were present in the sample, cluster analysis was used following the protocol of Csizér and Jamieson (2013). First, the cluster forming variables were proposed, these were constructs 1, 5, 6, and 7, that is, those TPACK scales that included technology. A 20% subsample (N = 44) was randomly selected in SPSS for hierarchical clustering. The dendrogram revealed three distinctive clusters that still needed to be confirmed. To check the reliability of the three groups in the sample, now each participant (N = 216) was assigned to their corresponding cluster by SPSS. A further k-means cluster analysis was run on the entire sample that also confirmed the presence of the proposed three groups. As a final reliability check, ANOVA analyses were run on the cluster forming constructs to check whether informants belonging to the three clusters rated the cluster forming constructs differently. The ANOVA analyses confirmed statistically significant differences between the three clusters in the four cluster forming constructs, summarized in Table 3.

Table 3.

ANOVA and Post-hoc Duncan Analyses of the Three Established Clusters in the Sample

Cluster-forming scale Mean F and P Order (Post-hoc Duncan)
Beginners

N = 43
Independent users

N = 89
Advanced users

N = 84
1. Technological Knowledge 3.06 3.79 4.66 F = 164.74 P < 0.001 1 < 2 < 3
2. Technological Content Knowledge 3.26 4.15 4.78 F = 203.80 P < 0.001 1 < 2 < 3
3. Technological Pedagogical Knowledge 2.99 3.87 4.69 F = 364.25 P < 0.001 1 < 2 < 3
4. Technological Pedagogical Content Knowledge 2.99 4.00 4.79 330.48 P < 0.001 1 < 2 < 3

Note: Group 1: Beginners, Group 2: Independent users, Group 3: Advanced users.

Thus, based on the technology-related variables, three groups were established among Hungarian K12 teachers labeled as teachers reported displaying 1) beginner, 2) independent, and 3) advanced level techno-pedagogical knowledge. The proportion of K12 teachers in the groups could also be determined, 20% (N = 43) of the teachers perceived they were beginners, 41% (N = 89) independent, and 39% (N = 84) advanced users of techno-pedagogical skills.

Are there any statistically significant differences based on age, gender, teaching experience, qualification, place of work or career model among Hungarian K12 teachers in relation to their techno-pedagogical skills groups?

To illustrate the differences between the established groups based on age, gender, teaching experience, qualification, place of work or career model, Table 4 was created, which contains the collected descriptive statistics of the selected background variables. As far as the group sizes are concerned, one fifth of the participants belong to Beginners, two fifths to Independent, and two fifths Advanced users – size being the biggest difference observed among all the variables.

Table 4.

Profiling the Techno-pedagogical Groups

N = 216 Beginners Independent users Advanced users
In the cluster (group size and proportion) N = 43 N = 89 N = 84
20% 41% 39%
Average age (years) 49.72 (8.54) 49.34 (8.73) 48.79 (9.46)
Gender (Sample total: 85% female, 15% male) Female = 91% Female = 81% Female = 86.5%
Male = 9% Male = 18% Male = 13.5%
Not disclosed = 1%
Teaching experience in number of years 23.14 (10.57) 22.54 (10.79) 23.35 (11.29)
Qualification 53% BA 43% BA 43% BA
47% MA 55% MA 57% MA
2% PhD
Career model 2% Trainee 2% Trainee 2% Trainee
40% Teacher I 33% Teacher I 40% Teacher I
46% Teacher II 47% Teacher I 35% Teacher II
12% Master Teacher 16% Master Teacher 23% Master Teacher
0% Researcher Teacher 2% Researcher Teacher 0% Researcher Teacher
Place of work 56% Central Hungary 33% Central Hungary 48% Central Hungary
30% Western Hungary 37% Western Hungary 23% Western Hungary
14% Eastern Hungary 30% Eastern Hungary 29% Eastern Hungary

To confirm whether the groups portray any statistically significant differences, Chi-square tests were run on the nominal background variables. The only statistically significant difference between the groups (Chi-square = 10.63; df = 4; Sig. = 0.31) was found in the places of work. Significantly less people belong to Cluster 1 (Beginners) from Eastern Hungary than from the other two regions (Adjusted residual = −2.1), and significantly less people belong to Cluster 2 (Independent users) from Central Hungary (Adjusted residual = −2.6), but this might entirely be due to the relatively small sub-sample size. The groups showed no further significant differences, thus teachers in the three groups do not systematically differ in their age, gender, teaching experience, qualification and career model are concerned.

The Chi-square test confirmed that the established clusters based on Hungarian K12 teachers' techno-pedagogical skills are rather homogenous. Most background variables do not seem to be predictors of techno-pedagogical knowledge, and how teachers perceive their techno-pedagogical skills level rather depends on whether they opt for being digital citizens (Bayne & Ross, 2011), further influenced by beliefs about technology use (Voogt et al., 2013) and teacher education or training programs (Dringó-Horváth & Gonda, 2018).

Although the TPACK scales provide more complex grounds for analysis, informants were also asked to rate their perceived level of digital skills before and after emergency remote teaching on a 5-point scale. On the one hand, the TPACK questionnaire items were mixed up to maximize the validity of the measurement instrument; on the other hand, they measured teachers' perceptions without being able to reflect on the hypothesized boost in their skills growth. The reason for asking informants to rate their digital skills levels on two scales one after the other was to check whether they perceived any changes in their digital skills knowledge and to observe the scale of skills growth. As summarized in Table 5, Hungarian K12 teachers belonging to the established techno-pedagogical clusters reported statistically significant digital skills growth confirmed by paired sample t-tests.

Table 5.

Digital Skills Improvement before and during Emergency Remote Teaching

N = 216 Beginners Independent users Advanced users
Self-reported average digital skills before distant education 2.74 (0.66) 3.57 (0.58) 4.30 (0.67)
Self-reported average digital skills after distant education 3.56 (0.59) 4.13 (0.46) 4.60 (0.56)
Growth of the mean average +0.82 +0.56 +0.30
Statistically significant change? yes yes yes
t = 8.51 t = 9.77 t = 4.73
Sig. 2-tailed: P < 0.001 Sig. 2-tailed: P < 0.001 Sig. 2-tailed: P < 0.001

In all three clusters, the averages grew and the individual differences (as confirmed by the standard deviation values) lowered. A noteworthy positive tendency is that teachers in the Beginners cluster reported similar digital skills knowledge after distance education that of the Independent users reported before emergency remote teaching. On average, teachers in all three groups perceived that they gained digital skills throughout distance education, and the sudden complete shift required most effort to be invested from teachers belonging to the Beginners group in catching up with online possibilities. Yet, even though they caught up with the pre-remote teaching average of the Independent users, it does not mean that Beginner teachers have become very confident and competent in their digital skills. It can also be said, confirmed by the statistically significant skills gains of all three groups, that a major positive attribute of the exigency of teaching via technology elevated the levels of techno-pedagogical knowledge, creating new standards.

What are some development possibilities of Hungarian K12 teachers' techno-pedagogical skills?

In order to propose possible ways of development respective of the clusters, path analyses are proposed (Székelyi & Barna, 2002). The subsample sizes allowed for running such analyses (Dörnyei & Csizér, 2012, p. 82) as the smallest subgroup was close to the suggested minimum 50 subsample size.

Because the final goal is TPACK development (Koehler et al., 2014; Mishra & Koehler, 2006; Schmidt et al., 2009) – being able to synthesize technological, content, pedagogical, technological content, pedagogical content and technological pedagogical knowledge – the path ends were TPACK in all clusters. Then, backward, the strongest predictors were listed until there were no more statistically significant predictors found. This resulted in three proposed pathway models. Although, in the end, regression analyses were used to create the models, Pearson correlations were also run on the clusters to confirm the paths, which is a widely used and recommended additional reliability measure (Székelyi & Barna, 2002).

Figure 1 illustrates the path model for TPACK skills growth in Cluster 1. The strongest predictor of TPACK development is Technological Pedagogical Knowledge (TPK), the knowledge of how teachers can facilitate and support the learning processes of their students by using technological tools, such as e-mailing or learning management systems (e.g., Edmodo, Google Classroom, Microsoft Teams). For teachers reporting Beginner levels of techno-pedagogical skills, it mainly takes learning more about such specific technologies that enable keeping in touch with students asynchronously to show TPACK development, while technological content knowledge and pedagogical content knowledge components are not part of the model at all. Furthermore, for Beginners to develop TPK knowledge, the strongest predictor is technological knowledge (TK), that is, learning about how to operate computers and computer programs, such as e-mailing or learning management systems. This model suggests that Beginners need help on the very operational levels of technology, and technological knowledge is focused on educational technologies only.

Fig. 1.
Fig. 1.

Path analysis of TPACK in cluster 1

Citation: Journal of Adult Learning, Knowledge and Innovation 5, 2; 10.1556/2059.2022.00056

Figure 2 proposes a path model for Independent users that would ultimately result in their TPACK development. The biggest difference between the models of Beginners and Independent users is the absence of TK as a significant predicting factor in the latter model, suggesting that Cluster 2 teachers are confident users of computers and computer programs, and they do not have basic computer operational hindrances. The strongest predicting factor of TPACK is TPK, similar to Cluster 1, but with a lower Beta value. The most forceful predicting factor of TPK is TCK, that is, beyond finding ways of keeping in touch with learners (TPK), Cluster 2 teachers try to find or create relevant digital content (e.g., finding video demonstrations, creating presentations) to facilitate the knowledge construction of learners. This created or researched content (TCK) predicts how Cluster 2 teachers select the means and methods of delivering it to their learners (TPK).

Fig. 2.
Fig. 2.

Path analysis of TPACK in cluster 2

Citation: Journal of Adult Learning, Knowledge and Innovation 5, 2; 10.1556/2059.2022.00056

A possible way of development for Cluster 2 teachers could be to experiment more with online classes and teaching approaches that deviate more from traditional, offline classroom practices. That is, as opposed to implementing lessons online that resemble traditional classroom approaches such as frontal teaching in front of the camera or checking homework by asking students to provide the correct answers one by one, they could experiment with more progressive possibilities such as flipped classroom techniques and project work. Naturally, the subject they teach comes heavily into play in this respect, and Cluster 2 teachers should always rely on their generally good level of subject-specific knowledge and the criticality towards new approaches that originate from their subject-specific knowledge in determining which progressive practices to experiment with non-arbitrarily.

Figure 3 shows the path model for Advanced users. The biggest difference of this model is that TPACK is predicted by three different knowledge domains. Similar to Independent users, TPK is a predictor of TPACK, but TCK emerges as another predicting factor, and not as a predicting factor or TPK. This suggests that Cluster 3 teachers have an array of techno-pedagogical knowledge and knowledge about educational technology at their disposal compared to what is seen on Cluster 2 teachers' model, where the created or found digital content predicted the means of facilitating students' learning. It is interesting to see that in the case of Advanced users, pedagogical content knowledge (PCK), i.e., how teachers are able to teach their specific subject(s) with the most appropriate methods, is also part of TPACK predicting factors, although it is a non-technological scale.

Fig. 3.
Fig. 3.

Path analysis of TPACK in cluster 3

Citation: Journal of Adult Learning, Knowledge and Innovation 5, 2; 10.1556/2059.2022.00056

According to the path model, pedagogical content knowledge development also results in TPACK development, prompting that for Cluster 3 teachers, being able to teach their subject through appropriate digital technologies is considered to be part of their general, subject-specific teaching methodology. Cluster 3 teachers' further development could target trying out a vast array of digital teaching possibilities because their PCK, TCK and TPK knowledge as TPACK development predictors ensure sufficient pedagogical reasoning and learner-centeredness for the implementation of various progressive techno-pedagogical instructional methods.

Are there any statistically significant differences in how Hungarian K12 teachers perceived emergency remote teaching depending on which techno-pedagogical skills group they belong to?

Questionnaire constructs 8 to 17 were formed based on interviews with three Hungarian K12 teachers to identify common themes in what their perceptions were with digital education. The reason for including these scales was to relate them to the established clusters to learn more about the dimensions of emergency remote teaching in relation to techno-pedagogical skills. For this, ANOVAs and post-hoc Duncan analyses were run on questionnaire constructs 8 to 17 summarized in Table 6.

Table 6.

ANOVA and Post-hoc Duncan Analyses of Questionnaire Constructs 8 to 17

Scale Mean F and P Order (Post-hoc Duncan)
Beginners Independent users Advanced users
8. Teachers' fatigue no statistically significant differences among clusters
Total mean = 2.72 (SD = 1.05)
9. Teachers' motivation 4.14 4.34 4.52 F = 6.36 P = 0.002 1, 2 < 2, 3
10. Perceived motivation of learners 2.82 3.04 3.22 F= 5.08 P = 0.007 1, 2 < 2, 3
11. Perceived fatigue of learners 2.52 2.78 2.90 F = 4.07 P = 0.018 1, 2 < 2, 3
12. Giving online feedback 3.40 3.32 3.79 F = 6.65 P = 0.002 1, 2 < 3
13. Online learning management 3.01 3.14 3.70 F = 14.44 P < 0.001 1, 2 < 3
14. Digital skills improvement no statistically significant differences among clusters
Total mean = 3.31 (SD = 0.84)
15. Perceived quality of students' online learning 2.46 2.53 2.76 F = 3.20 P = 0.043 1, 2 < 2, 3
16. Disadvantages of online teaching no statistically significant differences among clusters
Total mean = 3.55 (SD = 0.85)
17. Offline vs. online teacherly self 3.15 3.64 4.06 F = 17.37 P < 0.001 1 < 2 < 3

Note: “,” stands for non-significant, “<” stands for statistically significant difference.

Based on the findings, the constructs can be grouped into four categories. The first category can be established based on the constructs showing no statistically significant differences among clusters (1 = 2 = 3), including constructs Teachers' fatigue, Digital skills improvement, Disadvantages of online teaching. This suggests that Hungarian K12 teachers experienced emergency remote teaching similarly irrespective of their techno-pedagogical groups in these respects. Although in the construct-forming interviews all three teachers expressed that they experience fatigue, Hungarian K12 teachers were only moderately tired specifically because of the online environment (M = 2.72; SD = 1.05), but the relatively high standard deviation value suggests many individual differences.

The second category consists of the constructs where teachers with Advanced techno-pedagogical skills rated the constructs significantly differently than Beginners and Independent users (1, 2 < 3). This category includes Giving online feedback and Online learning management. Giving online feedback is the only construct that was rated higher by Beginners than Independent users, but the exact methods and practices teachers used to give online feedback were not included in the study. All in all, the more advanced techno-pedagogical skills teachers reported, the more confident they were in the possibilities of providing online feedback on student work as well the more they feel confident in online learning management such as contacting learners quickly, letting them know clearly what tasks they should do and keeping track of online tasks submissions.

The third category consist of constructs Teachers' motivation in which constructs there is a statistically significant difference between Beginners and Advanced users, but Independent users can be included in both groups (1, 2 < 2, 3). In all cases, the mean values of clusters are always higher in relation to the level of techno-pedagogical skills. Teachers' motivation was rather high in all three clusters, although the motivation of learners was much lower than the teachers', and the perceived quality of students' online learning was also rather low. It is by no means surprising that the more advanced teachers' are with technology, the more motivated they are, and the more motivated they are, the more motivated they perceive their learners are with digital learning, yet the levels of teachers' motivation and the perceived levels of learner motivation and learning quality do not approximate one another.

Finally, the fourth and last category includes construct Offline vs. online teacherly self, where there is a statistically significant difference between all clusters (1 < 2 < 3). In this construct, teachers were asked to rate statements that compared offline teaching to online teaching. It was found that the more advanced techno-pedagogical clusters teachers belong to, the more they favor offline, in-class education as they feel teaching in the online space to be less effective, despite their high techno-pedagogical skills. It is important to note that although teachers in all clusters seem to prefer face-to-face instruction, this does not mean that they prefer not developing or using their techno-pedagogical skills. Although teachers in all clusters prefer face-to-face instruction to online instruction, this finding might be linked to the high levels of TPACK knowledge also confirmed by the TPACK pathway analysis (Fig. 3) of Cluster 3 teachers, because using digital technologies seems to be part of their general teaching methodology, thus being an expansion of instruction possibilities rather than a substitution. In this respect, the fact that the higher techno-pedagogical skills self-reported by teachers, the more they are aware of the fact that technology should be integrated into everyday teaching practices as opposed to substitute traditional education.

Conclusions

Summary of main findings

This non-representative study found that Hungarian K12 teachers can be classified into three distinctive groups as far as their techno-pedagogical skills are concerned on the basis of the technological scales (TK, TCK, TPK, TPACK) of Schmidt et al.'s (2009) TPACK questionnaire. In the study (N = 216), approximately 20% of the informants were Beginners, 41% were Independent, and 39% were Advanced users of techno-pedagogical tools and methods. Chi-square crosstab tests confirmed that the only statistically significant difference between the groups concerned where teachers worked, as there were less teachers among the Beginners from Eastern Hungary, and there were less Independent users from Central Hungary; however, this might be due to a small sub-sample size. Other than this, no statistically significant difference was found between Hungarian K12 teachers in the different groups based on their age, gender, teaching experience, qualification, and career model.

Teachers in all groups reported gains in their techno-pedagogical skills, and this gain proved to be statistically significant. It seems that Covid-19-triggered emergency remote teaching set up new techno-pedagogical standards for teachers, but there is still much room for improvement in teacher education and training programs (Dringó-Horváth & Gonda, 2018; Öveges & Csizér, 2018), because Beginners, for example, only now learnt how to use devices for keeping in touch with their students. Practicing teachers should be offered relevant, preferably subject-specific technological-methodological training courses, and such courses should be offered in teacher education programs, because as it was seen, techno-pedagogical levels were irrespective of age, qualification, and teaching experience. Furthermore, in the present study, it was found that the more advanced group teachers belonged to, the higher they rated their non-technological TPACK knowledge components: content knowledge (CK) and pedagogical knowledge (PK).

With TPACK growth as the ultimate goal (Koehler et al., 2014; Mishra & Koehler, 2006; Schmidt et al., 2009), path analyses were conducted to find development possibilities of the techno-pedagogical knowledge of Hungarian K12 teachers specific to their current techno-pedagogical levels. Beginners need to develop on the very basic levels of using technology to be able to supervise and correspond with their learners, and are rather far from complex, pedagogically reasoned implementations of online classes. Independent users seem to be the ones who try to put their traditional pedagogical practices into the online space by creating or finding relevant digital content to facilitate learning processes that are not necessarily implemented in the forms of synchronous online classes. Teachers in this group need to develop in the facilitation and implementation of online classes.

Advanced users are able to synthesize their subject pedagogical knowledge and teaching online classes. In this respect, for them, TPACK development could be trying out even more online pedagogical methods and practices with a sufficient level of criticality while always keeping their learners' needs in focus to test the effectiveness of certain digital teaching possibilities. Independent and Advanced users, furthermore, could also help their colleagues in their techno-pedagogical skills development, because development is most effective if it is related to the specific teaching-learning context, including educational level and specific school subjects. It is also important to add that, with the exception of Beginners, learning more about technology and digital devices would not result in techno-pedagogical gains (Dringó-Horváth & Gonda, 2018; Pintér, 2019).

Finally, as far as the remote teaching experiences and perceptions of Hungarian K12 teachers in the different groups are concerned, the most notable difference was that Advanced techno-pedagogical teachers rated items of the construct, Offline vs. online teacherly self the highest, meaning that they are the ones who most want to return to the classrooms. This phenomenon can be explained by what was observed on their TPACK path models, that is, Advanced users use techno-pedagogical methods and tools as part of their everyday, in-class teaching practices. In all three groups, teachers' motivation for online teaching was much higher than the perceived motivation of learners, and teachers reported that they observed growing fatigue of their learners. The lowest mean values were attached to the quality of students' learning throughout emergency remote education.

Limitations

Limitations of this study concern sampling, sample size and the online nature of data collection. These limitations are mainly due to Covid-19, as during spring curfew measures, there was no other way of collecting data than distributing an online questionnaire in various teaching methodology groups on Facebook. Although the number of members joining such Facebook groups grew rapidly because of Covid-19, it is possible that rather those teachers filled in the questionnaire who had more positive beliefs about technology and techno-pedagogy, or the ones who had had more experiences with it. However, the disposition of teachers (gender, age, experience, etc.) in each established techno-pedagogical group suggests that the sample, although non-representative and not generalizable, reflects Hungarian K12 teachers well.

Additionally, confirmatory factor analysis was not carried out on the scales adapted (translated into Hungarian) from the original TPACK questionnaire (Schmidt et al., 2009), nevertheless data analysis was preceded by a pilot phase, and construct reliability was confirmed with principal component analyses and Cronbach's alpha values. It is also important to mention that cluster analysis, lacking a statistical model, is exploratory rather than confirmatory (Csizér & Jamieson, 2013), thus the results represent and reflect the current sample only and are by no means generalizable. Nonetheless it is hoped that the findings are transferable to similar contexts.

Future directions

It would be worthwhile replicating the study using the same data collection instrument with more participants. A larger sample size could give grounds for comparing the findings of the two studies and would allow for more precise categorization of teachers based on their techno-pedagogical levels. A larger sample size would also allow for more detailed data analysis, for example by comparing specific school levels and regions of Hungary.

In the future, as a possible follow-up and expansion of this hypothesis-testing approach, interviews with selected teachers from all three clusters could be conducted to find out more about their specific ways of conducting online classes, their positive and negative experiences, the contents of their lessons, the ways they perceive their learners were engaged and the possibilities of testing their learners through digital devices.

It would also be important to offer subject-specific digital teaching methodology courses as part of teacher training and teacher education and following them up would be a viable continuation of this study. Based on the present findings, these courses could be personalized to a certain extent, that is, bearing in mind the already existing level of techno-pedagogical knowledge of teachers.

Finally, to help teachers more closely and reliably determining their current techno-pedagogical levels and to give them specific ideas and ways of how to develop their knowledge, the TPACK scales of the present data collection instrument could be used as a self-check survey that would determine which group the participants belong to. The teachers filling in the survey would also be data providers for observing the continuously changing normal and cut-off values that could help determine which cluster they belong to. Such an online tool would be profitable for both teachers and researchers.

Funding

Supported by the ÚNKP-20-3 New National Excellence Program of the Ministry for Innovation and Technology.

About the author

Imre Fekete is an instructor and academic developer at Budapest Business School. He is also a doctoral candidate in Language Pedagogy at Eötvös Loránd University. His main research area is the inclusion possibilities of ICT devices in the university context.

References

  • Bayne, S. , & Ross, J. (2011). ‘Digital native’ and ‘digital immigrant’ discourses: A critique. In R. Land , & S. Bayne (Eds.), Digital difference: Perspectives on online learning (pp. 159169). Sense Publishers.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Becker, H. (1999). Internet use by teachers: Conditions of professional use and teacher-directed student use. Center for Research on Information Technology and Organizations.

    • Search Google Scholar
    • Export Citation
  • Caena, F. (2014). Initial teacher education in Europe: An overview of policy issues. European Commission. https://ec.europa.eu/assets/eac/education/experts-groups/2014-2015/school/initial-teacher-education_en.pdf.

    • Search Google Scholar
    • Export Citation
  • Chai, C. S. , Koh, J. H. L. , Tsai, C.-C. , & Tan, L. L. W. (2011). Modeling primary school pre-service teachers’ Technological Pedagogical Content Knowledge (TPACK) for meaningful learning with information and communication technology (ICT). Computers & Education, 57(1), 11841193. https://doi.org/10.1016/j.compedu.2011.01.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C. H. (2008). Why do teachers not practice what they believe regarding technology integration? The Journal of Educational Research, 102(1), 6575. https://doi.org/10.3200/JOER.102.1.65-75.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Csizér, K. , & Jamieson, J. (2013). Cluster analysis. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics. Blackwell Publishing. https://doi.org/10.1002/9781405198431.wbeal0138.

    • Search Google Scholar
    • Export Citation
  • Dörnyei, Z. , & Csizér, K. (2012). How to design and analyze surveys in SLA research? In A. Mackey , & S. Gass (Eds.), Research methods in second language acquisition: A practical guide (pp. 7494). Wiley-Blackwell.

    • Search Google Scholar
    • Export Citation
  • Dringó-Horváth, I. , & Gonda, Zs. (2018). Tanárjelöltek IKT-kompetenciájának jellemzői és fejlesztési lehetőségei [The measurement of the ICT-competence in teacher training]. Training and Practice, 16(2), 2148. https://doi.org/10.17165/TP.2018.2.2.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2015). Digital competences: Self-assessment grid. European Commission. https://europass.cedefop.europa.eu/sites/default/files/dc-en.pdf.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2018). Communication from the commission to the European parliament, the council, the European economic and social committee and the committee of the regions on the digital education action plan. European Commission. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52018DC0022&from=EN.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2019). 2nd survey of schools: ICT in education: Hungary country report. European Commission. https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=57806.

    • Search Google Scholar
    • Export Citation
  • Eurydice (2020). Teachers and education staff. https://eacea.ec.europa.eu/national-policies/eurydice/content/teachers-and-education-staff-34_en.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2017). Learner responsibility and homework quality in secondary EFL blending. Training Practice, 15(1–2), 221242. https://doi.org/10.17165/TP.2017.1-2.13.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2020a). Information and communications technology (ICT) literacy of Hungarian English majors: A validation study. Journal of Adult Learning, Knowledge and Innovation, 19.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2020b). Information and communications technology use of Hungarian English majors: A large-scale questionnaire study. Journal of Foreign Language Education and Technology, 5(2), 251275.

    • Search Google Scholar
    • Export Citation
  • Főző, A. L. , & Racsko, R. (2020). Az iskolai digitális érettség értékelésének lehetőségei [The possibilities of assessment of the schools’ digital maturity]. Civil Szemle, 17(3), 93113.

    • Search Google Scholar
    • Export Citation
  • Heitink, M. , Voogt, J. , Verplanken, L. , van Braak, J. , & Fisser, P. (2016). Teachers’ professional reasoning about their pedagogical use of technology. Computers & Education, 101, 7083. https://doi.org/10.1016/j.compedu.2016.05.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hennessy, S. , Deaney, R. , Ruthven, K. , & Winterbottom, M. (2007). Pedagogical strategies for using the interactive whiteboard to foster learner participation in school science. Learning Media and Technology, 32(3), 283301. https://doi.org/10.1080/17439880701511131.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Illés, É. , & Csizér, K. (2018). A nyelvtanárok válaszai. In E. Öveges , & K. Csizér (Eds.), Vizsgálat a köznevelésben folyó idegennyelv-oktatás kereteiről és hatékonyságáról: Kutatási jelentés [Research into the framework and effectiveness of foreign language instruction in Hungarian public education: A report] (pp. 165185). Oktatási Hivatal. https://www.oktatas.hu/pub_bin/dload/sajtoszoba/nyelvoktatas_kutatasi_jelentes_2018.pdf.

    • Search Google Scholar
    • Export Citation
  • Kiss, L. B. (2018). A jövedelmi helyzet, az élettel való elégedettség és a környezeti terhelés összefüggései hazánk régióiban [Correlations between income position, satisfaction with life and the environmental load in various regions in Hungary]. Polgári Szemle, 14(1–3), 273286. https://doi.org/10.24307/psz.2018.0820.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koehler, M. J. , Mishra, P. , Kereluik, K. , Shin, T. S. , & Graham, C. R. (2014). The technological pedagogical content knowledge framework. In J. M. Spector , M. D. Merrill , J. Elen , & M. J. Bishop (Eds.), Handbook of research on educational communications and technology: Fourth edition (4th ed., pp. 101111). Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koehler, M. J. , Shin, T. S. , & Mishra, P. (2012). How do we measure TPACK? Let me count the ways. In R. N. Ronau , C. R. Rakes , & M. L. Niess (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 1631). IGI Global.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • MDOS (2016). Magyarország Digitális Oktatási Stratégiája [The digital educational Strategy of Hungary]. Magyarország Kormánya. https://digitalisjoletprogram.hu/files/55/8c/558c2bb47626ccb966050debb69f600e.pdf.

    • Search Google Scholar
    • Export Citation
  • Mishra, P. , & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 10171054. https://doi.org/10.1111/j.1467-9620.2006.00684.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pintér, M.T. (2019). Digitális kompetenciák a felsőoktatásban [Digital competences in higher education]. Modern Nyelvoktatás, 25(1), 4758.

    • Search Google Scholar
    • Export Citation
  • Niess, M. L. (2011). Investigating TPACK: Knowledge growth in teaching with technology. Journal of Educational Computing Research, 44(3), 299317. https://doi.org/10.2190%2FEC.44.3.c.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Öveges, E. , & Csizér, K. (Eds.) (2018). Vizsgálat a köznevelésben folyó idegennyelv-oktatás kereteiről és hatékonyságáról: Kutatási jelentés [Research into the framework and effectiveness of foreign language instruction in Hungarian public education: A report]. Oktatási Hivatal. https://www.oktatas.hu/pub_bin/dload/sajtoszoba/nyelvoktatas_kutatasi_jelentes_2018.pdf.

    • Search Google Scholar
    • Export Citation
  • Papp-Danka, A. (2013). Digitális bennszülött vagy digitális állampolgár? [Digital native or digital citizen?]. In J. Ollé , D. Lévai , K. Domonkos , O. Szabó , A. Papp-Danka , D. Czirfusz , L. Habók , R. Tóth , A. Takács , & I. Dobó (Eds.), Digitális állampolgárság az információs társadalomban [Digital citizenship in the information society] (pp. 3341). ELTE Eötvös Kiadó.

    • Search Google Scholar
    • Export Citation
  • Peters, M. A. , Rizvi, F. , Gibbs, P. , Gorur, R. , Hong, M. , Hwang, Y. , Zipin, L. , Brennan, M. , Robertson, S. , Quay, J. , Malbon, J. , Taglietti, D. , Barnett, R. , Chengbing, W. , McLaren, P. , Apple, R. , Papastephanou, M. , Burbules, N. , Jackson , … Misiaszek, L. (2020). Reimagining the new pedagogical possibilities for universities post-Covid-19. Educational Philosophy and Theory, 144. https://doi.org/10.1080/00131857.2020.1777655.

    • Search Google Scholar
    • Export Citation
  • Rienties, B. , Lewis, T. , O’Dowd, R. , Rets, I. , & Rogaten, J. (2020). The impact of virtual exchange on TPACK and foreign language competence: Reviewing a large-scale implementation across 23 virtual exchanges. Computer Assisted Language Learning, 128. https://doi.org/10.1080/09588221.2020.1737546.

    • Search Google Scholar
    • Export Citation
  • Schmidt, D. A. , Baran, E. , Thompson, A. D. , Mishra, P. , Koehler, M. J. , & Shin, T. (2009). Technological Pedagogical Content Knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123149. https://doi.org/10.1080/15391523.2009.10782544.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Székelyi, M. , & Barna, I. (2002). Túlélőkészlet az SPSS-hez: Többváltozós elemzési technikákról társadalomkutatók számára [Survival kit for SPSS: Inferential statistical analyses for researchers of social sciences]. Typotex Kiadó.

    • Search Google Scholar
    • Export Citation
  • Tongori, Á. (2012). Az IKT műveltség fogalmi keretének változása [The changing theoretical framework of ICT literacy]. Iskolakultúra, 22(11), 3447.

    • Search Google Scholar
    • Export Citation
  • Tóth-Mózer, Sz. (2017). Digitális nemzedék a tanulási folyamatban: Középiskolások internethasználati és tanulási preferenciái, énképe és digitális kompetenciája [Digital generation in learning: Internet use, learning preferences, self-image and digital competences of secondary learners]. ELTE Eötvös Kiadó.

    • Search Google Scholar
    • Export Citation
  • Voogt, J. , Fisser, P. , Pareja Roblin, N. , Tondeur, J. , & van Braak, J. (2013). Technological pedagogical content knowledge - a review of the literature. Journal of Computer Assisted Learning, 29(2), 109121. https://doi.org/10.1111/j.1365-2729.2012.00487.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Webb, M. , & Cox, M. (2004). A review of pedagogy related to information and communications technology. Technology, Pedagogy and Education, 13(3), 235285. https://doi.org/10.1080/14759390400200183.

    • Crossref
    • Search Google Scholar
    • Export Citation

APPENDIX

The translation of the reliable scales of questionnaire constructs 8 to 17

Please decide to what extent do you agree with the following statements. When you answer, please think of your experiences with online and hybrid teaching because of the pandemic.

Please rate the sentences like this: 1: completely disagree, 2: rather disagree; 3: neither disagree nor agree; 4: rather agree; 5: completely agree.

Teachers' fatigue

  • I feel I'm growing tired.

  • I feel that I'm less and less enthusiastic.

  • It's more and more difficult to get around working.

  • I'm getting bored of teaching.

  • Teaching is less and less a source of joy.

Teachers' motivation

  • I do everything I can to implement good lessons.

  • I give my everything to my classes.

  • I prepare for my classes conscientiously.

  • It's important to me to implement good lessons.

  • I prepare for my classes enthusiastically.

Perceived motivation of students

  • My students' enthusiasm is unbroken.

  • My students prepare for my classes enthusiastically.

  • My students are active during the lessons.

  • My students enjoy my classes.

  • My students do all their tasks conscientiously.

Perceived fatigue of learners

  • My students are getting less and less active in classes.

  • My students are becoming less enthusiastic.

  • My students are not very keen on participating actively in classes.

  • My students seem bored in my classes.

  • My students are aiming at investing minimum effort only.

Giving online feedback

  • I give more personalised feedback for my students.

  • I give more feedback than usual for my students.

  • I give more detailed feedback for my students.

  • I give feedback to my students in several different ways.

Online learning management

  • I can solve emerging problems quickly.

  • Misunderstandings are easy to settle.

  • It's easy to give straightforward instructions.

  • It's easy to monitor students' work.

  • It's easy to keep track of students' work.

Digital skills improvement

  • During the emergency remote period, I learnt more about the subjects I teach.

  • During the emergency remote period, I learnt more about teaching methodology.

  • During the emergency remote period, my digital skills improved.

  • Since working online, I pay more attention to my work.

  • Since working online, I'm more detail-oriented towards my work.

Perceived quality of students' learning

  • My students seem to be learning faster.

  • My students seem to be learning more.

  • My students seem to be learning more thoroughly.

  • My students seem to be learning more independently.

  • My students seem to be learning more effectively.

  • My students' learning seems to be more goal-oriented.

  • The critical skills of my learners seem to be developing.

Disadvantages of online teaching

  • Digital teaching requires the use of too many different platforms.

  • Digital teaching makes it more difficult to separate private and professional life.

  • Digital teaching blurs the boundaries of private and professional life.

  • It's unhealthy how much time I spend in front of the computer because of digital teaching.

  • While working from home, my attention is easily drawn to other things I have to deal with.

Offline vs. online teacherly self

  • It's very different to be a good teacher online than in the classroom.

  • It's difficult to transfer my classroom practices to the digital space.

  • I can't teach with my usual efficiency online.

  • It's difficult to implement similar quality lessons online than in the classroom.

  • I feel more confident in the classroom than online.

  • Bayne, S. , & Ross, J. (2011). ‘Digital native’ and ‘digital immigrant’ discourses: A critique. In R. Land , & S. Bayne (Eds.), Digital difference: Perspectives on online learning (pp. 159169). Sense Publishers.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Becker, H. (1999). Internet use by teachers: Conditions of professional use and teacher-directed student use. Center for Research on Information Technology and Organizations.

    • Search Google Scholar
    • Export Citation
  • Caena, F. (2014). Initial teacher education in Europe: An overview of policy issues. European Commission. https://ec.europa.eu/assets/eac/education/experts-groups/2014-2015/school/initial-teacher-education_en.pdf.

    • Search Google Scholar
    • Export Citation
  • Chai, C. S. , Koh, J. H. L. , Tsai, C.-C. , & Tan, L. L. W. (2011). Modeling primary school pre-service teachers’ Technological Pedagogical Content Knowledge (TPACK) for meaningful learning with information and communication technology (ICT). Computers & Education, 57(1), 11841193. https://doi.org/10.1016/j.compedu.2011.01.007.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, C. H. (2008). Why do teachers not practice what they believe regarding technology integration? The Journal of Educational Research, 102(1), 6575. https://doi.org/10.3200/JOER.102.1.65-75.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Csizér, K. , & Jamieson, J. (2013). Cluster analysis. In C. A. Chapelle (Ed.), The encyclopedia of applied linguistics. Blackwell Publishing. https://doi.org/10.1002/9781405198431.wbeal0138.

    • Search Google Scholar
    • Export Citation
  • Dörnyei, Z. , & Csizér, K. (2012). How to design and analyze surveys in SLA research? In A. Mackey , & S. Gass (Eds.), Research methods in second language acquisition: A practical guide (pp. 7494). Wiley-Blackwell.

    • Search Google Scholar
    • Export Citation
  • Dringó-Horváth, I. , & Gonda, Zs. (2018). Tanárjelöltek IKT-kompetenciájának jellemzői és fejlesztési lehetőségei [The measurement of the ICT-competence in teacher training]. Training and Practice, 16(2), 2148. https://doi.org/10.17165/TP.2018.2.2.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2015). Digital competences: Self-assessment grid. European Commission. https://europass.cedefop.europa.eu/sites/default/files/dc-en.pdf.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2018). Communication from the commission to the European parliament, the council, the European economic and social committee and the committee of the regions on the digital education action plan. European Commission. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52018DC0022&from=EN.

    • Search Google Scholar
    • Export Citation
  • EU [European Union] (2019). 2nd survey of schools: ICT in education: Hungary country report. European Commission. https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=57806.

    • Search Google Scholar
    • Export Citation
  • Eurydice (2020). Teachers and education staff. https://eacea.ec.europa.eu/national-policies/eurydice/content/teachers-and-education-staff-34_en.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2017). Learner responsibility and homework quality in secondary EFL blending. Training Practice, 15(1–2), 221242. https://doi.org/10.17165/TP.2017.1-2.13.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2020a). Information and communications technology (ICT) literacy of Hungarian English majors: A validation study. Journal of Adult Learning, Knowledge and Innovation, 19.

    • Search Google Scholar
    • Export Citation
  • Fekete, I. (2020b). Information and communications technology use of Hungarian English majors: A large-scale questionnaire study. Journal of Foreign Language Education and Technology, 5(2), 251275.

    • Search Google Scholar
    • Export Citation
  • Főző, A. L. , & Racsko, R. (2020). Az iskolai digitális érettség értékelésének lehetőségei [The possibilities of assessment of the schools’ digital maturity]. Civil Szemle, 17(3), 93113.

    • Search Google Scholar
    • Export Citation
  • Heitink, M. , Voogt, J. , Verplanken, L. , van Braak, J. , & Fisser, P. (2016). Teachers’ professional reasoning about their pedagogical use of technology. Computers & Education, 101, 7083. https://doi.org/10.1016/j.compedu.2016.05.009.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hennessy, S. , Deaney, R. , Ruthven, K. , & Winterbottom, M. (2007). Pedagogical strategies for using the interactive whiteboard to foster learner participation in school science. Learning Media and Technology, 32(3), 283301. https://doi.org/10.1080/17439880701511131.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Illés, É. , & Csizér, K. (2018). A nyelvtanárok válaszai. In E. Öveges , & K. Csizér (Eds.), Vizsgálat a köznevelésben folyó idegennyelv-oktatás kereteiről és hatékonyságáról: Kutatási jelentés [Research into the framework and effectiveness of foreign language instruction in Hungarian public education: A report] (pp. 165185). Oktatási Hivatal. https://www.oktatas.hu/pub_bin/dload/sajtoszoba/nyelvoktatas_kutatasi_jelentes_2018.pdf.

    • Search Google Scholar
    • Export Citation
  • Kiss, L. B. (2018). A jövedelmi helyzet, az élettel való elégedettség és a környezeti terhelés összefüggései hazánk régióiban [Correlations between income position, satisfaction with life and the environmental load in various regions in Hungary]. Polgári Szemle, 14(1–3), 273286. https://doi.org/10.24307/psz.2018.0820.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koehler, M. J. , Mishra, P. , Kereluik, K. , Shin, T. S. , & Graham, C. R. (2014). The technological pedagogical content knowledge framework. In J. M. Spector , M. D. Merrill , J. Elen , & M. J. Bishop (Eds.), Handbook of research on educational communications and technology: Fourth edition (4th ed., pp. 101111). Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koehler, M. J. , Shin, T. S. , & Mishra, P. (2012). How do we measure TPACK? Let me count the ways. In R. N. Ronau , C. R. Rakes , & M. L. Niess (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 1631). IGI Global.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • MDOS (2016). Magyarország Digitális Oktatási Stratégiája [The digital educational Strategy of Hungary]. Magyarország Kormánya. https://digitalisjoletprogram.hu/files/55/8c/558c2bb47626ccb966050debb69f600e.pdf.

    • Search Google Scholar
    • Export Citation
  • Mishra, P. , & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 10171054. https://doi.org/10.1111/j.1467-9620.2006.00684.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pintér, M.T. (2019). Digitális kompetenciák a felsőoktatásban [Digital competences in higher education]. Modern Nyelvoktatás, 25(1), 4758.

    • Search Google Scholar
    • Export Citation
  • Niess, M. L. (2011). Investigating TPACK: Knowledge growth in teaching with technology. Journal of Educational Computing Research, 44(3), 299317. https://doi.org/10.2190%2FEC.44.3.c.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Öveges, E. , & Csizér, K. (Eds.) (2018). Vizsgálat a köznevelésben folyó idegennyelv-oktatás kereteiről és hatékonyságáról: Kutatási jelentés [Research into the framework and effectiveness of foreign language instruction in Hungarian public education: A report]. Oktatási Hivatal. https://www.oktatas.hu/pub_bin/dload/sajtoszoba/nyelvoktatas_kutatasi_jelentes_2018.pdf.

    • Search Google Scholar
    • Export Citation
  • Papp-Danka, A. (2013). Digitális bennszülött vagy digitális állampolgár? [Digital native or digital citizen?]. In J. Ollé , D. Lévai , K. Domonkos , O. Szabó , A. Papp-Danka , D. Czirfusz , L. Habók , R. Tóth , A. Takács , & I. Dobó (Eds.), Digitális állampolgárság az információs társadalomban [Digital citizenship in the information society] (pp. 3341). ELTE Eötvös Kiadó.

    • Search Google Scholar
    • Export Citation
  • Peters, M. A. , Rizvi, F. , Gibbs, P. , Gorur, R. , Hong, M. , Hwang, Y. , Zipin, L. , Brennan, M. , Robertson, S. , Quay, J. , Malbon, J. , Taglietti, D. , Barnett, R. , Chengbing, W. , McLaren, P. , Apple, R. , Papastephanou, M. , Burbules, N. , Jackson , … Misiaszek, L. (2020). Reimagining the new pedagogical possibilities for universities post-Covid-19. Educational Philosophy and Theory, 144. https://doi.org/10.1080/00131857.2020.1777655.

    • Search Google Scholar
    • Export Citation
  • Rienties, B. , Lewis, T. , O’Dowd, R. , Rets, I. , & Rogaten, J. (2020). The impact of virtual exchange on TPACK and foreign language competence: Reviewing a large-scale implementation across 23 virtual exchanges. Computer Assisted Language Learning, 128. https://doi.org/10.1080/09588221.2020.1737546.

    • Search Google Scholar
    • Export Citation
  • Schmidt, D. A. , Baran, E. , Thompson, A. D. , Mishra, P. , Koehler, M. J. , & Shin, T. (2009). Technological Pedagogical Content Knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123149. https://doi.org/10.1080/15391523.2009.10782544.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Székelyi, M. , & Barna, I. (2002). Túlélőkészlet az SPSS-hez: Többváltozós elemzési technikákról társadalomkutatók számára [Survival kit for SPSS: Inferential statistical analyses for researchers of social sciences]. Typotex Kiadó.

    • Search Google Scholar
    • Export Citation
  • Tongori, Á. (2012). Az IKT műveltség fogalmi keretének változása [The changing theoretical framework of ICT literacy]. Iskolakultúra, 22(11), 3447.

    • Search Google Scholar
    • Export Citation
  • Tóth-Mózer, Sz. (2017). Digitális nemzedék a tanulási folyamatban: Középiskolások internethasználati és tanulási preferenciái, énképe és digitális kompetenciája [Digital generation in learning: Internet use, learning preferences, self-image and digital competences of secondary learners]. ELTE Eötvös Kiadó.

    • Search Google Scholar
    • Export Citation
  • Voogt, J. , Fisser, P. , Pareja Roblin, N. , Tondeur, J. , & van Braak, J. (2013). Technological pedagogical content knowledge - a review of the literature. Journal of Computer Assisted Learning, 29(2), 109121. https://doi.org/10.1111/j.1365-2729.2012.00487.x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Webb, M. , & Cox, M. (2004). A review of pedagogy related to information and communications technology. Technology, Pedagogy and Education, 13(3), 235285. https://doi.org/10.1080/14759390400200183.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collapse
  • Expand
  • Top

The author instruction is available in PDF.
Please, download the file from HERE

 

Senior editors

Editor(s)-in-Chief: Helga DORNER

Associate Editors 

  • Csíkos, Csaba (Eötvös Lorand University, Hungary)
  • Csizér, Kata (Eötvös Lorand University, Hungary)
  • Dorner, Helga (Central European University, Hungary)

 

Editorial Board

  • Basseches, Michael (Suffolk University, USA)
  • Billett, Stephen (Griffith University, Australia)
  • Cakmakci, Gültekin (Hacettepe University, Turkey)
  • Damsa, Crina (University of Oslo,Norway)
  • Dörnyei, Zoltán (Nottingham University, UK)
  • Endedijk, Maaike (University of Twente, The Netherlands)
  • Fejes, Andreas (Linköping University, Sweden)
  • Guimaraes, Paula (University of Lisbon, Portugal)
  • Halász, Gábor (Eötvös Loránd University, Hungary)
  • Hansman, Catherine A. (Cleveland State University, USA)
  • Kroeber, Edith (Stuttgart University, Germany)
  • Kumar, Swapna (University of Florida, USA)
  • MacDonald, Ronald (University of Prince Edward Island, Canada)
  • Marsick, Victoria J. (Columbia University, USA)
  • Martensson, Katarina (Lund University, Sweden)
  • Matei, Liviu (Central European University, Hungary)
  • Matyja, Malgorzata (Wroclaw University of Economics, Poland)
  • Mercer, Sarah (University of Graz, Austria)
  • Nichols, Gill (University of Surrey, UK)
  • Nokkala, Terhi (University of Jyvaskyla, Finland)
  • Ostrouch-Kaminska (Uniwersytet Warminsko-Mazurski, Poland)
  • Pusztai, Gabriella (University of Debrecen, Hungary)
  • Ramesal, Ana (Barcelona University, Spain)
  • Reischmann, Jost (Bamberg University, Germany)
  • Rösken-Winter, Bettina (Humboldt, Germany)
  • Ryan, Stephen (Waseda University, Japan)
  • Török, Erika (Pallasz Athéné University, Hungary)
  • Wach-Kakolewicz, Anna (Poznan University of Economics and Business, Poland)
  • Watkins, Karen E. (University of Georgia, Athens, Georgia)

 

 

 

Journal of Adult Learning, Knowledge and Innovation
Editorial Office
Eötvös Loránd University
Faculty of Education and Psychology
Institute of Research on Adult Education and Knowledge Management
Address: 1075 Budapest, Kazinczy u. 23-27. HUNGARY

E-mail: jalki@ppk.elte.hu

2020  
CrossRef Documents 0
CrossRef
Cites
8
CrossRef H-index 4
Days from submission to acceptance 130
Days from acceptance to publication 222

 

2019  
WoS
Cites
4
CrossRef
Documents
14

 

Journal of Adult Learning, Knowledge and Innovation
Publication Model Gold Open Access
Submission Fee none
Article Processing Charge none
Subscription Information Gold Open Access

Journal of Adult Learning, Knowledge and Innovation
Language English
Size A4
Year of
Foundation
2016
Volumes
per Year
1
Issues
per Year
2
Founder Eötvös Loránd Tudományegyetem
Founder's
Address
H-1053 Budapest, Hungary Egyetem tér 1-3.
Publisher Akadémiai Kiadó
Publisher's
Address
H-1117 Budapest, Hungary 1516 Budapest, PO Box 245.
Responsible
Publisher
Chief Executive Officer, Akadémiai Kiadó
ISSN 2631-1348 (Online)

Monthly Content Usage

Abstract Views Full Text Views PDF Downloads
Feb 2022 0 0 0
Mar 2022 0 0 0
Apr 2022 0 0 0
May 2022 0 0 0
Jun 2022 0 0 0
Jul 2022 0 104 46
Aug 2022 0 32 18