Measuring aptitude

In order to explore how quickly and easily one can learn foreign languages and to predict the likelihood of the success of learners, several attempts have been made to design aptitude tests. The most well-known instruments are the Modern Language Aptitude Test (MLAT; Carroll & Sapon, 1959), and the Pimsleur Language Aptitude Battery (PLAB; Pimsleur, 1966). Within these instruments, the MLAT is the most influential and it has several adaptations (Ottó & Nikolov, 2003). For example, as mentioned by Ottó and Nikolov, there is a version designed for primary school students (EMLAT; Carroll & Sapon, 1967), and a version for native speakers of Japanese (LABJ; Sasaki, 1993). Besides, there are also tests designed for the American military (DLAB; Petersen & Al-Haik, 1976; VORD; Parry & Child, 1990).

There are two aptitude tests designed to measure Hungarian learners' language aptitude and the present research used the aptitude test called Magyar Egységes Nyelvérzékmérő Teszt or Hungarian Language Aptitude Test (MENYÉT or HUNLAT; Ottó, 2002). This is the only aptitude test to measure Hungarian students' aptitude besides INYÉT 6 (Kiss & Nikolov, 2005), which is based on the tasks of the HUNLAT, but INYÉT 6 is particularly designed for 12-year-old test takers. Since the intention was not to measure the aptitude of children but adults, the HUNLAT was chosen as the instrument to measure aptitude in this research. The strength of these instruments is that these are validated by various researchers in the field, but a notable weakness of these aptitude tests is that researchers did not conceptualise the construct of language aptitude before designing these instruments. As it highlighted by Dörnyei (2005), designing an aptitude test and then formulating the definition of language aptitude may lead to various non-identical – and even contrasting – conceptualisations of aptitude. In this respect, construct validity in these tests may be questionable to a certain extent.

Research questions

After reviewing the relevant literature and establishing the research niche, two research questions were formulated. This research paper is going to present the findings of a small-scale study on the possible relationships between aptitude and language learning habits of the selected participants. The research attempted to find answer to the following research questions:

1. What age and gender differences seem to be in the characterised aptitude of the selected sample?

2. What kind of learning habits do high aptitude and low aptitude participants have?

The researcher proposed the following hypotheses prior to the research: there are significant age and gender differences at least in one component of the aptitude test and there is a difference in the learning habits of high-aptitude and low-aptitude participants.

Participants

The participants of this research (N = 27) were chosen from four Hungarian institutes except for the eldest participant, who is not studying anymore, but is still an active user of English and thus included in the analysis because the aim was to study a more diverse sample. There were altogether 6 males and 21 females, and their age ranged from 18 to 59 (M = 22.59, SD = 7.68). The first language of the participants is Hungarian, and their second language is English. All of the 27 participants took part in completing the aptitude test and six of them were asked to participate in the in-depth interview. Convenience sampling was applied, and participants were chosen from four different institutes because the intention was to see multiple variations of possible scores (i.e., diversity of the selected sample). The most important background information of the selected six interviewees can be seen in Table 1.

Instruments

The HUNLAT as the quantitative instrument employed is to be discussed in detail in the following subsection. After examining the HUNLAT (see Appendix A), the qualitative instrument, an in-depth interview is going to be described (see Appendix B).

Data collection procedures

Measuring students' aptitude took approximately one hour in each occasion. The completion of the second, third, and fourth task of the aptitude test within the three think-alouds were recorded with the participants' consent for later analysis of the cognitive processes, but are not analysed in the present paper. The interview was conducted on a one-to-one basis with each of the six participants in Hungarian, the mother tongue of the interviewees and the interviewer. Some of the interviews' transcriptions were done on the spot during the interviews, some were recorded with the participants' consent and later transcribed verbatim. One interview transcription was double-checked by member-checking after which no modifications were necessary in the transcript. The transcripts yielded a fairly rich database of approximately 6,000 words and the English excerpts are the author's translations.

Data analysis

Descriptive and inferential statistics were calculated with the SPSS software version 22. As an internal consistency (reliability) measure, Cronbach alpha coefficient was calculated (see Table 2), and for each of the four tasks, the reliability coefficients are relatively high. After calculating the internal consistency value, no items were left out as even the lowest reliability coefficient is acceptable and exceeds 0.60. According to Dörnyei (2016), the Cronbach alpha coefficient can be lower due to the complexity of L2 acquisition processes, but there is only a problem when the scale does not reach 0.60. Consequently, the constructs within the aptitude test are considered to be a reliable.

In an attempt to answer the research questions, it was necessary to categorise students to two groups; those who have high aptitude and those who have low aptitude based on the scores on the aptitude test. For characterising the aptitude of participants, defining a cut-off score was indispensable. It was performed in a norm-referenced way: the average score was calculated, and then students who met the average (M = 48.48, see Table 3) or scored above the average were considered to be high aptitude scorers and those students who scored below average were regarded as low aptitude scorers. The reliability of this interpretation was also checked with cluster analysis in SPSS. Two groups were created with cluster analysis thus having students fall into either of the two categories. There were two instances where the researcher had to make a decision as to which category should students be assigned because they fall exactly in between the two categories. In these two instances, they were placed to the high aptitude category, though the cluster analysis put them to the low aptitude category. Considering the qualitative part, content analysis was used to code the emerging themes, establish patterns, and make interpretations (Dörnyei, 2016).

Quantitative results

The analysed descriptive and inferential statistics of the aptitude test are going to be discussed. This paper shall start with discussing the descriptive statistics of the scores on HUNLAT, presented in Table 3.

Paired samples t-tests were calculated to analyse the data gained from the aptitude test. The results of the paired samples t-tests show that there is a statistically significant difference between the mean values of Words in sentences and Grammar analysis, t(26) = 6.78, P < 0.001 (two-tailed) and between the mean values of Words in sentences and Vocabulary learning, t(26) = −6.08, P < 0.001 (two-tailed). Grammar analysis and Vocabulary learning show the highest mean values (M = 14.00 for Grammar analysis and M = 14.22 for Vocabulary learning), which probably means that for this sample, analysing phrases and word memorisation were the two easiest exercises within the four tasks. The most difficult task appeared to be Words in sentences as it can be seen from the lowest mean value (M = 8.22). Regarding previous research (e.g., Ottó & Nikolov, 2003; Sáfár & Kormos, 2008), Words in sentences, which measures grammatical sensitivity, is likely to be the most challenging part of the aptitude test. Furthermore, it is important to mention that nobody of the 27 participants could reach the maximum points in the Words in sentences task (Max = 18), whereas in the other three tasks at least one person has scored maximum points. Sáfár and Kormos's results are very similar to the findings of this study: in their study, the control group (N = 21) and the experimental group (N = 40) both showed similar tendencies to the present sample: the highest mean score achieved was on the Grammar analysis task, the second highest on Vocabulary learning, while the lowest mean value was achieved on the Words in sentences task (Sáfár & Kormos, 2008).

The highest standard deviations are in Hidden sounds (SD = 5.18) and Vocabulary learning (SD = 5.04), whereas the lowest standard deviation can be seen in the scores of Words in sentences (SD = 3.65). The high standard deviation on Vocabulary learning can be interpreted as that there seem to be some students who managed to memorise the words quite well, but there were also some who did not. The low standard deviation along with its lowest mean value on the Words in sentences task can be interpreted as if almost every student managed to score relatively similarly low on this task. Although the participants scored relatively high in this aptitude test – altogether, the mean value of the total scores is 48.48, which is 60.6% – nobody could actually reach the maximum of 80 points of the 27 participants. The highest total score on the HUNLAT was 78, and the lowest was 24.

In addition, the means of the achieved scores were calculated to see the differences in the scores based on participants' age. Ottó and Nikolov (2003) also examined HUNLAT scores with regard to age differences. The results of the t-test are presented in Table 4 below. The present paper shall elaborate on the answer to the first research question in the following paragraphs. Students were grouped into two groups and the cut point was set to 22, this way there were 13 students at the age of 22 or above (≥22) and there were 14 students below 22 (<22). According to the independent samples t-test, the performance of the older students was higher on average (M = 56.76) compared to younger students (M = 40.78) and this was a statistically significant difference based on the probability measure, t(25) = 3.17, P < 0.005 (two-tailed). Moreover, the standard deviation from the average in the case of older students was higher (SD = 16.23) than that of younger learners (SD = 9.23). Hidden sounds, Grammar analysis and Words in sentences tasks all displayed significant differences between ages, including the total score on the HUNLAT. Cohen's delta (Cohen, 1988) was also calculated to study the effect size and it proved to be large on the tasks except for the Vocabulary learning task where were no statistically significant differences regarding age. Ottó and Nikolov (2003) showed that in their sample, younger students scored higher on the aptitude test as compared to older learners and the difference was significant. The assumption that can be made from this information is that other factors – such as social background and education – also play a crucial role in language aptitude. Sáfár and Kormos (2008) has shown, for example, that language learning experience has an effect on LA.

After examining the age differences, in an attempt to answer the second part of the first research question, possibly statistically significant gender differences were analysed in the aptitude test. Although there were almost four times as many females than males in the present study, it is still worthwhile to analyse the differences. The independent samples t-test (see Table 5) showed that the difference was only statistically significant in the Words in sentences task t(25) = 4.73, P < 0.001 (two-tailed). Consequently, males seemed to achieve significantly higher mean score (M = 12.83) on the Words in sentences task as compared to females (M = 6.90). Cohen's delta was calculated as well to analyse the effect size since the two groups compared were unequal: d = 2.28, which belongs to large effect size (Cohen, 1988).

Qualitative findings

In the process of data analysis of the interviews, several themes emerged, which were grouped into the following categories: emotions (mainly positive: openness, enjoyment, being interested) and experiences (both positive and negative), self-regulation (including motivation; strengths and weaknesses, goals, attitude; language learning strategies; language learning habits), language usage (frequency and context). It must be noted that while conducting the interview, there was a particular focus on participants' English language learning.

Considering the emerging theme of emotions and experiences, the majority of the participants claimed to be interested in British and American culture. Mária, Bendegúz, and Attila mentioned some positive emotions (e.g., openness, enjoyment, and being interested) and experiences in connection with learning English: “it gives a feeling of openness towards meeting foreigners and getting to know them and their culture” (Mária, p. 2). Bendegúz has given an account of enjoyment: “I like learning it because it's enjoyable to learn this language and getting to know more and more about English.” (p. 1). Attila said that he likes learning English because he is interested in it: “I started to be interested in learning English at a very young age, in my childhood when I was in primary school.” (p. 1). Edgár emphasised that he likes English because it is simple and logical, but he does not like that there are twelve tenses. Ilona added that she likes the sound of it and she also likes the culture. György mentioned that he likes learning English because it is a very good feeling when he can understand a lot of things in English. From this data, it can be concluded that the selected participants have mainly positive emotions regarding learning English.

Ilona said that although she stopped learning languages, her goal is: “to be able to communicate with people who don't speak Hungarian and to be able to understand lyrics” (p. 1). György said that his aim is to obtain the other half of the language exam certificate. Edgár admitted that he does not have a goal in learning English other than being able to communicate with people while working abroad.

Mária said the following regarding her strengths and weaknesses: “I would say that grammar is easy for me, I can understand it quite quickly. What causes difficulties is listening and vocabulary.” Ilona emphasised that “tenses caused difficulties, and communication always went smoothly.” (p. 1). György mentioned that “My strength is in studying words, because I always cram words and terms, my weakness is grammar and ‘the gift of the gab’ [speaking].” (p. 2). Edgár said the complete opposite: “Grammar is always easy for me, while vocabulary learning is not… and this applies to all of the languages I study.” (p. 2).

Regarding goals and attitude within self-regulation, three of them (Mária, Bendegúz, and György) admitted that they had to consult a private tutor of English when it came to challenges. Mária is now preparing to obtain a BME type language exam certificate and in order to achieve this, she believes that the help of the private teacher is necessary. Bendegúz decided to consult a private tutor when he wanted to obtain both the NATO STANAG 2.2.2.2 certificate – equivalent of level B2 based on the Common European Framework of Reference for Languages (CEFR; Council of Europe, 2011) – and the GE ECL C1 certificate. György admitted that he had five private teachers altogether. Four of the five teachers have prepared him for specific challenges, two of them for obtaining a language exam certificate, two of them for other exams and one in primary school to catch up with the material. Attila mentioned that he only had a private tutor in primary school when he started to be interested in English. His objective with the help of the private teacher was to develop his skills. This is the same for Edgár, who turned to a private teacher because he wanted to catch up with the material and develop his skills. Ilona said she only studied German with the help of a private teacher.

What is also interesting in the emerging theme of language usage is that four participants (Edgár, Mária, Bendegúz, and Ilona) do not use English every day. They say that they use English quite often, but apparently it does not play a main part of their lives every single day. Naturally, Attila, – who works at British Council as an English language tester, works as a translator and interpreter and studies English applied linguistics – is using English every single day. György is also using English every day on his computer for collecting data for statistics, finding streams, looking up and checking things on the web.

In connection with their learning strategies within self-regulation, Bendegúz emphasised that he organises his studying beforehand to make it as diverse as possible, for example, using tasks involving pronunciation practice, vocabulary practice, looking up grammar rules thoroughly and using flashcards. Mária claimed that she likes watching series in English either with English subtitles or Hungarian, reading books in English and playing on computer games which are not translated to Hungarian. Edgár mentioned that he likes watching TV channels and films to learn languages; besides, he likes to read newspapers. Attila reported that he is always creating chunks or units when he wants to study, he learns these units one-by-one and after this, he tries to say out loud the whole to himself thus understanding all the material. Ilona said that she used to facilitate her learning by listening to music while studying. György said the opposite, he does not like listening to music because it causes a great distraction for him and he does not employ any strategies for studying.

Turning to the learning habits of the students – still considered to be in self-regulation –, the aim was to see whether these participants use explicit (conscious) or implicit (unconscious) learning mechanisms and which type of learning they prefer. Two groups were formed based on the mechanisms the researcher hypothesised the students usually apply: the first group is the explicit group including György and Bendegúz. The second group contains four individuals who use rather implicit learning mechanisms: Mária, Attila, Ilona and Edgár. It must be noted that the mechanisms the students usually apply may in some cases differ from the mechanisms they find more efficient.

It is engaging to see the opinion and learning habits of the eldest active language learner of this study. It can be concluded that he finds it essential to employ explicit learning methods for practical challenges, for example, when preparing for a language exam. However, he is convinced that implicit learning methods are more efficient if students are to master a foreign language at a native-like level.

Continuing the analysis of their learning habits, the level of consciousness involved in their learning was in the focus (if they plan or monitor their learning or not). Within the first group (explicit-users), Bendegúz, Edgár and György do not plan a logical order that they follow in their studies/homework assigned. However, Attila claimed that he likes to start with the most difficult task because he sees it as a challenge, while Mária and Ilona said that they like(d) to begin with the easiest tasks and with tasks that they like(d) in order to maintain motivation.

Regarding the habits of reading, Bendegúz and Ilona said that they are paying attention to pronunciation when reading, György only concentrates on the main message of the reading. Attila argued that he is focussing on new words and on reading critically, while Mária only concentrates on the main message of the reading. Edgár said that he is focussing on the main message but can be easily distracted. Although in some cases the perceived efficiency of either learning mechanisms did not seem to correspond to what was generally used by the participants, it can be claimed that in most cases used habits and favoured habits corresponded with each other.