Abstract
Content analysis plays a pivotal role in the field of educational science. This paper delves into an examination of studies within the discipline that have employed a deductive approach when applying this method. Our research focus revolved around the thematic patterns present in the corresponding scientific discourse and the techniques utilized for deductive content analysis.
We conducted a systematic literature review within the Web of Science database to identify journal articles employing a theory-driven approach to content analysis. The results of the investigation revealed that empirical studies in the domains of health studies, professional development, and learning enhancement seldom adhere exclusively to deductive reasoning during content analysis. Instead, they typically blend deductive reasoning with inductive coding processes.
Introduction
Content analysis is considered as a widely used method within the realm of education sciences. Although theoretical and methodological sources frequently underscore the disparities between the different logical branches of this particular method (Charmaz, 2008; Krippendorff, 2019) and of research activities in general (Bryman, 2016; Creswell & Creswell, 2017), this type of reasoning is less commonly emphasized in the empirical studies of this discipline. Accordingly, it is worthwhile to explore the utilization of the different logical pathways within the field of educational science. Thus, our initial focus was to investigate different approaches within content analysis, with a specific emphasis on deductive reasoning.
As outlined above, the possible ways of content analysis can be classified based on their underlying types of reasoning. Consequently, one can distinguish between deductive, inductive, and abductive (and/or retroductive – cf. Chiasson, 2005) approaches. Although all of these lines employ different codes during their examination, a significant distinction lies in their origin. In particular, the deductive approach derives its labels from existing theories (literature), whereas inductive reasoning extracts codes from empirical data or the phenomenon under analysis. The third approach, abductive reasoning, whose raison d'etre has been scrutinized by numerous scholars (Hoffmann, 1999; Psillos, 1996; for further writings, see Sántha, 2008), aims at providing a plausible explanation for the observed phenomena.
It is noteworthy that scientific papers frequently amalgamate these three branches, either in narrowly defined analyses, such as the central report section, or in other parts like the introduction and discussion. These integrated studies can employ different modes of reasoning sequentially, such as transitioning from a deductive phase to an inductive stage, or these arguments can also coexist simultaneously. This is exemplified in various types of hybrid grounded theory approaches (GTA), as explored by Corbin and Strauss (2008) and Charmaz (2014). In the former example, the different subtypes deviate from the original inductive version presented by Glaser and Strauss (1967) by incorporating deductive components. Nevertheless, the extent and the way of their application in the research process can exhibit significant variations among the different branches of GTA and other content analysis methods that are at least partially inductive. Their categorisation is further complicated by the fact that GTA (and even content analysis in general – cf. Krippendorff, 2019) is often categorised as a (partially) abductive method rather than inductive (see e.g. Charmaz, 2008).
From the previously described complex set of content analysis approaches, we intend to examine those studies, where it was assumable that they apply a purely deductive reasoning for their examination. Highlighting the deductive approach stems from the fact that there are fewer methodological guidelines available for deductive-qualitative content analysis in studies that consolidate qualitative research methods (Kibiswa, 2019). Consequently, the overarching aim of our paper is to assess the status of deductive content analysis, as outlined by Krippendorff (2019), in the publications within the global educational-related scientific discourse, employing a systematic literature review. We chose the approach of Krippendorff as a sign of this logic as he can considered as one of the main theorists of deductive content analysis (cf. Bengtsson, 2016) and as a pioneer in systematizing the research process. Besides this, his definition on content analysis (Krippendorff, 2019) – more precisely, the necessity of replicability – is also connected to deductive approach (see also Gavora, 2015).
In pursuit of this goal, we have formulated the following research questions:
What thematic patterns can be identified within the articles that have employed deductive content analysis?
How does the deductive reasoning appear in articles that explicitly state the utilization of deductive analysis in their research?
Methods
In designing such a study, relevant databases can be divided into two main categories. Firstly, there are dedicated portals that exclusively compile education-related research, such as Education Resources Information Center (ERIC) and FIS Bildung Literaturdatenbank. On the other hand, there are general sources that aggregate publications from various disciplines, such as Web of Science, Scopus, and EBSCO, which can also be explored for this purpose.
The collection of data for the systematic review was based on the Web of Science (WoS) databases (the analysis was performed on 18 August 2023). The choice of this population was motivated by the known selectivity and quality of the WoS content (cf. Garfield, 2007), as opposed to the broader coverage of other data sources, as we aimed at representing the most consolidated body of knowledge on our research topic as a methodological issue. Sampling of the WoS databases was based on an iterative search strategy – in accordance with the main principles of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol (Page et al., 2021).
TS = “content analys*” AND (deductive OR Krippendorff OR “theory driven” OR theory-driven)
Whereby TS is a combination of several bibliographic descriptors consisting of the Title, the Keywords and the Abstract. In other words, an occurrence of a phrase conforming to the query in any of these fields will qualify the paper as a positive result for our search.
The query was run on the WoS Core Collection (including the Science Citation Index, the Social Science Citation Index, and the Arts & Humanities Citation Index). This first step resulted in a sample of approximately two thousand papers (n = 1,829). In the next step, we identified the distribution of these papers over research fields covered by the source database, namely, WoS Subject Categories, and selected those fields that we considered the most relevant to our research question. These include Education Educational Research, Education Scientific Disciplines, and Education Special as governing categories. Based on this step, we narrowed down the results to these categories adding them as further criteria to the search query in the new iteration of data retrieval.
Refining the results with the three categories yielded n = 177 (75%), n = 82 (33%) and n = 6 (2.5%) papers, respectively. The percentages signal the share of each category within the first-round sample; the sum of percentages exceeds 100%, indicating that papers belong to more than one Subject Category. In sum, the second round yielded a refined sample of n = 244 papers. This sample was considered final (relative to the search process and before screening) as we did not want to decrease topic coverage by further diminishing the sample with algorithmic means, letting the screening process to take over the task of achieving the necessary precision. As the final step, the bibliographic and citation record, and the abstracts of the sample documents were retrieved from the WoS databases.
Tool
We utilized an online spreadsheet generated from the results of the WoS database query to record the decisions made by Coder A and Coder B. The worksheet also incorporates the final result of the process following consultations between the coders, forming the basis for the review presented below.
Results
We refined the incoming data by limiting it to the past five years (from 2019 to 2023) and specifying English as the language for analysis. We exclusively considered complete research studies while excluding writings of other genres (e. g. early access writings). As a result, we obtained a total of 153 relevant hits from data published after 2019. Subsequently, we scrutinized the average citation count, which stood at 4.23. Therefore, we focused our attention on articles with a minimum of 5 citations, resulting in a selection of 46 articles. Following a thorough review of the abstracts, we eliminated false positive results, specifically those which appeared to employ both deductive and inductive analysis methods.
The reliability analysis of the selection process employed semantic content analysis (Janis, 1965). Two of the authors, referred to as Coder A and Coder B, served as independent reviewers responsible for determining the inclusion of papers based on their abstracts. The primary selection criterion was the clear presence of purely deductive-logic-related elements, which were coded using a binary (yes/no) variable. Subsequently, Krippendorff's alpha was computed with a bootstrap sample (N = 10,000), considering standard minimum and optimal values, specifically 0.67 and 0.8 (Hayes & Krippendorff, 2007, as also discussed in Amidei, Piwek, & Willis, 2018).
After the coding process, four disagreements occurred and the examination of inter-coder reliability resulted in a high value (α = 0.83, for more details, see Appendix 1). These have been discussed by Coder A and Coder B until reaching the level of perfect agreement. Following this refinement, we were left with 25 studies where the term “deductive” was frequently used to describe the methodology.
An examination of the full texts of the papers enabled us to categorize them into three primary thematic groups, which exhibit some degree of overlap; the intersectional studies were categorized according to their primary focus, based on the decision of the authors of this paper. The emerging themes (sets) and the associated papers are introduced below (Table 1).
The three main thematic groups and the related articles
The responses to research question 1 are delineated through the three thematic directions and their corresponding studies. The answers to research question 2 are subsequently provided within the description of each study. To enhance clarity and accessibility, these responses are not presented in a distinct subsection but are directly linked to the pertinent articles.
Health studies, medicine, physical activity and well-being
The methodological approaches of the reviewed studies within the first group can be divided into two main categories. The first one relates to work of Elo and Kyngäs (2008), an article on the possible applications of qualitative content analysis in nursing studies. The second set of articles is associated with another, albeit more discipline-specific, methodological literature.
Elo and Kyngäs (2008) methodological work served as the basis for Holt et al. (2019), who directed their attention toward psychosocial well-being by investigating the impact of a physical activity intervention in two classes of varying ages in two different schools. Their research approach, a qualitative case study, encompassed semi-structured focus group interviews along with additional participant observations. The authors underscored that the analysis of empirical data predominantly followed a deductive approach, evident from the outset of the results section through the utilization of a categorization matrix that included frequencies and time periods. Nevertheless, the subsequent part of the result section also entailed a qualitative-inductive analysis of student quotations.
Following the same approach, Hörberg et al. (2019) investigated the challenges of registered nurses in the first year of their practice. They conducted semi-structured interviews with a combined sample, using both purposive and snowball sampling methods. The applied content analysis predominantly followed a deductive approach, drawing from the social learning theory of communities of practice. Nevertheless, additional 'other' category emerged in their analysis, along with the inductive subcategories. However, the text was subsequently recoded using a (partially) inductive approach to gain a deeper understanding of the phenomena under examination and to give in-depth description of the theoretical categories.
The third study, Keats et al. (2019) conducted a qualitative content analysis involving childhood cancer survivors, their families, and family physicians. The research employed semi-structured interviews to uncover the key elements essential for nurturing a survivorship care plan. While the paper initially introduces deductive content analysis as its methodology, later inductive approach appears (with additional reference to Elo & Kyngäs, 2008). The inclusion of open coding and emerging categories suggests the incorporation of inductive reasoning as well.
In the last study of this subgroup (Stacey et al., 2020), content analysis was applied to the verbatim transcripts of focus group interviews. The authors indicated that they conducted a “deductive and collaborative” analysis (p. 1). Following the initial deductive phase, they proceeded to a second phase involving ‘memo development’, ‘axial coding,' and ‘selective coding’ which suggests the utilization of inductive reasoning, specifically a hybrid form of GTA; accordingly, the authors indicated the work of Corbin and Strauss (1990) as an additional methodological guide besides the central one of this article group.
In the second subset, Cheung et al. (2019) aimed at examining the advantages and disadvantages associated with evaluating the performance of medical trainees. Their investigation relied on the analysis of semi-structured interviews conducted within a hospital setting. The conversations involving 12 residents and 13 faculty members were subject to a direct also known as theory-driven or deductive content analysis approach, which is integral to both the interview methodology and the subsequent analysis. In the Method section, the authors referred to the work of Hsieh and Shannon (2005) regarding qualitative content analysis.
In the next study, Kolar and Janke (2019) performed a deductive content analysis using transcripts from focus group interviews to uncover relevant changes in the participants' perspectives regarding specific concepts associated with a patient care guideline, namely the Pharmacists' Patient Care Process, among nursing students. However, the coding process, particularly the grouping of the labels and concept definitions after the coding) and the referred methodological work (Thomas, 2006) may imply the application of inductive logic.
Roca et al. (2020) examined reflective journals to investigate the aspects of the learning process among nurses, particularly focusing on reflective competence. They employed a theory-driven analysis using the 5Rs Reflective Writing Scale as a framework to assess their development. The authors applied a computer-assisted form of ‘Classical Content Analysis’, following the principles Leech and Onwuegbuzie (2007). This approach led to a theory-driven analysis that demonstrated the rigorous application of a deductive methodology, including elements such as mutual exclusiveness.
Pham et al. (2022) employed semi-structured interviews to investigate the integration of Mental Health First Aid training into medical, nursing, and pharmacy curricula in Australia. The authors indicated deductive content analysis (Crowe, Inder, & Porter, 2015) as their method, which was performed by one of the authors.
Professional development
Similar to health studies, articles focusing on professional development also utilized the methodological insights offered by Elo and Kyngäs (2008). The intersectional nature of the analysed articles, discussed in the Methods section of our study, partly explains this connection. However, as detailed below, a non-medical paper also incorporated these insights. Additionally, the contributions of Hsieh and Shannon (2005) and Mayring (2014) were influential in the below-mentioned articles.
Walker (2019) used questionnaires to assess the ability of health professionals to engage in discussions with pregnant women about issues related to overweight and obesity during antenatal care, both before and after completing an online course. The pertinent responses to open-ended questions were analysed using a deductive approach (Elo & Kyngäs, 2008) However, this phase was preceded by an initial coding process conducted by two of the authors, which contributed to the finalization of the framework.
In their mixed-method study, Harju and Niemi (2020) investigated the perspectives of secondary school principals regarding teachers in the early stages of their careers, with a particular focus on fostering their professional development. Since the questionnaire also included open-ended questions, the responses were subjected to a theory-based content analysis technique and subsequent statistical analysis. The authors refer to several methodological sources, including Hsieh and Shannon (2005) and Elo and Kyngäs (2008). According to the authors, the implementation of predefined (theory-driven) dimensions proved to be effective, and they were further expanded with additional categories and subcategories.
In their study, Altan et al. (2019) conducted a comprehensive and collaborative two-part systematic literature review with the aim of establishing a stronger connection between a specific model, Costa and Kallick's description of intelligent behavior, and the field of educational science. They employed directed or deductive content analysis techniques following the guidelines outlined by Hsieh and Shannon (2005) and Mayring (2014). To prepare the analysis, the authors integrated both domains, the model, and education sciences, to develop a coding framework.
In a separate study, Allas et al. (2020) examined reflections on video materials from student teachers' teaching practice. The objective of their deductive content analysis was to help characterize the key segments, referred to as ‘meaningful events,' within the reflections on these recordings. This process was further complemented by additional inferential statistics. The examination of the content adhered to the principles of Mayring (2000), and the events were categorized based on the concept of instructional core.
In the paper of Viinikka and Ubani (2019), the ‘twenty-first-century skills framework' was employed to uncover the expectations of religious education student teachers regarding their teacher training, using a deductive content analysis approach. The empirical data were categorised into four theory-driven sets, in line with the reliability-related criteria put forth by Cohen, Manion, and Morrison (2004).
Different ways of enhancing student learning
The articles within the final set, which explore potential methods for enhancing students' learning performance, draw from a varied methodological background. However, there are notable disparities in terms of how explicitly the content analysis method is referenced. Some studies provide precise details about the approach, while others make more general references such as “qualitative analysis.” In the case of the former, akin to previous thematic groups, Elo and Kyngäs (2008) are of significant influence, often alongside the concurrent mention of Krippendorff (2004).
Herranen and Aksela (2019) performed a systematic literature review in order to map the position of student's questions in education sciences. The deductive review intends to outline the different types of student inquiries and their potential interaction with teacher questions. Although the authors use the definition of Krippendorff (2004), they applied the work of Elo and Kyngäs (2008) with further methodological references.
In McMahon et al.'s (2019) paper, the results of focus group interviews conducted with participants were subjected to deductive content analysis. This analysis aimed to create an environment that could enhance the effective learning of indigenous (Native American) students. The empirical data were examined with the application of deductive content analysis (Elo & Kyngäs, 2008). However, those data segments, which cannot be categorized into the previously established sets, were analysed via open coding, which could refer to an inductive approach.
Decker et al.'s (2021) study addresses the topic of early intervention. The authors examined the characteristics of 'Part C services' and how parents perceive them. The interviews were subjected to deductive analysis, following the method outlined by Elo and Kyngäs (2008). In this process, potential codes were initially developed separately by two researchers and were subsequently merged and discussed collaboratively to establish a unified framework.
Shea and Parayitam (2019) explored student satisfaction with their training through the use of e-portfolios. They conducted an examination of the essay collections authored by graduating MBA students using a Krippendorffian (2004) approach to content analysis with additional techniques. However, the two independent coders also identified and compared the revealed themes, which imply an inductive approach. Later, they used Likert-scale to evaluate the intensity of the emerging variables.
In addition to the theoretical and methodological sources introduced earlier, one can also consider other guidelines as an initial point for the analysis. Within this subset, Pnevmatikos et al. (2019) examined the peculiarities of the introduction of an educational approach (i. e. values and knowledge education – VaKE) which seeks to enhance students' critical thinking skills. In order to delve into their experiences, the study initially subjected their translated structured diaries to a content analysis, which was further enriched by a thorough examination of relevant quotations. The deductive analysis (Patton, 2002) aimed at using a critical-thinking-related theory in order to establish the main categories, which were extended by further subcategories.
Vartiainen et al. (2019) conducted a content analysis of interviews and with particular emphasis on the interviews, to reveal the progression of connected learning in a kindergarten supported by digital media. The study employed a qualitative content analysis. In its third phase, the paper/authors claimed to apply deductive reasoning, as per the framework established by Chi (1997), to identify three key themes, which were subsequently refined.
Havu-Nuutinen et al. (2022) conducted a bilateral comparative analysis in the field of early childhood education with the goal of identifying constructivist components within curricula through a deductive approach. The authors performed document analysis (Bowen, 2009) which involved connecting the key terms within the curricula to the theoretical framework. To enhance the robustness of the results, they also employed triangulation, following the guidelines provided by Bryman (2012).
As outlined above, some articles rather introduce their broader methodological background. Berg et al. (2019) facilitated students in creating stop-motion animations as a means to enhance their understanding of chemistry. The recorded animations of the process were then subjected to a thematic (deductive) content analysis. After the creation of the coding framework, three coders conducted the analysis, who regularly discussed the partial results of the process.
Salovaara-Hiltunen et al. (2019) employed virtual reality simulations with healthcare workers. Subsequently, they conducted a deductive content analysis on thematic interview transcripts following the stimulus. This research, which encompassed multiple stages, applied theory-driven content categories to classify the empirical data, with an examination of the differences between the coding patterns at each stage.
Poirier et al. (2020) sought to implement ‘visual thinking strategies' in a course for health professional students and assess the impact on their perceptiveness. The students' responses, based on their interpretation of various images, were analysed using a deductive approach. The results of pre- and post-tests were compared with additional statistical examination. Although the paper does not refer to a content-analysis-related work directly, it indicates the mixed-methods-related antecedents of the study.
Finally, Alhammadi (2021) investigates the potential effects of the COVID-19 pandemic on learning by establishing an internet café where students can engage in discussions on curriculum-related topics. Interviews with student administrators were analysed using a deductive approach. Instead of referring to an exact content analysis method, the authors cited several methodological works to introduce their research design.
Discussion
Interpretation
After the review process, it can be concluded that the examined journal articles offer a wide range of thematic perspectives. Despite education sciences being traditionally categorized as a discipline within the humanities or social sciences, the nature of the topics introduced, and the use of the content analysis method lend it an interdisciplinary character within the corpus under examination. Furthermore, these papers are frequently published in journals not primarily associated with education sciences, but rather with other disciplines or topics. However, from a content perspective, the articles can be categorized into three broader sets, demonstrating significant diversity in their exploration of various aspects of the examined phenomena. The research methodology also encompassed a broad spectrum, including the application of deductive approach and/or other logical branches (as outlined below):
Within the first group, health studies, rigorous application of the theory-driven approach could be observed, however, they are often supplemented by inductive components. While content-analysis-based health studies are frequently associated with the (partially) inductive GTA (for its reasons and historical background, see Watling & Lingard; 2012; Williams, Wiles, Smith, & Ward, 2022), both within and outside of educational science, our review underscores the significant role of the deductive reasoning within this subdiscipline, in which the position of Elo and Kyngäss’ (2008) article is pivotal.
Regarding the second group, the development of different professionals, methodological diversity can be identified, whose approaches are often applied in a combined way, i.e. the different compositions of Elo and Kyngäs (2008), Hsieh and Shannon (2005) and Mayring (2000, 2014). Furthermore, this particular set frequently incorporates an inductive approach into various stages of its investigations.
A similar, partially inductive rationale can be recognized in the third and largest group. Much like the second group, this set encompasses a combination of methodological influences, with concurrent references to Elo and Kyngäs (2008) and Krippendorff (2004). However, it's worth noting that references to other specific works or methodologies in a broader sense, such as the qualitative nature of the study, can also be identified.
In summary, based on what has been presented thus far, it can be concluded that the examined papers introduced various interpretations of what constitutes a deductive study. Furthermore, the incorporation of an inductive approach also frequently played a remarkable role.
Limitations & further development
Our study obtains several limitations. From a methodological perspective, it's important to note that while the screening process itself achieved the highest possible reliability due to automation (as per Janis, 1965), and the intercoder-reliability of the two coders pattern also showed an acceptable value, the grouping of the final set's elements inevitably resulted in lower replicability. This aspect stems from the inherently inductive and qualitative nature of this section, which aimed to construct a more valid understanding from the emerging data.
The second limitation is related to the self-identification of the papers. Although many papers indicated the use of deductive (or Krippendorffian) logic, and several terms implied this type of reasoning (e.g., ‘theory-based', ‘classical' or ‘direct’), they didn't necessarily employ a purely deductive approach. The initial exclusion of non-relevant, partially inductive articles (characterized by terms such as ‘emerged,' ‘qualitative,' ‘cluster,' ‘group,' etc.) was also based on the coders' decision. According to the suggestions of Armat, Assarroudi, Rad, Sharifi, and Heydari (2018) these concepts should be approached with greater flexibility due to their inherent ambiguities.
In addition to Krippendorff's approach, various other methodological works make appearances in the method sections of these papers. Although, for the sake of minimizing false positive identifications, we did not incorporate these works in the theoretical background of this study, it appears they should be considered, even if their primary focus isn't on deductive content analysis. Following the examination of the applied methods, the substantial role of Elo and Kyngäs (2008) became notably apparent. Although their article on content analysis was published in a health science journal (namely Journal of Advanced Nursing), it has become extremely frequent not only in this field of research but also in other disciplines. Regarding the additional references, alongside content-analysis-related works (e.g. Corbin & Strauss, 1990; Krippendorff, 2004) writings demonstrating broader qualitative perspective (e.g. Hsieh & Shannon, 2005; Mayring, 2000, 2014) also play a significant role. Furthermore, the research highlighted that the analysed writings do not necessarily follow the guidance of general theoretical and methodological handbooks; instead, they often rely on more focused, even discipline-specific articles.
Beyond the distinctions mentioned above, these papers present a diverse landscape with respect to their ontological, epistemological background, and research paradigms. For potential categorization, one might refer to Mitev (2012). These characteristics may warrant a broader, more comprehensive review.
Conflict of interest
The authors reported no conflict of interest related to the paper.
References
Theoretical and methodological background, discussion
Amidei, J., Piwek, P., & Willis, A. (2018). Rethinking the agreement in human evaluation tasks. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 3318–3329). Association for Computational Linguistics. https://aclanthology.org/C18-1281.pdf.
Armat, M., Assarroudi, A., Rad, M., Sharifi, H., & Heydari, A. (2018). Inductive and deductive: Ambiguous labels in qualitative content analysis. The Qualitative Report, 23(1), 219–221. https://doi.org/10.46743/2160-3715/2018.2872.
Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8–14. https://doi.org/10.1016/j.npls.2016.01.001.
Bryman, A. (2016). Social research methods. Oxford University Press.
Charmaz, K. (2008). Grounded theory as an emergent method. In S. N. Hesse-Biber, & P. Leavy (Eds.), Handbook of emergent methods (pp. 155–170). The Guilford Press.
Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Sage.
Chi, M. T. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315.
Chiasson, P. (2005). Abduction as an aspect of retroduction. Semiotica, 2005(153), 223–242. https://doi.org/10.1515/semi.2005.2005.153-1-4.223.
Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Sage. https://doi.org/10.4135/9781452230153.
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.
Garfield, E. (2007). The evolution of the science citation Index. International Microbiology: The Official Journal of the Spanish Society for Microbiology, 10(1), 65–69. https://doi.org/10.2436/20.1501.01.10.
Gavora, P. (2015). The state-of-the-art of content analysis. Neveléstudomány, 3(1), 8–18. https://nevelestudomany.elte.hu/downloads/2015/nevelestudomany_2015_1_6-18.pdf.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Aldine.
Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89. https://doi.org/10.1080/19312450709336664.
Hoffmann, M. (1999). Problems with peirce's concept of abduction. Foundations of Science, 4, 271–305. https://doi.org/10.1023/A:1009675824079.
Janis, I. (1965). The problem of validating content analysis. In H. D. Lasswell, & N. Leites (Eds.), Language of politics: Studies in quantitative semantics (pp. 55–82). Cambridge: MIT Press.
Kibiswa, N. K. (2019). Directed qualitative content analysis (DQlCA): A tool for conflict analysis. The Qualitative Report, 24(8), 2059–2079. https://doi.org/10.46743/2160-3715/2019.3778.
Krippendorff, K. (2019). Content analysis: An introduction to its methodology (4th ed.). Sage. https://doi.org/10.4135/9781071878781.
Mitev, A. Z. (2012). Grounded theory, a kvalitatív kutatás klasszikus mérföldköve (Grounded theory, the classic milestone of qualitative research). Vezetéstudomány. Budapest Management Review, 43(1), 17–30. https://doi.org/10.14267/veztud.2012.01.02.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372(71). https://doi.org/10.1136/bmj.n71.
Psillos, S. (1996). On Van Fraassen’s critique of abductive reasoning. The Philosophical Quarterly, 46(182), 31. https://doi.org/10.2307/2956303.
Sántha, K. (2008). Abduktív következtetés a kvalitatív pedagógiai kutatásban. Új Pedagógiai Szemle, 58(8–9), 32–44.
Watling, C. J., & Lingard, L. (2012). Grounded theory in medical education research: AMEE guide No. 70. Medical Teacher, 34(10), 850–861. https://doi.org/10.3109/0142159X.2012.704439.
Williams, T., Wiles, J., Smith, M., & Ward, K. (2022). Combining action research and grounded theory in health research: A structured narrative review. SSM. Qualitative Research in Health, 2, 100093. https://doi.org/10.1016/j.ssmqr.2022.100093.
Results (analysed articles and their methodology)
Alhammadi, S. (2021). The effect of the COVID-19 pandemic on learning quality and practices in higher education – Using deep and surface approaches. Education Sciences, 11(9), 462. https://doi.org/10.3390/educsci11090462.
Allas, R., Leijen, Ä., & Toom, A. (2020). Guided reflection procedure as a method to facilitate student teachers’ perception of their teaching to support the construction of practical knowledge. Teachers and Teaching, 26(2), 1–27. https://doi.org/10.1080/13540602.2020.1758053.
Altan, S., Lane, J. F., & Dottin, E. (2019). Using habits of mind, intelligent behaviors, and educational theories to create a conceptual framework for developing effective teaching dispositions. Journal of Teacher Education, 70(2), 169–183. https://doi.org/10.1177/0022487117736024.
Berg, A., Orraryd, D., Pettersson, A. J., & Hultén, M. (2019). Representational challenges in animated chemistry: Self-generated animations as a means to encourage students’ reflections on sub-micro processes in laboratory exercises. Chemistry Education Research and Practice, 2019(20), 710–737. https://doi.org/10.1039/C8RP00288F.
Bowen, G. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027.
Bryman, A. (2012). Social research methods (4th ed.). Oxford University Press.
Cheung, W. J., Patey, A. M., Frank, J. R., Mackay, M., & Boet, S. (2019). Barriers and enablers to direct observation of trainees' clinical performance: A qualitative study using the theoretical domains framework. Academic Medicine: Journal of the Association of American Medical Colleges, 94(1), 101–114. https://doi.org/10.1097/ACM.0000000000002396.
Cohen, L., Manion, L., & Morrison, K. (2004). A guide to teaching practice (5th ed.). Routledge.
Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13(1), 3–21. https://doi.org/10.1007/BF00988593.
Crowe, M., Inder, M., & Porter, R. (2015). Conducting qualitative research in mental health: Thematic and content analyses. The Australian and New Zealand Journal of Psychiatry, 49(7), 616–623. https://doi.org/10.1177/0004867415582053.
Decker, K. B., Meldrum, J., Vaterlaus, J. M., & Foster, T. D. (2021). Parents' Part C experiences in rural areas: Alignment with recommended practices. Journal of Early Intervention, 43(2), 155–175. https://doi.org/10.1177/1053815120953484.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x.
Harju, V., & Niemi, H. M. (2020). Newly qualified teachers’ support needs in developing professional competences: The principal’s viewpoint. Teacher Development, 24(1), 52–70. https://doi.org/10.1080/13664530.2019.1685588.
Havu-Nuutinen, S., Kewalramani, S., Veresov, N., et al. (2022). Understanding early childhood science education: Comparative analysis of Australian and Finnish curricula. Research in Science Education, 52(4), 1093–1108. https://doi.org/10.1007/s11165-020-09980-4.
Herranen, J., & Aksela, M. (2019). Student-question-based inquiry in science education. Studies in Science Education, 55(1), 1–36. https://doi.org/10.1080/03057267.2019.1658059.
Holt, A.-D., Smedegaard, S., Pawlowski, C. S., Skovgaard, T., & Christiansen, L. B. (2019). Pupils’ experiences of autonomy, competence and relatedness in ‘move for well-being in schools’: A physical activity intervention. European Physical Education Review, 25(3), 640–658. https://doi.org/10.1177/1356336X18758353.
Hörberg, A., Lindström, V., Scheja, M., Conte, H., & Kalén, S. (2019). Challenging encounters as experienced by registered nurses new to the emergency medical service: Explored by using the theory of communities of practice. Advances in Health Sciences Education: Theory and Practice, 24(2), 233–249. https://doi.org/10.1007/s10459-018-9862-x.
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687.
Karim, A., Shahed, F. H., Rahman, M. M., & Mohamed, A. R. (2019). Revisiting innovations in ELT through online classes: An evaluation of the approaches of 10 minute school. The Turkish Online Journal of Distance Education (TOJDE), 20(1), 248–266. https://files.eric.ed.gov/fulltext/EJ1201872.pdf.
Keats, M. R., Shea, K., Parker, L., Stewart, S. A., Flanders, A., & Bernstein, M. (2019). After childhood cancer: A qualitative study of family physician, parent/guardian, and survivor information needs and perspectives on long-term follow-up and survivorship care plans. Journal of Cancer Education: The Official Journal of the American Association for Cancer Education, 34(4), 638–646. https://doi.org/10.1007/s13187-018-1349-1.
Kolar, C., & Janke, K. K. (2019). Aiding transformation from student to practitioner by defining threshold concepts for the pharmacists' patient care process. American Journal of Pharmaceutical Education, 83(8), 7335. https://doi.org/10.5688/ajpe7335.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Sage.
Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557–584. https://doi.org/10.1037/1045-3830.22.4.557.
Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung Forum: Qualitative Social Research, 1(2), 20. https://doi.org/10.17169/fqs-1.2.1089.
Mayring, P. (2014). Qualitative content analysis: Theoretical foundations, basic procedures and software solution. Social Science Open Access Repository. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173.
McMahon, T. R., Griese, E. R., & Kenyon, D. B. (2019). Cultivating native American scientists: An application of an indigenous model to an undergraduate research experience. Cultural Studies of Science Education, 14(1), 77–110. https://doi.org/10.1007/s11422-017-9850-0.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Sage.
Pham, L., Moles, R. J., O’Reilly, C. L., Carrillo, M. J., & El-Den, S. (2022). Mental health first Aid training and assessment in Australian medical, nursing and pharmacy curricula: A national perspective using content analysis. BMC Medical Education, 22(1), 70. https://doi.org/10.1186/s12909-022-03131-1.
Pnevmatikos, D., Christodoulou, P., & Georgiadou, T. (2019). Promoting critical thinking in higher education through the values and knowledge education (VaKE) method. Studies in Higher Education, 44(5), 892–901. https://doi.org/10.1080/03075079.2019.1586340.
Poirier, T. I., Newman, K., & Ronald, K. (2020). An exploratory study using visual thinking strategies to improve undergraduate students' observational skills. American Journal of Pharmaceutical Education, 84(4), 7600. https://doi.org/10.5688/ajpe7600.
Roca, J., Reguant, M., Tort, G., & Canet, O. (2020). Developing reflective competence between simulation and clinical practice through a learning transference model: A qualitative study. Nurse Education Today, 92, 104520. https://doi.org/10.1016/j.nedt.2020.104520.
Salovaara-Hiltunen, M., Heikkinen, K., & Koivisto, J.-M. (2019). User experience and learning experience in a 4D virtual reality simulation game. International Journal of Serious Games, 6(4), 49–66. https://doi.org/10.17083/ijsg.v6i4.305.
Shea, T., & Parayitam, S. (2019). Antecedents of graduate student satisfaction through e-portfolio: Content analysis. Education + Training, 61(9), 1045–1063. https://doi.org/10.1108/ET-04-2019-0064.
Stacey, G., Cook, G., Aubeeluck, A., Stranks, B., Long, L., Krepa, M., & Lucre, K. (2020). The implementation of resilience based clinical supervision to support transition to practice in newly qualified healthcare professionals. Nurse Education Today, 94, 104564. https://doi.org/10.1016/j.nedt.2020.104564.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237–246. https://doi.org/10.1177/1098214005283748.
Vartiainen, H., Leinonen, T., & Nissinen, S. (2019). Connected learning with media tools in kindergarten: An illustrative case. Educational Media International, 56(3), 233–249. https://doi.org/10.1080/09523987.2019.1669877.
Viinikka, K., & Ubani, M. (2019). The expectations of Finnish RE student teachers of their professional development in their academic studies in the light of twenty-first-century skills. Journal of Beliefs & Values, 40(4), 447–463. https://doi.org/10.1080/13617672.2019.1618153.
Walker, R., Bennett, C., Kumar, A., Adamski, M., Blumfield, M., Mazza, D., & Truby, H. (2019). Evaluating online continuing professional development regarding weight management for pregnancy using the new world Kirkpatrick model. The Journal of Continuing Education in the Health Professions, 39(3), 210–217. https://doi.org/10.1097/CEH.0000000000000261.
Appendix 1: Krippendorff's Alpha Reliability Estimate (comparison of the results of Coder A and Coder B)
| Krippendorff's Alpha (nominal variable) | LL95%CI | UL95%CI | Critical values (left) and the probability of failing (right) |
| 0.8250 | 0.6500 | 0.9563 | 0.8000 0.3716 |
| 0.6700 0.0420 | |||
| 0.6000 0.0049 |
Number of bootstrap samples: 10,000.
