Abstract
This paper summarizes the trends in research about student experiences in out-of-school time (OST) in countries of Europe. The analysis is based on a review of the content of research papers published between 1999 and 2019 in about 36 European countries. OST is recognized as an important aspect of students’ educational experiences that deserves increased research attention. A significant portion of students in all countries participate in OST activities either to improve their school performance or to engage in social activities. Under conditions where parents and students believe that the formal school system is weak, the OST educational experiences may be considered to be necessary to make a significant or necessary contribution to a student’s success compared with formal school itself. Because OST activities are undertaken in a free market their form and structure vary because of differences in the countries’ historical development, condition of public education, and the social, economic, cultural, and political factors that influence the educational systems. The analysis presents examples of how in some European countries conceptual models of OST have been expanded, adapted OST practices for regular school systems, and evaluated the outcomes. This review of the definitions of OST, of evaluations of its impact, and of the evidence for its effect on equality of educational opportunity throughout 36 European countries concludes that the studies provide contradictory messages. Greater consistency in conceptual development could be increased over time as researchers across countries review each other’s strategies and share methods and results.
Introduction1
Only about 20% of a child’s waking hours between ages 5 to 16 is confined to formal class time study (Bell, Lewenstein, Shouse, & Feder, 2009). Thus, a substantial amount of a child’s time is available for out-of-school time (OST) activities during the calendar year either after school days or on days with no class. Students in all countries engage in formal and informal activities during these hours that may have significant influence their cognitive and social-psychological development (Mahoney, Larson & Eccles, 2005; Noam & Shaw, 2013). If one of the functions of receiving education through OST is to ensure that individual students have greater opportunity to learn academic school subjects, then the distribution and management of those resources are a matter for educational research interest and concern for public policy (Entrich, 2014; Holloway & Pimlott-Wilson, 2020).
The phenomenon of after-school time use takes many forms and therefore different researchers in different countries working from different theoretical frameworks have applied labels to OST that may be unique to researchers or country practices (Kobakhidz & Suter, 2020). The number of different terms used in the titles of 640 research papers from 61 countries (or economic units) identified 150 unique terms. Of these 126 were used only once or twice. The five most common terms are private tutoring (127), shadow education (117), after-school (71), out-of-school time (56), and private supplementary education (26). In the research papers of Europe private tutoring is the most common term; in Asia shadow education is most common; and in North America, after-school is most common. Shadow education has been defined as a specific form of private supplementary tutoring that occurs after school hours, addresses content areas of the formal school, and involves a payment (Bray, 1999; Buchmann, Condron, & Roscigno, 2010; Stevenson & Baker, 1992). It is unique from private tutoring because it more closely follows the formal school system curriculum and assessment (therefore, shadowing it). The term OST, on the other hand, is a broader term that overlaps in meaning with shadow education because it may include forms of educational activities that do not necessarily directly mimic the formal school system. Following a review of the bibliography identified for this paper, the meaning and types of OST will be re-examined.
The purpose of this study is to describe the variety of research studies of out-of-school-time (OST) activities either undertaken by scholars in European institutions or of studies conducted about countries of Europe. The approach of the analysis is to collect and systematically review a large number of research studies. The paper is intended to contribute toward developing a stronger theoretical and empirical basis for new explorations of the outcomes of student activities conducted outside of formal classes.
The role of student’s different experiences outside of regular school hours has become a matter of growing academic interest over the past 20 years as shown by the increase in publications displayed in Figs 1 and 2 (also see: Baker, Akiba, LeTendre, & Wiseman, 2001; Bray, 1999, 2010; Bonn & Haag, 2002; Fraij, Janzen, & Kielblock, 2017; Kim & Jung, 2019a; Verger, Fontdevila, & Zancajo, 2016). The role of OST practices in the education of students in Asia has been well documented (Bae & Hong, 2016; Bray, 2014; Lee, Lee, & Jang, 2010). But the level of adoption and influence of OST on student learning in countries of Europe has been studied less often and has rarely been treated as an entirety or as a unique aspect of a particular culture or geographic boundary. This bibliographic analysis is intended to describe the changes in topics and research questions found in descriptive and analytical interpretations of tutoring, extracurricular activities, and other forms of supplemental education conducted by researchers in or about European countries. The summary of trends shows that the amount and quality of research on OST has increased more in some European countries than in others due to some extent to the availability of research funding and governmental interest.
The paper seeks to obtain an overall view of research about OST in Europe. Because space is not sufficient to conduct a country by country analysis, the presentation follows the following themes:
The first section describes the methods of collecting research studies about OST in all countries and specifically identifying those about Europe. This methods section includes a description of the resulting bibliography of research and types of studies and places the set of research reports in international context.
The second section summarizes four themes running throughout the papers: defining OST, reasons for attending OST, effects of OST on learning, and the relationship between access to OST and equal opportunities to learn.
A final section examines issues about methods of data collection of OST. This topic is addressed because analyses of large-scale international comparative studies have produced inconclusive findings (Bray & Kobakhidze, 2014b; Bray, Kobakhidze, & Suter, 2020). A summary review of the issues raised in the European OST literature concludes the paper.
Methods of analysis and research questions
The resource for this analysis is based on a collection of OST research studies conducted around the world. These studies report the results of empirical research that have used methodologies such as ethnographic study, sample survey, experimental design, syntheses, and meta-analyses of multiple studies. All studies accumulated for this paper had been published in academic journals, government reports, or in special study reports between 1999 and 2019. While an effort has been made to be comprehensive, the task of assembling research papers on a single topic published by different types of institutions in written in different languages cannot claim that the body of papers fully represents the entire universe of research in this area.
Selection
Without a comprehensive catalog of all international research about OST that could serve as a sampling frame to select a representative sample of research of European authors and topics, a new bibliography was created. Initially a bibliography of research about educational practices in all countries was assembled, then specific references about OST were added, research papers not related to OST were flagged, and all entries were coded for country of study and country of authorship. The initial source of publications is a bibliography of 3,000 comparative education publications that had been assembled from many bibliographies to analyze trends in comparative education (Suter, 2019b; Suter, Smith, & Denman, 2019). Additional studies from 50 published reference lists of research and syntheses studies of OST were added (Bray, 2013, 2020; Gromada & Shewbridge, 2016; Kuger, Klieme, Jude, & Kaplan, 2016; OECD, 2016, 2017a). The list was further expanded with the use of bibliographic tools Mendeley, Google, and Academia and by recommendations of other researchers including reviewers of the first draft of this paper. Research reports about OST in individual countries of Europe were added. Documents from international organizations that had a particular emphasis on Europe were identified and included. Electronic copies of papers were obtained from the University of Michigan Library. This set of papers was further reviewed to remove duplicates, studies of countries outside of Europe, and studies of topics that were outside the scope of OST. These additions produced a final set of approximately 588 papers about OST from 55 countries of all regions over a 20 year period formed the basis for selecting 191 papers addressing OST in Europe. The distribution of countries of study and authorship is presented in Appendix A.
The final set of 191 papers contains a wide variety of approaches and research questions and contains a broad geographic representation of countries of Europe. Without a basic sampling frame of all OST research papers with which to compare, the representativeness studies for the entire body of knowledge of OST cannot be firmly established. The bibliography represents a reasonably large number of articles that had been selected in a systematic way, but it may have unknown biases due to the representation of source materials. Geographic classification of the research content and authorship was conducted by the lead author based on titles and searches of each article. Articles that addressed more than one country were coded either by general region (such as Europe) or global depending on the scope. Papers that were written about OST that might be relevant to Europe but were oriented to geographic areas outside Europe were used for framing questions but not for description of European practices.
Coding
The studies were coded for, country of analysis, country of residence of author, name for OST, language of publication, journal name, publisher, purpose of analysis, type of publication, and year published. The papers and official reports that were published in English, German, Polish, Hungarian, French, Italian, Romanian, and Spanish were translated to English with Google translation when necessary. Authors were identified by current institution of residence. Papers written by multiple authors were coded by the origin of the first author.
Description of publication set
Classifying research papers by geographic reference to Europe presented two technical issues. First, identifying the boundaries of “Europe”. The geographic identity assigned to a paper required a broad definition of the geographic area of Europe. The United Nations statistics division includes 51 countries or areas (U.N. Statistics Division, 2020) as Europe while the European Union is composed of 27 members (as of January 2020). Within some country boundaries, multiple economic or social systems are occasionally the subject of analysis (such as England, Wales, Scotland, and Northern Ireland of the United Kingdom; language groups of Flemish, French and German in Belgium which form separate educational systems as occurs in Spanish and Catalan regions in Spain). Studies of specific areas within a country boundary were coded by the name of the collective (such as United Kingdom). Most countries that made up the former Soviet Union are included in the expanded definition of Europe. Papers that referenced any of these countries were included in the Europe classification.
The second issue with geographic classification is in classifying the content of the research itself to a geographic region and classifying the identity of the author. Most papers were written by authors about the country of their residence (170 out of 191). Other papers were written by experts in the field of OST studies about countries outside of their residence. The final selection and coding of papers resulted in analyses of OST in 35 European countries with authors from 35 countries.
The set of references can be used to demonstrate levels of change over time in the representation of European countries in papers about OST compared with the number in Asia, and the Americas (Fig. 1). A total of 583 OST papers were selected as a representative set of all global research on OST. Only 1% of international set of papers were written about regions other than Asia, Americas, or Europe. The changes over time shown in the chart indicate that the number of OST research papers published about European countries increased from only a few per year in 1999 to around 12 to 15 publications per year by 2012–2019; while during this period, the number of publications by United States authors declined. About a third of all international publications by 2019 included analysis of countries of Europe.
The number of papers published by European authors, compared with authors from Asia and North America, is shown in Fig. 2. Authors in European universities and research centers were increasingly represented between 2008 and 2019. However, authors based in Asia consistently had the largest number of publications addressing OST.
Many studies of OST do not appear in peer-reviewed journals. About 69 publications (30%) were published as reports from research centers, government agencies, international agencies, conference proceedings, and books or book chapters. Thus, a little over half (55%) of all publications were published in 78 peer-reviewed journals, of which 60% were published in English. Dissertations were excluded.
The variety of authors, countries, and research methods represented in these collections provides evidence that organized forms of OST are found in all European countries and that many researchers have investigated them. Though the types of organizations and motivation of individual participants and suppliers are similar, the practices are not identical from country to country. Differences in OST practices will be described in the following analysis.
Description of research publication types
The research questions and methods of study in each country of Europe differ because of the political and socio-economic contexts. For example, unlike other countries, the government of Germany decided in 2003 to increase funding for extended study and to support an extensive research and evaluation program (Holtappels, Kamski, & Schnetzer, 2007a).
The collection of research studies can provide an initial basis for identifying the research questions in the region and for guiding the selection of future research studies of changes and effectiveness of OST practices. The issues most addressed are: conceptual framework for OST, evaluation of student performance, equity of participation, quality of instruction, and program development.
The methods of study found in the collection of OST research included 92 papers applying ethnographic methods descriptive of OST in individual, or a small set, of European countries. Another 115 studies applied empirical methods of survey design, controlled experiment, or hypothesis-driven case study. The research studies often addressed OST with conceptual frameworks from educational research such as learning theory, social equity, effectiveness, and teaching practices. Other studies addressed political and economic issues that affected the choices of students and families to participate in educational activities outside of formal school.
Many research reviews of OST activities in Europe have been conducted over the past 20 years. These papers have established the extent to which OST is practiced in each country and have commented on the policy issues involved in its development (Ecarius, Klieme, Stecher, & Woods, 2013). The issues addressed in these review studies help formulate the structure and meaning of OST research studies. Some of the more general reviews of the field are: Baker et al., 2001; Bray, 1999, 2010, 2011, 2013, 2020; Bray & Kobakhidze, 2014a; Chappell, Nunnery, Pribesh, & Hager, 2011; D’Agostino & Murphy, 2004; Faganel & Trnavčevič, 2013; Gromada & Shewbridge, 2016; Jun, Ramirez & Cumming, 2010; Kuger et al., 2016.; Manzon & Areepattamannil, 2014; Matterson & Holman, 2012; Šťastný, 2014; Zaff & Redd, 2001. Journals have also published special issues with collected multiple papers on issues related to OST practices that provided a useful view of variations in perception of the purposes and implications of OST not unified by theory or method. For example, in 2014 a special issue of the Journal for Educational Research Online includes 10 papers with empirical evidence about the effectiveness of forms of OST found in several European countries. Five paper collections provide high quality reviews of specific countries’ OST (Bray, Mazawi & Sultana, 2013; Ecarius et al., 2013; Schüpbach & Lilla 2019; Silova, Būdienė, & Bray, 2006). Bray has collected and synthesized studies from each country in the European Union to estimate the frequency of OST participation similarities and differences among those countries on three occasions (Bray, 2011, 2020).
Good syntheses of research about OST concepts (but not always focused specifically on Europe) were organized by Elbaum, Vaughn, Tejero Hughes, and Watson Moody, (2000), and Chappell et al. (2011). Manzon and Areepattamannil (2014) edited a special issue of a journal concentrated on the Asian region. However, the “editorial” by Manzon and Areepattamannil provides very useful observations of shadow education and can serve as a guide to research on OST studies in all countries (Manzon & Areepattamannil, 2014, p 390). A series of conference proceedings was published on extended education (a form of OST found in Germany) that contains nuanced discussion of OST that is worthy of investigation (Ecarius et al., 2013; Fischer & Klieme, 2013). An issue of the Journal of Educational Research Online edited by Brehm and Silova included 4 articles of empirical analysis and 3 reviews of the field (Brehm & Silova, 2014). Syntheses of studies have also been published by OECD (2016, 2017c), Gromada and Shewbridge (2016), Bray, (2013 and 2020), Kuger et al. (2016), that provided an extensive bibliography and background about the content and methodology for the study of OST.
Topics addressed in European research on OST
This section on defining OST activities is based on a post hoc reading of papers and thus is a product of the empirical analysis rather than following an a priori theoretical framework. Four themes were consistently addressed in the sample of research papers identified: Defining OST, discussing reasons students attend OST, analysis of the effects of attending OST, and discussion of how OST affects equality of educational opportunity.
Definitions of out-of-school time
Many publications apply terms, such as shadow education, private tutoring, all-day education, out-of-school time, and extended education, which assume that readers have an informal understanding of their meaning. However, few researchers have defined the behaviors involved and therefore writers and readers may not always refer to the same set of practices in the same way. To enable comparisons of geographic areas, over time, or between different groups in the same place, data collection must be standardized with. For example, data obtained in the PISA student surveys that asked students whether they had participated in “additional study, not including homework” may be labeled by an author as shadow education or tutoring without evidence of the actual behaviors undertaken by students. Labels for OST practices used by some authors may include or exclude practices assumed to be intended by other authors (such as whether the tutoring was fee free or not, or whether a tutor followed the curriculum of the formal school or not).
Several efforts have been made to conceptualize general frameworks (Bray, 1999, 2014; Kim & Jung, 2019a; Kuger et al., 2016; Šťastný, 2014). But none of these has been widely adopted. The unique names from of OST found in European research studies were usually applied to single publications without acknowledgment of connection to other related research studies. The common thread for all of these terms was that they refer to a student’s opportunity to learn during OST; otherwise, differences are found in cost, schedule, location, instruction, and curriculum (Kim & Jung, 2019b; Kobakhidze & Suter, 2020; Kuger et al., 2016).
An extensive body of cross-national research uses three terms: shadow education, private tutoring, and tutoring exclusively. Because OST occurs during time periods that vary from place to place (week days, weekends, and vacation weeks) and may occur for either short or extended periods of time, and may or may not involve an instructor with educational training, and that may incur a cost (or not), the term OST has been used throughout this paper as a broad collection of organized study of school subjects conducted outside school hours, other than homework, that may entail additional costs. The term excludes other afterschool activities that are not directly tied to formal school content, such as sports, arts, library use or museum attendance, usually referred to as extra-curricular activities which are not specifically intended to increase student performance. The social and psychological benefits that might be gained from extracurricular activities has been studied as a separate topic (Covay & Carbonaro, 2010; Kielblock, 2015; Mahoney et al., 2005; Marsh, 1992; Schiefer et al., 2019; Suter, 2014).
The participants and suppliers of OST are investigated by researchers at multiple levels of aggregation. For example, in defining the scope of shadow education Bray et al., (2013) identified a broad number of country level social and economic conditions that may affect the extent of OST use. They show how a country’s historical and socio-political conditions influenced the choices for OST participation for different geographic areas. Educational opportunities of children in all countries are affected by macro forces such as economic markets, cultural politics, parental strategies, social status, as well as individual whims that may take different forms at the local, region, or global level (Huang, 2013; Silova et al., 2006). At the micro level, other researchers have investigated how participation affects a student’s social and psychological skills or career orientation (such as for science and mathematics fields) as well as cognitive skills that are oriented to the school curriculum (Bell et al., 2009; Ecarius et al., 2013; Fischer et al., 2011; Klieme, Pauli, & Reusser, 2009; MacBeath et al., 2001; Noam & Shaw, 2013). Many studies have investigated the effects of OST practices on improvements in academic performance at formal school (Berberoğlu & Tansel, 2014; Huang, 2013; Smyth, 2008) some of which are reviewed in a later section of this paper.
The editors of a special issue on shadow education published by the Asia Pacific Journal of Education found that researchers of shadow education in all countries struggled to “establish the conditions of truth about what shadow education is or ought to be” (Manzon and Areepattamannil, 2014, p. 390). The difficulties occurred because the observations of each author were affected by the status of the country’s economy, by the type of private market systems that had been established, and by the history of country reforms. Thus, the authors from different countries were not able to define consistent concepts of tutoring over time or place. Consequently, the editors felt compelled to call for additional systematic research on the topic of defining the structures. At the country level, Manzon and Areepattamannil (2014) identified key research issues at the social system level as equity, family burden, and corruption.
Researchers in Germany and the United Kingdom together produced 40% of the research papers about OST during the 20-year period 1999–2019 (Appendix A). The most common terms found in the German literature are private tutoring, all-day or extended education, while the researchers of the United Kingdom followed little consistency except for a tendency to call it private tutoring or tuition.
Researchers in Germany defined the term tutoring (Nachhilfe) broadly as a continuum extending from informal parental support “at the kitchen table” to enrollment in private organizations of the non-formal sector (Guill, Lüdtke, & Köller, 2019). The type of instructors in the extended school time ranged from retired teachers to university students or laypeople such as the neighbor’s children or parents. The result was that the professional background of the instructors had wide variation. Studies of the effectiveness of instructors with different levels of qualifications have failed to empirically demonstrate an effect of quality (Fischer & Klieme, 2013; Guill et al., 2019; Steinhäuser, 2019).
Germany instituted school reforms in some regions of the country in 2003 in response to the low performance of German students in PISA 2000 compared with other European countries. The reforms changed the daily routine of a student’s study hours (Deutsche Welle, 2003; Stecher & Maschke, 2013; Steinhäuser, 2019). The new programs, called “extended education” (or all-day schools) are intended to provide extra time for students to engage in a variety of educational activities including sports, graphic art, literature, theater, dance, music and nature outings in the hope of improving student learning and personal adjustment (Deutsche Welle, 2003; Klieme, Kühnbach, Radisch, & Stecher, 2005). The research reports noted that extended education programs are more often oriented toward the client rather than the school system (Ecarius et al., 2013 and therefore expected to have benefits for students and families at a higher level than without the reforms.
Both United Kingdom and Germany have adopted programs for the formal school that include activities formerly reserved for after-school. Both countries have experimented with changes in the school day and are evaluating the outcomes. The U.K. government initiated a National Childcare Strategy in 1993 to expand the number of student clubs and other extended activities throughout the country (Fordham, 2004). The program theory proposed that improving student self-esteem and motivation in after school programs could affect achievement levels during school hours (Fordham, 2004). The emphasis of this program was curriculum enrichment, extension of study time, and homework. The activities also encouraged by schools included sports, drama, chess, photography, and “study support” opportunities, such as homework clubs, extra revision classes, and extra after-school tuition (Fordham, 2004, p. 49). The program goal of the policy of study support was modified again in 1997 to de-emphasize social activities and increase attention to types of study that would lead to improvements in school achievement; illustrating that expectations from alterations in the use of student time require evaluation and theoretical development. The evaluation of the “study support” programs was conducted by a government funded analysis carried out by researchers at a Scottish university in 1999–2000 (MacBeath et al. 2001).
Reasons for attending OST
Common issues addressed in research papers of OST are the amount of time spent in OST, variations in modes of instruction, reasons for attending, the role of tutoring on student performance, and how family economic status affects participation. In Bray’s assembly of research studies of shadow education in the European Union (Bray, 2011, 2020) he reported that the incidence of participation ranged from a low of 10% in Belgium to 95% of students in the last year of secondary school in Greece (Bray, 2020, p. 5). Bray examined case studies of individual countries, such as Ireland, England, Slovakia, Croatia, Poland, and Lithuania to illustrate how family background influenced motivation to attend OST. The patterns of differences between countries reported by Bray are consistent with PISA survey results in country rank order (OECD, 2012; Suter, 2019a, and special analysis of PISA 2018 unpublished).
Another collection of empirical research studies about OST edited by Silova et al. (2006) contained 20 chapters on the condition of supplementary education in 9 countries that were members of the former Soviet Union or belonged to the Soviet sphere of interest: Azerbaijan, Bosnia and Herzegovina, Croatia, Georgia, Lithuania, Mongolia, Poland, Slovakia, and Ukraine. These papers examine the educational decisions made by parents and students during a time of large-scale political changes. The research reports about education in countries that formerly were members of the Soviet Union noted that after 1991 the standard practices and daily routine of students and instructors were likely to be changed and that some families sought educational resources outside of formal school (Silova, 2010; Šťastný, 2014) because they perceived weaknesses in the formal education system. Private tutoring was viewed as an effective educational method for adopting to circumstances of changes in the formal educational system. The researcher’s observations addressed how country and school conditions encourage the growth of private supplementary educational systems. The private systems grow in an uncontrolled private market during periods of political instability and increased parental concern about the quality of education for their children. Other motivations for obtaining an individual tutor were instances when students had low levels of confidence in assistance from family members or had an inability to study alone (Silova, 2010; Silova et al., 2006).
Other researchers also observed that parents encouraged students to participate in after-school tutoring to make up for perceived poor education in formal schools. The editors of a series of papers on OST in Bosnia-Herzegovina, Croatia, Cyprus, Egypt, France, Greece, Italy, Malta, Portugal, Slovenia, Turkey, found that systems of additional study have grown differently in each country in response to the level of demands from parents who perceive lack of quality in formal schools or to student failure in meeting testing standards (Mazawi, Sultana, & Bray, 2013). Students living in countries that required examinations for graduation were drawn to OST after the policies changed. In fact, tutors were in increasing demand at the end of secondary school in the Czech Republic to prepare for specific types of examinations, or to excel above other students in a subject area important to careers, such as mathematics (Šťastný, 2014).
Šťastný has conducted a review of literature about the growth of tutoring in the Czech Republic (Šťastný, 2014, 2016, 2017). In his earlier paper he reviews the definition of shadow education using a model known as the “Bray and Thomas cube” (Bray & Thomas, 1995) and reviews several definitions found by those who have conducted extensive research. For example, he cites Brehm and Silova (2014) as recommending four conceptualizations of tutoring around the concepts of cultural values of education, enrichment for high-stakes examinations, remedial preparation for poor performance school, and as a mechanism for additional income of teachers (see also Kobakhidze, 2018). He points out that some countries in Asia had created laws to prevent families from seeking tutoring. But he found that some countries have not established regulations, perhaps because private tutoring may be a large segment of the educational opportunities of students. Then he argues in his 2014 paper that family income is likely to have a significant impact on private tutoring because of the expense. Thus, he sees future growth of tutoring as a possible source of unequal opportunities because families use it to get around the limitations of a public school system that offers the same educational opportunities for all students regardless of ability or ambition.
Šťastný conducted a survey of tutors in Czech Republic with an on-line method that produced comparisons of tutor participation and the supply of tutors within regions of the country (Šťastný, 2017). He reports observing a significant variance throughout the country in supply and qualifications of tutors. Tutors with higher education were more expensive than others and therefore would likely be unavailable to lower income students. He noted, however, that the large variation in price of tutoring was likely to allow students of lower means to find some affordable type of tutoring services, especially in rural areas. But, he was not able to determine whether low income families were able to receive high quality tutoring.
Effectiveness
Does evidence support the belief that additional time in organized out-of-school programs increases student academic performance (Bray, 2014)? Some policy makers have argued that greater study time will lead to higher achievement (Berliner, 1990; Farbman, 2015; Heyneman, 2011; Kuger et al., 2016); Štastný, 2016), while others do not find evidence for a relationship between additional study time and student performance (Elbaum et al., 2000; Husén, 1972; Karweit, 1984; Long, 2014). John Carroll, an American education theorist and researcher provided a model of learning and time use in 1963 that provides a conceptual basis for analysis of time for learning. He argued that a student’s aptitude (or ability) interacts with the length of study time necessary for each student to become equally proficient (Carroll, 1963). Carroll (1963, 1989) defined aptitude as the amount of time a student needed to learn; thus, time itself is defined as an integral part of the definition of learning even though its constituent parts are not clearly delineated in most research studies (Berliner, 1990, p. 9). He noted that students of lower ability would be most likely to benefit from additional study time because they require more time and effort to reach achievement levels of higher ability students (Carroll, 1963, 1989). His theory has been cited by European researchers who developed frameworks for study time analysis in international studies (Kuger, 2016; Suter, 2016, 2019a) and it will guide the following analysis.
Berliner (1990), in a delightful essay on the nature of time and learning, examined historical, philosophical, and recent research about how instructional time has been defined. He reported that educators conducted studies of student time and learning as early as 1918 either finding no relationship or only complex relationships. He argued, however, that time is a family of concepts many of which are not yet formally defined or operationalized in a way that they are useful for research or policy making.
Even though a large number of studies have been conducted about the influence of additional study time on student achievement, agreement about the role of study time as an explanatory factor for student achievement and country differences in academic achievement has not been reached (Berliner, 1990; Farbman, 2015; Long, 2014; OECD, 2012; Rappleye & Komatsu, 2018; Suter, 2019a). In 1972, Torsten Husén, then director of the IEA, aware of the increasing media interest in length of school time, raised concerns about appealing to study time as having a distinct role in country achievement levels (Husén, 1972). In the field of OST studies, Bray (2014) concluded that the relationship between extra study and achievement is ambiguous because different countries display different relationships. In some countries, additional study time appears to lead to higher achievement while in other countries it may not. He proposed that the lack of establishing a relationship was more likely due to invalid measurement of time use than to the lack of influence of additional study (Bray & Kobakhidze, 2014b; Bray, Kobakhidze & Suter, 2020). Nevertheless, some European countries have adopted policies to increase student exposure time to classroom content.
Educational reforms were initiated in Germany in 2003 intending to increase the ability of schools to “offer more time for learning and provide more variety in education” (Deutsche Welle, 2003, p. 1). The purpose was to provide extra time for students to engage in a variety of educational activities including sports, graphic art, literature, theater, dance, music, and nature outings. The Leibniz Institute for research and Information in Education (DIPF) was established to evaluate the programs nationwide with a systematic research program, “Study on the Development of All-Day Schools” (Studie zur Entwicklung von Ganzstagsschulen, or: StEG) (Fischer, Brümmer, & Kuhn, 2011; Holtappels, Klieme, Rauschenbach, & Stecher, 2007b). The research program included a longitudinal survey of individuals and institutions and measurements of the quality of the education and the effects on students (Stecher & Maschke, 2013). Consequently, the amount and quality of empirical research in Germany concerning the effects of expanding the school day has increased in the past decade. An extensive series of reports has been published on the evaluation of the ongoing research on OST in Germany (Fischer, Brümmer, et al., 2011; Fischer, Holtappels, et al., 2011).
The research model that formed the basic backdrop of the evaluation of the German extended education program was based on assumptions of the learning processes developed from formal school learning theory would also be appropriate for analysis of informal settings (Ecarius et al., 2013). The model asserted that learning is improved by having a highly structured learning environment and an environment to promote positive emotional relationships between peers and adults (Klieme et al., 2009; Stecher & Maschke, 2013). Other topics considered in the research studies include the nature of the instructors, the effect of not having performance assessments, the effect of mixed-age groups, and whether curricular systems would be useful or not (Ecarius et al., 2013, p. 8). Thus, research studies were conducted on the processes of school development, staff collaboration, parental satisfaction and students’ outcomes and included interviews with principals, external school partners, teachers, caregivers, parents and students (Fischer & Klieme, 2013).
The first results of the German longitudinal study of the new reforms showed evidence that, by itself, longer day participation was not a key ingredient for increasing student outcomes. They reported in 2013 that research on extended education was still at an early stage of development, that research methods suited for the context of after-school time needed to be developed, and that theoretical approaches have yet to be well enough developed to direct the research processes (Fischer & Klieme, 2013; Stecher & Maschke, 2013). In 2017 Steber and colleagues observed that all-day schools did not always achieve their intended effectiveness. They found that participation and duration affected academic performance in mathematics and German significantly, but that attendance intensity was not significant. Some studies observed that private tutoring was not performed as a remedial strategy but more as a general strategy for improving the student’s competitiveness (Fischer et al., 2011; Holtappels et al., 2007b). The researchers found that the effects of extended days on students learning were different for each grade level and subject matter (Ecarius et al., 2013).
Guill et al. (2019), concluded that evidence from the longitudinal survey of German students showed that student grades in mathematics or language (German) were not affected by attending private tutoring nor by the instructional quality of the tutoring as they had expected. However, they found that student attitudes toward mathematics was higher for students after receiving tutoring lessons (Guill et al., 2019; see also; Suter, 2019a). This is a useful observation that should inspire other researchers to consider the non-academic as well as the academic influences of OST on student behavior.
Equity and OST participation
The extent that family social status has affected a student’s chances for engaging in studies outside of formal school requirement is addressed frequently in the studies located for this analysis. About a fifth of all papers discuss the issue of the effect of tutoring on equity of access to learning opportunities. Some topics included reasons for parental motivation to pay additional funds for supplementary education (Ireson & Rushforth, 2005).
Several research studies in the collection of OST articles have suggested that socioeconomic differences between students and families could be increased by encouraging tutoring because students of higher status families (higher in educational levels and occupational status) would be more likely to be able to give their children access to tutoring and thus benefit from individual instruction (Buhagiar & Chetcuti, 2013; Entrich, 2018; Galinié & Heim, 2016; Jokic’, Soldo, & Dedić, 2013; Lamprianou & Lamprianou, 2013; Luplow & Schneider, 2014; Schüpbach, 2014; Tsiplakides, 2018). Some authors argued that tutoring was common among upper classes to ensure that their children would maintain their family status (Neto-Mendes, Costa, Ventura, Azevedo, & Gouveia, 2013). On the other hand, the researchers noted that middle class parents who wished to give their children some advantages in competing for higher education would also able to use OST as a means of increasing those chances. The question of whether the relationship of tutoring to family status is biased against low status families is likely to remain an unsolved mystery because observational studies reviewed in this compilation generally conflict with evidence from large-scale research studies like PISA (Entrich, 2018; Suter, 2019a). The results of the first series of studies in Germany showed that participation in extra study programs was not related to family background, such as parent’s educational level.
To explore the relationship of equity and OST participation further, an analysis of family status and OST participation was conducted with the 2018 PISA survey. The PISA survey analysis indicates that in most countries, students from lower status are more likely to attend OST than students of higher class (measured by parent’s education or occupational status). Exceptions to this generalization are found in Korea, Hong Kong, and China (Suter, 2019a). The relationship of parent’s education and occupational status to student participation in OST is similar in all European countries that participated PISA 2018. Students from families of higher education or occupational status were less likely to participate in additional study than those of lower status. Large differences in size of relationship, but not the direction, are found with the 4 different PISA status scales (parent’s education, parent’s occupation, wealth, and home environment). If all types of OST activities are equally likely to be associated with higher participation by students of lower status families, then the mechanism that creates this condition in each country of Europe should be identified through further analyses of surveys and case studies of European countries.
Measurement of OST with cross national comparisons
If a large international survey had included a sample of all European countries, it would be possible to identify differences and similarities in OST participation. However, none of the large international studies have included all European countries in their surveys of OST. The participants are voluntary and change from survey to survey. The recent 2018 PISA survey included the largest number of countries of Europe participating in the optional student’s questionnaires of a supplementary survey of “additional study” since 2000. The participants included 10 countries that were members of the old soviet bloc (Albania, Bulgaria, Croatia, Hungary, Lithuania, Poland, Serbia, Slovakia, and Slovenia) and 9 in Western and Northern Europe (Belgium, Denmark, Greece, Ireland, Iceland, Italy, Malta, Spain, and England). The increase in participation from Europe signifies an increasing level of interest in OST among funding agencies and statistical units of European countries. Future studies of Europe may be able to rely on systematic survey evidence.
The two large-scale international surveys, the IEA’s Trends in Mathematics and Science Study (TIMSS) and OECD’s Programme for International Student Assessment (PISA) (Mullis, Martin, Kennedy, Trong, & Sainsbury, 2015; OECD, 2016) have collected student responses to questions about tutoring and additional study. Both survey organizations have included survey items about student participation in school-related study outside of regular school hours. Analyses of these surveys have provided insights into the differences in country to country adoption of OST and the analyses also give examples of types of difficulties that occur with defining the content of OST its relationships to achievement and attitudes in a student response survey. About 12 European OST research papers have used these sources. Official OECD reports contain analyses of differences in impact of class time, homework time, and additional study time on learning of specific subject matter domains (OECD, 2011a; 2011b; 2011c, 2014, 2016; 2017a; 2017b; 2017c). An example of the types of analyses that might be produced by these studies is about the relationship of time spent in OST and achievement. The analyses of PISA surveys indicate that greater time spent in OST programs is associated with students of lower achievement. The meaning of these observations is still uncertain (Suter, 2019a). The surveys are cross-sectional therefore they do not allow an analysis of effects of participation after OST experiences have occurred. Other types of errors that can occur through application of multivariate statistical models with large scale surveys have been discussed by Huang (2013) and Suter (2019a). Huang shows that simple regression models of cross sectional surveys cannot correctly identify country differences in the relationships between status and OST participation (Huang, 2013), while Suter displays the distributions of both independent and dependent variables frequently applied in regression models to show that they are not equally distributed across countries and therefore the models will not display the more complex underlying relationships between status, time learning, and subject matter (Suter, 2019a). Researchers are tempted to assume that one item is equal across all countries and then include the item as a single measure across 40 countries in a multivariate model, not testing for underlying distributions or second order effects of the aggregate country economy or political situation (e.g., Baker et al., 2001) A number of good examples of use of large-scale research have been presented by European researchers in the Czech Republic, Croatia, Germany, Spain, and the U.K. They have used large-scale surveys to explore important issues of equity, computer education, and comparative analysis. These studies demonstrate that research with OST measured in large-scale surveys can provide new insights and explanations of the conditions that are necessary for student study time to affect student performance (Entrich, 2018; Jokić et al., 2013; Juhaňák, Zounek, Záleská, Bárta, & Vlčková, 2018; Klieme & Kuger, 2016; Lavy, 2015; Runte-Geidel, 2013) and should help lead toward improved analyses in future studies.
Although the PISA surveys have included OST items in each of its 7 surveys, the wording of the survey items on OST has been altered from year to year (OECD, 2011a; 2011b; 2017c) and different sets of countries have participated; therefore, measurement of change for any single country is not possible.
Moreover, few European countries participated in the PISA option of asking OST questions until 2018. The PISA OST items in each survey have only one trait in common which is that the students participated in study during time that was not outside the regular school day and was not assigned as homework (Bray & Kobakhidze, 2014b). Since the survey questions use words like “additional study” or remedial and enhanced tutoring that could be interpreted by some students differently (such as including or excludingparental instruction, group classes, peer assistance, teaching by college students, or instruction from a knowledgeable teacher within the school building), the survey items do not clarify the meaning of OST or shadow education (Klieme et al., 2005).
Future design of large-scale international comparative studies may benefit from the results of many individual country case studies and theoretical developments found in the European literature. The PISA and TIMSS items included in each survey are created by international researchers following an analysis framework that had been created for each survey cycle (Kuger et al., 2016; Mullis et al., 2015; OECD, 2016). The research frameworks include cognitive and subjective student attributes that are believed to be significant factors for using statistical measurement to explain student differences in academic achievement. The fact that the international study committees of each of these major surveys have included measurement of OST practices is evidence that the study designers believe that student activities outside of regular school time are important topics of investigation. The conceptual frameworks have not yet been proven to be empirically sound under all country conditions, as shown by analysis of the PISA indicators (Bray & Kobakhidze, 2014; Bray et al., 2020; Kuger et al., 2016). Further research and development is needed to reach agreement on such a framework.
Summary and conclusions
This summary selects issues about the measurement and meaning of OST that are apparent in the papers written about European practices with OST. The most common themes addressed in the research papers examined for this paper were how OST is defined (tutoring, assisted study, supplementary education, extended time for learning during school days, whether payment was required or not and more); whether OST increases inequality in education; and whether OST is an effective means of increasing student knowledge.
Most of the researchers believed that the main function of OST was to improve student performance for regular school programs (such as foreign language, native language, mathematics and science). Only in a few countries were systematic surveys of after-school programs conducted through their official statistics departments; thus researchers have attempted to estimate the level of participation by students and evaluating the outcomes for students and public-school systems with individually applied methods. Consequently, the definitions and research procedures are less standardized. The topics addressed by individual studies most frequently addressed cognitive levels of student learning while fewer papers discussed the effects of tutoring on social and emotional outcomes.
The research papers clearly noted that the meaning of OST has many nuances. Exchanges of research studies between countries may eventually lead to a common set of definitions. For example, in the experimental all-day German and English schools, the extension of regular school was accompanied by the adoption of typical OST activities conducted after-school in the United States (Kielblock, 2015; Miller, 2003). Thus, the experiments integrated the allocation of social and study time into required school time. Evaluations of acceptance and effectiveness of the reforms using quantitative methods of measurement found that the changes did not lead to the desired effect in the early stages of analysis. Consequently, the German researchers and policy makers continue to develop a more nuanced model for implementing OST experiences in the school system.
Should OST be evaluated and even conceptualized as an educational activity identical to regular education? If so, the model of OST would include measurements of test performance, grades, amount of time in class, time conducting homework, student motivation, measures of learning, teaching quality, curriculum opportunities to learn, financing levels and social status. Another standard for OST that has been recognized is its effect on a student’s social and psychological adjustment as measured by efficacy in school subjects and adjustment to social life. Since OST exists in a free market, the instruction available may not be tied strictly to the curriculum and assessment methods of formal school. Thus, a theoretical framework for identifying causes and effects of OST would have to include the additional measures such as degree of access and quality of instruction. For example, Stecher and Maschke (2013), were not satisfied with the current status of research on tutoring and proposed a general model of additional study that could include all-day schools and tutoring.
Researchers in comparative education have made significant contributions to the definition of OST and have provided insights into the structure and function of tutoring in many countries (Baker et al., 2001; Bray, 1999, 2009; Entrich, 2018). The framing of OST as a complement to public education, and sometimes competing with it, is a research structure that was found in nearly all countries. Examples of the changes in conceptualization of after-school activities as a significant component of increasing student achievement were found in England and Germany during the past two decades as the policies of the governments created programs to increase student involvement with positive educational experiences in off-school hours (Fordham, 2004; Klieme et al., 2005).
The distribution of resources for the costs of individual tutoring requires additional study. In some countries tutoring is provided for students as a fee-free service but only a few data sources exist to measure exactly where and how large the differences are between countries. Since OST education is rarely evaluated by a government agency, the supply and demand for tutoring usually operates in a free market. Studies of the supply and demand for instructors are necessary to provide insights into the process of all forms of OST in the future.
Since not all of the countries of Europe have participated in data collection organized by the IEA (The Trends in Mathematics and Science Study, TIMSS) or OECD (Programme for International Student Assessment, PISA) only a few research studies reviewed for this paper relied on those surveys. However, the international comparative studies have influenced the conceptual design of many reports. For example, the PISA reports of European educational comparisons have been instrumental in maintaining public and political interest on the test scores levels across countries. Researchers using these studies treat OST as a likely source of increasing performance on test scores (Long, 2014). The results of these analyses do not always support the claims of causal effect (Baker et al., 2001.; Huang, 2013) but additional study is needed to identify benefits and negative effects of OST. An examination of country differences in student participation in OST from the 2018 PISA for 18 European countries show that OST participation rates are higher for families of lower social status in all countries, the reverse of the predicted direction of many case studies. Thus, large-scale surveys are useful to generate and inform new hypotheses about the causes and effects of OST on student outcomes.
Topics for further study
The growth of research studies of OST noted in this review shows that many valuable conceptual developments and advances in data collection methods have been addressed by researchers in Europe. Some consistency regarding the purpose and function of OST and the definition of terms has occurred over the past 20 years thanks to extensive reporting from comparative studies (Bray, 2020). Frameworks have been designed for the international studies (Kuger et al., 2016) and for individual country studies (Fischer & Klieme, 2013).
This review of 214 research papers of OST in European countries has shown that the study of outof-school-time has contributed to educational policies and theories of learning. The sharing of conceptual models, such as shadow education, extended hours, additional study, and other identities, across countries has improved the integration of conceptual development. Further studies may lead to common frameworks and research questions. New researchers in Eastern Europe, Germany, England, France, Hungary and Portugal are making advances toward understanding the social and educational significance of student time use after school. The study of all aspects of OST will help advance understanding of educational practice generally including policy, measurement, learning methods, instructional methods, and the influence of family life on student outcomes.
References
Bae, S. H., & Hong, J. I. (2016). Expanding the horizon of research in extended education: perspective, fields, and methods. International Journal for Research on Extended Education, 4(2), 129–136.
Baker, D. P., Akiba, M., LeTendre, G. K., & Wiseman, A. W. (2001). Worldwide shadow education: outside-school learning, institutional quality of schooling, and cross-national mathematics achievement. Educational Evaluation and Policy Analysis, 23(1), 1–17.
Bell, P., Lewenstein, B., Shouse, A., & Feder, M. (Eds.) (2009). Learning science in informal environments: People, places, and pursuits. Washington DC: The National Academies Press.
Berberoğlu, G., & Tansel, A. (2014). Does private tutoring increase students’ academic performance? Evidence from Turkey. International Review of Education, 60(5), 683–701.
Berliner, D. C. (1990). What’s all the fuss about instructional time. In M. Ben-Peretz & R. Bromme (Eds.), The nature of time in schools: Theoretical concepts, practitioner perceptions (pp. 3–35). New York: Teachers College Press.
Bonn, M. & Haag, L. (2002). Expansion and effectiveness of private tutoring. European Journal of Psychology of Education, 17(3), 263–273.
Bray, M. (1999). The shadow education system: Private tutoring and its implications for planners. Fundamentals of Educational Planning 61. Paris: UNESCO International Institute for Educational Planning.
Bray, M. (2010). Researching shadow education: Methodological challenges and directions. Asia Pacific Education Review, 11(1), 3–13.
Bray, M. (2011). The challenge of shadow education: Private tutoring and its implications for policy makers in the European Union. Brussels: European Comission.
Bray, M. (2013). Shadow education: Comparative perspectives on the expansion and implications of private supplementary tutoring. Procedia-Social and Behavioral Sciences, 77, 412–420.
Bray, M. (2014). The impact of shadow education on student academic achievement: Why the research is inconclusive and what can be done about it. Asia Pacific Education Review, 15(3), 381–389.
Bray, M. (2020). Shadow education in Europe: Growing prevalence, underlying forces, and policy implications. ECNU Review of Education, [East China Normal University], 3. https://doi.org/10.1177/2096531119890142.
Bray, M., & Kobakhidze, M. N. (2014a). The global spread of shadow education. In D.B. Napier (Ed.), Qualities of education in a globalized world (pp. 185–200). Rotterdam: Sense Publishers.
Bray, M., & Kobakhidze, M. N. (2014b). Measurement issues in research on shadow education: Challenges and pitfalls encountered in TIMSS and PISA. Comparative Education Review, 58(4), 590–620.
Bray, M., Kobakhidze, M. N., & Suter, L. (2020). The challenges of measuring outside-school-time educational activities: Experiences and lessons from the Programme for International School Assessment (PISA). Comparative Education Review, 64(1), 87–106.
Bray, M., Mazawi, A. E., & Sultana, R. G. (2013). Introduction: Situating private tutoring. In M. Bray, A. Mazawi & R.G. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 1–10). Rotterdam: Sense Publishers.
Bray, M., & Thomas, R. M. (1995). Levels of comparison in educational studies: Different insights from different literatures and the value of multilevel analyses. Harvard Educational Review, 65(3), 472490.
Brehm, W. C., & Silova, I. (2014). Hidden privatization of public education in Cambodia: Equity implications of private tutoring. Journal for Educational Research Online, 6(1), 94–116.
Buchmann, C., Condron, D. J., & Roscigno, V. J. (2010). Shadow education, American style: Test preparation, the SAT and college enrollment. Social Forces, 89(2), 435–461.
Buhagiar, M. & Chetcuti, D. (2013). The private tuition phenomenon in Malta: Moving toward a fairer education system. In Bray, M, Mazawi, A.E., & Sultana, R. G. (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity. Rotterdam: Sense Publisher.
Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64, 723–733.
Carroll, J. B. (1989). The carroll model: A 25-year retrospective and prospective view. Educational Researcher, 18(1), 26–31.
Chappell, S., Nunnery, J., Pribesh, S., & Hager, J. (2011). A meta-analysis of supplemental educational services (SES) provider effects on student achievement. Journal of Education for Students Placed at Risk, 16(1), 1–23.
Covay, E., & Carbonaro, W. (2010). After the bell: participation in extracurricular activities, classroom behavior, and academic achievement. Sociology of Education, 83(1), 20–45.
Deutsche Welle (2003). Germany moves to all-day schools. DW-World.de. Retrieved from www.dwworld.de/dw/article/0,864144,00.html. Accessed 30 May 2020.
D’Agostino, J. V., & Murphy, J. A. (2004). A meta-analysis of Reading Recovery in United States schools. Educational Evaluation and Policy Analysis, 26(1), 23–28.
Ecarius, J., Klieme, E., Stecher, L., & Woods, J. (Eds.), (2013). Extended education: An international perspective: Proceedings of the international conference on extracurricular and out-of-school time educational research. Opladen: Barbara Budrich Publishers.
Elbaum, B., Vaughn, S., Tejero Hughes, M., & Watson Moody, S. (2000). How effective are one-to-one tutoring programs in reading for elementary students at risk for reading failure? A meta-analysis of the intervention research. Journal of Educational Psychology, 92(4), 605.
Entrich, S. R. (2014). Effects of investments in out-of-school education in Germany and Japan. Contemporary Japan, 26(1), 71–102.
Entrich, S. R. (2018). Worldwide Shadow Education and Social Inequality: When do shadow systems become a threat to equality of educational opportunities. A quantitative analysis across 63 countries paper presented at the XIX. ISA World Congress of Sociology Toronto 15–21 July. Metro Toronto Convention Centre, Toronto (CA).
Faganel, A., & Trnavčevič, A. (2013). Constructions of private tutoring in Slovenian online chatrooms: A content analysis. In M. Bray, A. Mazawi & R.G. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 167–176). Rotterdam: Sense Publishers.
Farbman, D. A. (2015). The case for improving and expanding time in school: A review of key research and practice. Updated and Revised February 2015. Retrieved from https://files.eric.ed.gov/fulltext/ED561994.pdf. Accessed 30 May 2020.
Fischer, N., Brümmer, F., & Kuhn, H. P. (2011). Entwicklung von Wohlbefinden und motivationalen Orientierungen in der Ganztagsschule: Zusammenhänge mit der Prozess-und Beziehungsqualität in den Angeboten [Development of well-being and motivational orientations in the all-day school:Connections with the process and relationship quality in the offers]. Paper presented at the Ganztagsschule: Entwicklung, Qualität, Wirkungen. Längsschnittliche Befunde der Studie zur Entwicklung von Ganztagsschulen (StEG).
Fischer, N., Holtappels, H. G., Klieme, E., Rauschenbach, T., Stecher, L., & Züchner, I. (2011). Ganztagsschule: Entwicklung, Qualität, Wirkungen: Langsschnittliche Befunde der Studei zur Entwicklung von Ganztagschulen (StEG), Studien zur ganztagigen Bildung [All-day school: Development, quality, effects: Longitudinal findings of the study on the development of all-day schools (StEG), studies on all-day edcation] (pp. 227–244). Weinheim: Beltz Juventa.
Fischer, N., & Klieme, E. (2013). Quality and effectiveness of German all-day schools: Results of the study on the development of all-day schools. In J. Ecarius, E. Klieme, L. Stecher & J. Woods (Eds.), Extended education: An international perspective: Proceedings of the international conference on extracurricular and out-of-school time educational research (pp. 27–52). Opladen: Barbara Budrich Publishers.
Fordham, I. (2004). Out‐of‐school‐hours learning in the United Kingdom. New Directions for Youth Development, 43–74.
Fraij, A., Janzen, F., & Kielblock, S. (2017). Recommended readings in extended education. International Journal for Research on Extended Education, 5(1). 117–120.
Galinié, A., & Heim, A. (2016). Inégalités scolaires: Quels rôles jouent les cours privés? [Educational inequalities: What roles do private lessons play?]. Paris: UNESCO.
Gromada, A., & Shewbridge, C. (2016). Student learning time: A literature review. Paris: OECD Publishing.
Guill, K., Lüdtke, O., & Köller, O. (2019). Assessing the instructional quality of private tutoring and its effects on student outcomes: Analyses from the German National Educational Panel Study. British Journal of Educational Psychology, 90(2), 1–19.
Heyneman, S. P. (2011). Private tutoring and social cohesion. Peabody Journal of Education, 86(2), 183–188.
Holloway, S.L. & Pimlott-Wilson, H. (2020). Marketising private tuition: Representations of tutor’ competence, entrepreneurial opportunities and service legitimation in home tutoring business manuals. British Educational Research Journal, 46(1), 205–221.
Holtappels, H. G., Kamski, I., & Schnetzer, T. (2007a). Ganztagsschule im Spiegel der Forschung: zentrale Ergebnisse der Ausgangserhebung der “Studie zur Entwicklung von Ganztagsschulen” (StEG)-eine Informationsbroschüre. [All-day school in the mirror of research: Central results of the initial survey of the “Study on the development of all-day schools” (StEG): An information brochure]. “Werkstatt” Entwicklung und Organisation von Ganztagsschulen”, Institut für Schulentwicklungsforschung (IFS): Deutsche Kinder-und Jugendstiftung.
Holtappels, H. G., Klieme, E., Rauschenbach, T., & Stecher, L. (2007b). Ganztagsschule in Deutschland: Ergebnisse der Ausgangserhebung der “Studie zur Entwicklung von Ganztagsschulen” (StEG). [All-day school in Germany: Results of the initial survey of the “Study on the development of all-day schools”]. Weinheim: Juventa.
Huang, M. H. (2013). After-school tutoring and the distribution of student performance. Comparative Education Review, 57(4), 689–710.
Húsen, T. (1972). Does time in school make A difference? Saturday Review, 55(18), 32–35.
Ireson, J., & Rushforth, K. (2005). Mapping and evaluating shadow education. London: Institute of Education, University of London.
Jokić, B., Soldo, A., & Dedić, Z. R. (2013). Private tutoring and social equity in Croatia and Bosnia & Herzegovina: A comparative qualitative study. In M. Bray, A. Mazawi & R. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 1127). Rotterdam: Sense.
Juhaňák, L., Zounek, J., Záleská, K., Bárta, O., & Vlčková, K. (2018). The relationship between students’ ICT use and their school performance: Evidence from PISA 2015 in the Czech Republic. Orbis Scholae, 12(2), 37–64.
Jun, S., Ramirez, G., & Cumming, A. (2010). Tutoring adolescents in literacy: A meta-analysis. McGuill Journal of Education, 45(2), 219–238.
Karweit, N. (1984). Time-on-task reconsidered: Synthesis of research on time and learning. Educational Leadership, 41(8), 32–35.
Kielblock, S. (2015). Program implementation and effectiveness of extracurricular activities: An investigation of different student perceptions in two German all-day schools. International Journal for Research on Extended Education, 3(2), 79–98.
Kim, Y. C., & Jung, J.-H. (2019a). Conceptualizing shadow curriculum: Definition, features and the changing landscapes of learning cultures. Journal of Curriculum Studies, 51(2), 141–161.
Kim, Y. C., & Jung, J.-H. (2019b). Shadow education as worldwide curriculum studies. New York, NY: Palgrave Macmillan.
Klieme, E., & Kuger, S. (2016). PISA 2015 context questionnaires framework. In OECD (Ed.), PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy (pp. 101– 127). Paris: OECD Publishing. https://doi.org/10.1787,9789264255425-7-en.
Klieme, E., Kühnbach, O., Radisch, F., & Stecher, L. (2005). All-day learning: Conditions for fostering cognitive, emotional, and social development. Frankfurt am Main: DIPF.
Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras study: Investigating effects of teaching and learning in Swiss and German mathematics classrooms. In T. Janik & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 137–160). Münster: Waxmann.
Kobakhidze, M. N. (2018). Teachers as tutors: Shadow education market dynamics in Georgia. Springer.
Kobakhidze, M. N. & Suter, L.E. (2020). The global diversity of shadow education. European Journal of Education, 55(3), 316–321.
Kuger, S., Klieme, E., Jude, N., & Kaplan, D. (2016). Assessing contexts of learning. Springer.
Lamprianou, I., & Lamprianou, T. A. (2013). Charting private tutoring in Cyprus: A socio-demographic perspective. In M. Bray, A. Mazawi & R.G. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 29–56). Rotterdam: Sense Publishers.
Lavy, V. (2015). Do differences in schools’ instruction time explain international achievement gaps? Evidence from developed and developing countries. The Economic Journal, 125(588), F397–F424.
Lee, C. J., Lee, H., & Jang, H.M. (2010). The history of policy responses to shadow education in South Korea: Implications for the next cycle of policy responses. Asia Pacific Education Review, 11(1), 97–108.
Long, D. A. (2014). Cross-national educational inequalities and opportunities to learn: Conflicting views of instructional time. Time Educational Policy, 28(3), 351–392.
Luplow, N. and T. Schneider (2014), “Nutzung und Effektivität privat bezahlter Nachhilfe im Primarbereich” Social Selectivity and Effectiveness of Private Tutoring among Elementary School Children in Germany. Zeitschrift für Soziologie, 43(1), 31–49.
MacBeath, J. E., Kirwan, T., Myers, B., McCall, J., Smith, I., & McKay, E. (2001). The impact of study support: A report of a longitudinal study into the impact of participation in out-of-school-hours learning on the academic attainment, attitudes and school attendance of secondary school students. London: Department for Education and Skills.
Mahoney, J. L., Larson, R. W., & Eccles, J. S. (Eds.), (2005). Organized activities as contexts of development: Extracurricular activities, after school and community programs. Mahwah: Lawrence Erlbaum.
Manzon, M., & Areepattamannil, S. (2014). Shadow education s: Mapping the global discourse. Asia Pacific Journal of Education, 34(4), 389–402.
Marsh, H. W. (1992). Extracurricular activities: Beneficial extension of the traditional curriculum or subversion of academic goals? Journal of Educational Psychology, 84(4), 553–562.
Matterson, C., & Holman, J. (2012). Informal science learning review: Reflections from the wellcome trust. London: Wellcome Trust.
Mazawi, A.E., Sultana, R.G., & Bray, M. (2013). Beyond shadows. In M. Bray, A. Mazawi & R.G. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 205–216). Rotterdam: Sense Publishers.
Miller, B. M. (2003). Critical hours: Afterschool programs and educational success. Brookline, MA: Miller Midzik Research Associates.
Mullis, I. V., Martin, M. O., Kennedy, A., Trong, K., & Sainsbury, M. P. (2015). Assessment frameworks. Boston: TIMMS and PIRLS International Study Center, Boston College.
Neto-Mendes, A., Costa, J. A., Ventura, A., Azevedo, S., & Gouveia, A. (2013). Private tutoring in Portugal: Patterns and impact at different levels of education. In M. Bray, A. E. Mazawi & R. G. Sultana (Eds.), Private tutoring across the mediterranean: Power dynamics and implications for learning and equity (pp. 151–165). Rotterdam: Sense Publishers.
Noam, G., & Shaw, A. (2013). Afterschool, informal science, and the assessment predicament: A gamechanging strategy. Cambridge, MA: PEAR - Program in Education, Afterschool, and Resiliency.
OECD (2011a). Does investing in after-school classes pay off? Paris: OECD Publishing.
OECD (2011b). Education at a glance .Paris: OECD Publishing.
OECD (2011c). Quality time for students: Learning in and out of school. Paris: OECD Publishing.
OECD (2012). Strengthening Integrity and fighting corruption in education. Paris: OECD Publishing.
OECD (2014). Education at a glance 2014. Paris: OECD Publishing.
OECD, P. (2016). Results (Volume I): Excellence and equity in education. Paris: OECD Publishing.
OECD (2017a). PISA 2015 results (Volume II) .Paris: OECD Publishing.
OECD (2017b). PISA 2015 results (Volume III): Students’ well-being. Paris: OECD Publishing.
OECD (2017c). Do students spend enough time learning? Paris: OECD Publishing.
Rappleye, J., & Komatsu, H. (2018). Stereotypes as Anglo-American exam ritual: Comparisons of students’ exam anxiety in East Asia, America, Australia, and the United Kingdom. Oxford Review of Education, 44(6), 730–754.
Runte-Geidel, A. (2013). La incidencia de las clases particulares en España a través de los datos de PISA [The incidence of private classes in Spain through PISA data]. Revista Española de Educación Comparada, (21), 249–284.
Schiefer, J., Golle, J., Tibus, M., Herbein, E., Gindele, V., Trautwein, U., et al. (2019). Effects of an extracurricular science intervention on elementary school children’s epistemic beliefs: A randomized controlled trial. British Journal of Educational Psychology, 90(2), 382–402.
Schüpbach, M. (2014). Extended education and social inequality in Switzerland: Compensatory effects?: An analysis of the development of language achievement with regard to structural and process-related aspects of social background. Journal for Educational Research Online, 6(3), 95–114.
Schüpbach, M., & Lilla, N. (2019). Extended education from an international comparative point of view: WERA-IRN extended education conference Volume. Springer.
Silova, I. (2010). Private tutoring in eastern Europe and central Asia: policy choices and implications. Compare: A Journal of Comparative and International Education, 40(3), 327-344.
Silova, I. Būdienė, V. and Bray, M. (Eds.), (2006). Education in a hidden marketplace: Monitoring of private tutoring. New York: Open Society Institute.
Smyth, E. (2008). The more, the better? Intensity of involvement in private tuition and examination performance. Educational Research and Evaluation, 14(5), 465–476.
Stecher, L., & Maschke, S. (2013). Out-of-school education: A new chapter in educational research. In J. Ecarius, E. Klieme, L. Stecher, & J. Woods (Eds.), Extended education: An international perspective: Proceedings of the international conference on extracurricular and out-of-school time educational research (pp. 11–26). Barbara Budrich: Opladen.
Steinhäuser, H. (2019). Challenges, concepts and support structures in Bavarian all-day schools. In J. Ecarius, E. Klieme, L. Stecher, & J. Woods (Eds.), Extended education: An international perspective: Proceedings of the international conference on extracurricular and out-of-school time educational research (pp. 185–192). Opladen: Barbara Budrich Publishers.
Stevenson, D. L., & Baker, D. P. (1992). Shadow education and allocation in formal schooling: Transition to university in Japan. American Journal of Sociology, 97(6), 1639–1657.
Suter, L. E. (2014). Visiting science museums during middle and high school: A longitudinal analysis of student performance in science. Science Education, 98(5), 815–839.
Suter, L. E. (2016). Outside school time: An examination of science achievement and non-cognitive characteristics of 15-year olds in several countries. International Journal of Science Education, 38(4), 663–687.
Suter, L. E. (2019a). Results of an exploratory analysis of PISA 2015 survey of student participation in outside-school-time programs. International Journal for Research on Extended Education, 7(1), 36–59.
Suter, L. E., (2019b). The status of comparative education research in the 21st century: An empiricist’s view. In L. E. Suter, B. Denman, & E. Smith (Eds.), The SAGE handbook of comparative studies in education (pp. 3–24). London: Sage Publishing.
Suter, L. E., Smith, E., & Denman, B. D. (Eds.), (2019). The SAGE handbook of comparative studies in education. SAGE Publications Limited.
Šťastný, V. (2014). Soukromé doučování a vzdělávací politiky v Evropě [Private supplementary tutoring and education policies in Europe]. Pedagogická Orientace, 24(3), 353–374.
Šťastný, V. (2016). Private supplementary tutoring in the Czech Republic. European Education, 48(1), 1–22.
Šťastný, V. (2017). Private tutoring lessons supply: Insights from online advertising in the Czech Republic. Compare: A Journal of Comparative and International Education, 47(4), 561–579.
Tsiplakides, I. (2018). Shadow education and social class inequalities in secondary education in Greece: The case of teaching English as a foreign language. International Journal of Sociology of Education, 7(1), 71–93.
U.N. Statistics Division. (2020). Methodology: Standard country or area codes for statistical use (M49). United Nations, Department of Economic and Social Affairs, Statistics Division. Source: https://unstats.un.org/home/.
Verger, A., Fontdevila, C., & Zancajo, A. (2016). The privatization of education: A political economy of global education reform. New York: Teachers College Press.
Zaff, J., & Redd, Z. (2001). Logic models and outcomes for out-of-school time programs: Report to the DC children and Youth Investment trust Corporation. Washington, DC: Children and Youth Investment Trust Corporation.
This paper benefited greatly from advice provided by Nutsa Kobakhidze, and two excellent anonymous journal reviewers. Only the authors are responsible for all remaining errors. The reference list does not contain all of the research papers that were used in the analysis of trends in OST research in Europe. Another 300 paper titles are available upon request: larrysuter@me.com.