Definitions of study success

As noted above, defining “study success” in an international comparative context is difficult (Thomas & Hovdhaugen, 2014); hence, the reliance so far in this article on the broad and vague phrase “study success.” Thus, it is not surprising (see Hagedorn, 2004) that the HEDOCE research identified a range of interpretations relating to the phrase “study success.” However, even when definitions appear explicit and widespread, interpretation, implementation, and consensus can diverge in practice between different stakeholders in the HE sector and beyond. Drawing on the literature, it is useful to clarify some key terms.

The first is “completion” or the “completion rate,” which is usually understood as the number of students who have successfully finished a study program at a HEI compared to the number who started. Although apparently a straightforward calculation, difficulties and concomitant inaccuracies can arise (see Chalmers, 2008; Thomas & Hovdhaugen, 2014). However – and usually – completion rates refer to a cohort and a point in time when it can be assumed that most students would have completed their study program (e.g., 1–2 years after the nominal study time). The length of time considered “normal” or acceptable varies between countries. This has led to another definition and associated indicator: time to degree – which emerged as the most popular definition across Europe in the HEDOCE project – but which refers to vastly different periods of time that are deemed acceptable and “normal.”

Next is retention, or continuation (rate). In its simplest sense, retention means the number of entrants who remain on a given course at a selected census point. This becomes a little more complicated when used to signify reenrollments over longer periods, and one potential problem in calculating retention or continuation rates is that it may include “limbo” students who are not actively studying for a degree, yet have not formally withdrawn either – and, or, they have reenrolled but they are not gaining credits. In the US, and beyond, these are sometimes known as “stop-outs” as opposed to dropouts.

“Dropout” or “withdrawal” (rate) refers to those who leave HE before completing their target award. In this paper, the term “dropout” is avoided as it is pejorative, and withdrawal or non-continuation is used in preference. As with other definitions, the major issue here is to identify the two groups of students that need to be compared in order for the relevant indicators to produce accurate data. It can be difficult to define and measure as students do not always formally withdraw but rather drift away, while others may take a break from their studies and reenter picking up at the same point as they left off. This makes it difficult to know at what point should they be deemed to have left the system for good.

Both “Switching” and “transfer” rates refer to the number of students that change their program of study and, or, HEI. Again, the basis for the calculation is mostly the entrance cohort. Transfer rates are calculated for different points in time, but mostly they refer to the switch of students after the first year of study; in some national systems, this is easier to do and easier to measure than in other systems.

In this study, the generic term “study success” is used to encompass student retention and completion, in contrast to withdrawal or drop-out and non-completion: “Study success comprises all major achievements of students in the higher education system, including dropout/persistence, completion of a degree and time-to-degree” (Vossensteyn et al., 2015, p. 24). The HEDOCE study examined interpretations of study success across Europe based on three definitions:

1. –Completion: to have students successfully complete their study program with a degree.
2. –Time-to-degree: to have students complete their study program within a reasonable time period.
3. –Retention: the aim to have students reenroll in a study program until they complete their degree and to reduce the likelihood they drop out before completing their program.

Responses from 35 European countries indicated that “time-to-degree” is the most frequently used understanding of study success underpinning national policy making in Europe (18 countries), with completion being the second most common (13 countries).

This response however provides a rather simplistic understanding of the multifaceted and contested nature of study success across and within European countries. The case-study research revealed that, only in England, there is an explicit definition that is widely agreed upon by all stakeholders of what constitutes study success. In Germany, stakeholders have some degree of consensus on what they regard as study success – all of them seeing it as “the successful completion of an academic degree,” but this has not resulted in a more specific definition. In all other countries, there are variations in the definition of study success among stakeholders; several understandings of the concept exist side by side, and stakeholders choose to use the type of definition that they find most suitable. In Italy, for example, several stakeholders identified “finding employment” as study success, while “completing the degree,” particularly in a “timely fashion” was important to other stakeholders. In other countries, such as Norway and the Netherlands, timely completion is regarded as the most important understanding of study success, at least at the national policy level. The Norwegian case study however finds that HEIs do not tend to share the view that completion within a limited time period is paramount, but rather see completion at some point to be more important. Indeed, Norwegian student unions are concerned that there is too much emphasis on efficiency at the expense of the quality of the students’ study experience and attainment levels.

These examples illustrate the divergence of interpretations of study success and its definitions across European HE systems – and crucially within national HE systems. This in turn then makes it hard to identify the relevant policies that seek to promote or stimulate study success and address student withdrawal.

National policies for study success

The lack of consensus relating to definitions of “study success” might be taken as an indicator of the lack of priority accorded to it by policy makers and HEIs and others in the HE sector. However, the survey of national experts from 35 European countries suggested that study success is widely regarded as an important topic across Europe. In the majority of countries (28, which is 80%), experts indicated that study success is a policy issue that is being given consideration, while 16 country experts stated that study success is high or very high on the policy agenda. Experts in only seven countries indicated that study success is not really a matter for concern within their country’s HE policy agenda.

The study collected details of national policies and approaches used to address study success from a 10-year period, i.e., 2005–2015. The information was collected in a variety of ways, including two surveys with national experts and desk research. In the majority of countries, universities and other HEIs are actively engaged in improving study success, and governmental authorities such as ministries are engaged in some countries – approximately two thirds of the countries reviewed have study success policies in place, and about half of the countries have implemented regulations to improve study success. In total, more than 170 policies that explicitly and intentionally address study success were identified.

Drawing on the literature, three broad national policy areas were identified: “funding and financial incentives,” “organization of HE,” and “information and support for students” being used to explicitly address study success.

1. –Funding and financial incentives: Financial policies often include incentives to encourage (or discourage) specific activity. Such policies can directly target students, e.g., through tuition fees, grants, scholarships, or loans; financial incentives can be used to influence students’ study success behavior, for example, by linking financial reward to credits gained. They can also influence HE providers – but here such policies tend to work indirectly by “inviting” HEIs to develop their own policies.
2. –Organization of HE: This policy area refers primarily to teaching and learning but can also include structural characteristics such as the duration of programs or the types of degrees offered. Policy detail can be delegated, i.e., devolved to HEIs. Organizational regulations can address the quality of teaching and learning (and its accreditation, etc.), and also things such as student–teacher ratios, number of contact hours, assessment regulations, pathways toward a degree or “soft selection mechanisms” such as applicant interviews, etc., to influence who has access to HE courses.
3. –Information and support for students: These policies relate to different stages in the student “life cycle.” Prospective students (and those transferring or thinking about leaving) are provided with information to inform their decisions. Information and support, pre- and post-entry provide course, study and career information, and address academic development and attainment, personal well-being, and professional development.

The 170 policies aiming to improve study success were divided into these three categories, and 22 “typical” policies were identified. A typical policy is one that is used in several countries and informed by similar rationales across these countries.

Nine typical policies were identified focusing on the organization of HE – and more specifically, the study experience. This may include “access” (what kinds of students governments and HEIs want to attract to HE) but also covers the way in which degree programs are structured. Finally, it encompasses changes to the curriculum design and associated pedagogy, such as a focus on the first year transition experience, different assessment regimes, and developing students’ academic confidence and skills.

Eight typical policies were identified focusing on funding – these can be divided into those focusing on student financing issues and other addressing funding for HE providers. Financial support for students can help them focus on their studies, improving both success and completion rates. Conversely, tuition fees may encourage students to choose their courses more carefully and to study efficiently. Public funding to HEIs often includes incentives to make institutions prioritize study success – e.g., through performance-based funding rewarding credits gained or degrees completed. In approximately a third of the countries, “study success” (however it is defined) plays a role in determining the funding of HEIs; this is especially true in countries study success is high or very high on the policy agenda. It is important to note that both student and institutional funding works in positive or negative ways, i.e., by incentivizing or rewarding some behavior through funding, and discouraging other behavior or outcomes through punitive measures.

Five typical policies focusing on information and (non-financial) support for students were identified. These include the provision of information and support to students – both prior to and after enrollment, and the provision of additional support for students, such as guidance with course choices, academic development, and health and welfare services. The provision of information and guidance is intended to improve the fit between students and their programs of study, and to shape and manage students’ expectations with regard to their studies and future employment careers. This includes national information services, such as student choice portals, league tables or institutional matching instruments, capability, and interest tests. Support services include extracurricular courses to enhance skills, competencies, and academic preparedness of students, e.g., in reading, writing, and language skills; specialist support provisions for students with a physical or learning disability; and health and mental well-being services. The main rationale underlying such policies is that a better match between students and programs as well as a growth in competencies are regarded as prerequisites for successful study.

Indicators and outcomes for study success

In light of the differing definitions of study success, and the varying ways in which national policies address it, it is perhaps not surprising that gauging the effectiveness of policy interventions is often difficult. This is the case both within and between countries; in this paper, the focus is on measurement within a national system.

Indicators of “study success” are difficult to operationalize and calculate, and the literature illustrates that study success is measured in very different ways (Hagedorn, 2004; Thomas & Hovdhaugen, 2014), and this is reflected in the findings from this study. In this study (Vossensteyn et al., 2015), indicators were sought in relation to any of the three definitions of study success articulated previously: completion rate, retention or continuation rate (or dropout or withdrawal rate), and time-to-degree (based on the length of time, it takes students to complete their undergraduate studies).

National experts indicated that 12 countries publish an indicator for completion and 23 do not, even though some of them have the data to calculate such a completion rate. Only six countries reported that some kind of retention rate is available, although for Denmark the indicator refers to withdrawal. For only a quarter of the countries, information on time-to-degree was identified. It should be noted that some countries have more than one indicator, but nearly half the European countries (17 out of 35) do not appear to have any national indicator and data about study success; this is surprising as only seven countries reported that study success is absent from the national HE policy agenda. Furthermore, there seems to be a mismatch between definitions of study success and indicators. For example, 18 countries identified time-to-degree as the national focus of study success, yet this is only an indicator in half of these countries.

Czech Republic

The Czech Republic has been selected as an example of a Central European country, which is broadly representative of the Visegrad group of countries: Czech Republic, Hungary, Poland, and Slovakia (Stiburek, Vlk, & Svec, 2017), which have much shared history and are working together toward greater integration within Europe. In the Czech Republic, study success is on the HE policy agenda, although it is not seen as a high priority (Vossensteyn et al., 2015), and some of the concern placing it on the agenda is externally driven by the European Commission (Stiburek et al., 2017; Vlk, 2015). Data from the Czech Ministry of Education, Youth and Sports indicate that withdrawal rates are increasing, e.g., from 38% in 2003 to 48% in 2010, and this trend is continuing with less than 50% of students who enrolled in a bachelor’s degree in 2009 being awarded the target qualification (Vlk, 2015).

In the Czech Republic, successful study is broadly understood to mean completing the target program of study, and withdrawal includes students who have transferred to other programs. Retention refers to students progressing from the first to second year of undergraduate study. However, the extent to which there is concern about study success is largely in terms of the efficiency of the HE system. Therefore, this shifts the policy agenda toward consideration of time to completion, and completing a degree within a specific time period (Stiburek et al., 2017), although there does not appear to be a formal definition specifying a time period (c.f. Vlk, Stiburek, & Svec, 2016). Rather than achieving a degree, or completing with a specific time period, students view study success in terms of employment outcomes, and some academic staff also share this perspective (Vlk, 2015). Thus, within the Czech Republic, there are some definitions and measures of study success, but these are not widely accepted and shared and there are competing alternatives based on time to completion and employment outcomes. There is, arguably, a need for “a thorough debate on dropout calculation and comparison with other countries” (Vlk et al., 2016, p. 651).

This lack of engagement in defining study success may be because improving student retention and completion (or conversely reducing student withdrawal) is not perceived as a priority by the majority of HE stakeholders; rather a high rate of non-completion is viewed as an essential aspect of the Czech HE system (Stiburek et al., 2017; Vlk, 2015). In other words, academic failure and student withdrawal operate as ex post selection mechanisms, compensating for limited restrictions on who gains access to HE. A lack of selectivity have been exacerbated by a demographic decline in 19-year olds resulting in the lowering of entry qualifications and European efforts to expand the diversity of participation in HE as part of the social dimension of the Bologna process (noted above).

A vague or non-universal understanding of study success, and a lack of priority to accord to addressing student withdrawal and non-completion seem to contribute to an ineffective national policy approach. At the national level, there are very few policies explicitly designed to address or improve study success (Vlk, 2015), and policies are vague, and other regulations and policies undermine efforts to improve study success, or at least render them less effective.

The HE Strategic Plan for 2000–2005 identified the need to enable students who make poor study choices to switch programs and complete their courses, and drop-out was noted as an inefficiency in the system for the government – and for students. However, no specific measures of dropout were identified and very little detail was provided as to how this matter should be addressed (Ministry of Education, Youth and Sports, 2000; Vlk, 2015). However, HEIs were asked to publish some basic data about student withdrawals, and their associated measures to reduce the rates of withdrawal in their annual reports. Notwithstanding, the recommended structure of an annual report, set by the MEYS, is not obligatory to follow, and no shared definitions of dropout are set for this purpose (Vlk, 2015).

In drawing up the current Strategic Plan for 2016–2020, efforts were made to by the Ministry of Education, Youth and Sports to address student withdrawal more directly, but these were rejected by the HEIs. Instead, institutions are required to “identify the causes of increasing dropout rates.” To support this process, the government intends to provide funding to institutions, drawn from the European Structural and Investment Fund, and national funding. However, within universities, study success policy and management are decentralized, with faculties, departments, or even courses being responsible for this issue. (Vlk, 2015). There appear to be no direct consequences for institutions that do not address study success in their annual plans, other than some financial implications. Vlk (2015) notes that the government is linking retention to institutional funding, and higher rates of withdrawal may lead to a reduction in the number of funded student places; thus, the aim is to incentivize institutions to address early withdrawal. The impact of this financial measure will however depend on the proportion of income that is affected. It appears that the efficacy of this policy tool is likely to be limited as only a decreasing element of teaching funding is per capita, and the overarching structure of funding for HE encourages institutions to prioritize research over learning and teaching (as research generates income and teaching largely does not).

The wider institutional funding model incentivizes institutions to act in the opposite direction. Since 2010, a quality funding element has been introduced; thus, institutional funding is based on a mix of per capita funding and quality-related funding, with a gradual increase in the latter (Vlk, 2015). In a national context, in which withdrawal may actually be perceived as a positive aspect of quality, this funding strategy may inadvertently be driving institutional behaviors against study success.

Therefore, the focus on quality seems to be hindering efforts to improve study success, but this is largely a consequence of the way in which quality is conceptualized and implemented. The national quality assurance regime reviews the academic qualifications of staff and their research performance, with little focus on curriculum, pedagogy, assessment, and student support, and indeed no site visits take place (Vlk, 2015). There appears to be little focus on the organization and contents of HE programs, and how this might contribute to study success.

Indeed, the problem is further compounded by the implementation of the Bologna process (moving from long-cycle degrees, to the bachelor and masters structure of three plus two years). These changes have been implemented rather mechanistically, splitting up existing courses, rather than redesigning them (Vlk, 2015). This has resulted in some programs being unattractive at undergraduate level, as they are overly theoretical with little of the more engaging applied contents, which is delivered subsequently, especially in engineering and professional fields such as teacher education. These kinds of organizational issues are likely to increase rather than decrease academic failure and student withdrawal.

There appears to be little attention paid to providing student information and support, either prior to entry or once they are in HE. The Higher Education Strategic Plan (2000–2005) identified the issue of students switching courses, due to poor course choices. According to Vlk (2015), a portal is planned to provide prospective students with better information to inform their choices about progression to HE, and funding penalties for students are encouraging completion on time. The lack of on-course support reinforces the view that withdrawal is a positive aspect of quality assurance in some HE systems, and is not discouraged, whereas in other systems, efforts are made to support students to remain on their courses by addressing academic challenges, personal crises, etc.

In summary, it might be considered that there is a strong and pressing need for a national approach to improving study success. The rates of withdrawal and non-completion are increasing, and this is within a context of declining student numbers (due to demographic factors and in some subject areas, there occurs lower demand for the courses) and increasing diversity across the system, which appears not to be recognized or catered for the HE system. What appears to be missing in the Czech case, and potentially across the Visegrad countries, is national policy tools that aim to change or even transform the HE-learning experience itself. International research, e.g., from the US (Troxel, 2010) and Australia (Krause & Armitage, 2014), and European research (Thomas, 2012; Ulriksen, Madsen, & Holmegaard, 2010; Yorke & Longden, 2004) has identified the importance of the institution and institutional commitment to study success. In particular, research from Germany and the UK has identified the importance of learning, teaching, and assessment (e.g., Georg, 2009; Thomas, 2012), which is designed to improve student engagement and belonging. UK research identifies four dimension aspects of the academic experience that contributes to a sense of belonging – and retention and success. These are engagement with peers, interaction with staff, a developed capacity to engage, and an HE curriculum that is relevant to current interests and future aspirations (Thomas, 2012; Thomas, Hill, O’Mahony, & Yorke, 2017). In an institution with a strong commitment to study success, curriculum changes and additional support for study success are underpinned by monitoring students’ engagement and performance, and intervening when behavior indicates that they are at risk of early withdrawal (Thomas et al., 2017), allowing targeted interventions (Heublein, Schmelzer, & Sommer, 2008; Thomas, 2012). In the UK context, Buglear (2009) finds that poor data often underpin the institutions’ inabilities to intervene adequately to improve retention, and improved tracking is recommended by UK and other European research (Larsen, Sommersel, & Larsen, 2013; Thomas, 2012; Thomas et al., 2017). In the following case study, these and related issues are reviewed in the English context.

England

England was selected as a case study partly as it was identified by Vossensteyn et al. (2015) as having a comprehensive, coherent, and effective national approach to improving study success; furthermore, considerable research has been undertaken into the causes of student withdrawal and effective strategies to improve success for all students. Study success is high on the HE policy agenda across the UK, and in England in particular, and has been for approximately the past 15 years. This concern about study success has developed in response to two other policy directions: first overt efforts to increase the diversity of the HE student population, which have evolved into consideration of not just who enters HE, but equity in student outcomes and returns as well. Second, the introduction of tuition fees raised fears about both access to HE, and outcomes, for students from lower socioeconomic groups. Steps were therefore implemented at the national level to ensure that tuition fees and greater reliance on student loans (rather than non-repayable grants) did not have an unintentionally negative impact on study success in general and for students from lower socioeconomic groups in particular.

The English continuation rate (which refers to progression from first year of study to second year for full-time students) is high, currently around 94%, and only around 10% or 11% leave without an award or transfer to an alternative program of study at another institution (HESA, 2018). The rates have remained stable over the past decade or more, although the system has expanded in sized, increased in diversity, and introduced higher tuition fees. It should be noted however that there are significant variations between institutions; and between student groups. Therefore, broadly the English system can be judged to be effective with regard to study success.

In England, there is widespread agreement about what study success is; it is almost universally understood as the completion of a degree in a prescribed time period (with up to 1 year variance from the standard time allowed, i.e., 3 or 4 years for full-time degree programs). The completion rate is defined as “the proportion of starters in a year who continue their studies until they obtain their qualification, with no more than one consecutive year out of higher education,” this complemented the continuation rate, which is a more immediate measure, calculating the proportion of a higher education providers intake which is enrolled in the year following entry. (National Audit Office, 2007, p. 5). These definitions are not contested, although national bodies and institutions recognize and aspire to additional elements of study success. For example, the Higher Education Funding Council for England (HEFCE, 2013) has encouraged institutions to consider not just continuation and completion, but also attainment and progression into employment and further study (and work to improve these outcomes for all students is currently being taken forward by the newly created Office for Students). Institutions and other stakeholders in the sector recognize the value of an extended notion of success, taking account of issues such as personal goals and aspirations, and distance traveled.

In England, there is pressure to maintain and improve study success, especially in parts of the sector where it is lower than the national average, or in relation to students from less advantaged groups. A number of national policies have been employed to improve continuation and completion across the sector; these include different types of policies, which are largely aligned to improve study success. In particular, institutions’ funding is tied to the number of students enrolled and retained for the duration of the course, historically through the teaching grant paid to institutions, and more recently through student fees, which are directly related to the number of students studying. This provides a compelling financial impetus for HE providers to strive to improve study success, especially as the performance of each institution is measured and compared to its “expected” rates of continuation and completion and this information is widely published and utilized. The Higher Education Statistics Agency provides a “benchmark” for every institution, which is calculated by taking into account subject portfolio, entry qualifications, and student diversity. This benchmark figure is published alongside an institution’s actual performance with regard to the total institutional student population and in relation to subsections of the student population. National newspapers use this data to produce league tables about retention, and the information is fed into wider league tables about the “quality” of individual HE providers in England and the UK and which inform student-facing publicity such as the Good University Guide and similar, designed to inform students’ decision-making about entering HE. In addition, since 2003, additional funding has been paid to institutions to improve the continuation and completion of students from groups assessed to be at greater risk of withdrawal.

A further policy tool – the Teaching Excellence Framework (TEF) – was introduced in 2016 (BIS [Department of Business, Innovation and Skills], 2016). Its intention is to assess the quality of teaching and learning in each HE provider and to further inform students’ decision-making about where to study. It utilizes six metrics per institution, three of which are based on results from the National Students’ Survey, plus continuation data and two indicators of employment outcomes. The metrics are broken down in relation to student characteristics, identifying when particular target groups are scoring better, worse, or the same as the majority. Each institution (including private providers) has to meet baseline requirements to enter into the TEF and then receives an award of gold, silver, or bronze, based on an assessment of the metrics and the institution’s written submission by peers from across the HE sector. Therefore, the TEF provides an additional incentive for HE providers to maximize the continuation and completion of all student groups, as the data feed into publicly available assessment, which may be used to inform students’ decision-making regarding institutional choice. Further comparable information about the student experience is provided to students’ pre-entry through Key Information Sets, which present course-level data about every program in the UK through an online platform.

While the primary policy tools employed in England are institutional funding, and the provision of information to assist students with their decision-making, there is recognition that learning and teaching are integral to study success, and indeed much of the institutional funding is used to improve the learning, teaching and assessment experience to improve student engagement, belonging, retention and success (Thomas, 2012). This commitment to learning and teaching is reinforced by national initiatives: the government has contributed to the funding of a number of national organizations to improve learning and teaching: the Institute for Learning and Teaching in Higher Education (2000), Learning and Teaching Support Network (2000), Higher Education Academy (2004), Leadership Foundation (2004), and Advance HE (2018). These organizations, in various ways, have sought to develop and champion high-quality learning and teaching in HE, including its contribution to study success, through staff training and development, recognition and accreditation, and pedagogical research (Brooks, Baird, & Shenstone, 2014).

This drive for institutional change is reinforced by the requirement for institutions charging tuition fees over approximately £6,000 (this varies depending on their TEF rating) to submit an Access Agreement, or from 2018, an Access and Participation Plan, for approval from a government body (previously the Office for Fair Access and now the Office for Students). Approval is essential for HE providers to operate in England and charge fees above the threshold level. The Access Agreement, or Access and Participation Plan, specifies how a proportion of additional fee income is to be spent to ensure the access and success of disadvantaged student groups. This must include outreach work, financial aid, and spending to improve retention and success (including progression beyond HE). Research found that the process of developing and implementing an Access Agreement had a positive impact on institutional policies, planning, and behavior (Bowes, Thomas, Peck, Moreton, & Birkin, 2013), with most HEIs achieving or exceeding their targets. The process of producing and implementing an Access Agreement has impact on both institutions approach to increasing diversity and improving student success, and to the outcomes for students. The introduction of Access and Participation Plans in 2018 (with respect to students entering HE in 2019–2020) is intended to be more challenging for institutions, requiring them to focus on issues that they find the most challenging.

The combination of policies in England address study success from different angles, but they are largely reinforcing rather than in tension with each other. It might be anticipated that the introduction of high student fees (approximately £9,000 per year) would have a negative effect on study success, but rather it is stimulating and financing HEIs to care about and improve study success. However, it should be acknowledged that England has a fairly tight admissions system (institutional autonomy has been retained and is not regulated), which contributes to higher rates of study success. Furthermore, there is a widespread and embedded expectation that completion is possible within 3 years except for exceptional in circumstances. Institutions and students are not funded for more than 3 plus 1 years (except for longer courses), and students and their families do not expect to study for longer than the normal time period. This provides a good basis for retention and completion. National policy, guidance, and funding have been directed to maintaining and improving the retention of students in the context of expansion and increased diversity, and improving employability, and more recently the attainment outcomes of students.