Abstract
As the role that publicly owned and private think tanks play in evidence-based governance grows, several parameters that have a bearing on the agenda, schedule and timetable of the think tanks have already been studied. Here we look at the interplay between the funding model (public vs. private) and the analysis intention (the temporal outlook; ex-ante vs. ex-post approach), choosing Lithuanian education policy, system and agents as the study domain. Among other findings, we see that the funding model correlates with the temporal outlook: the investigated representative of private think tanks tends to engage in ex-ante work more intensively, while the publicly funded equivalent is exhibitive of both ex-ante (preparatory) and ex-post (evaluative) research tactic in almost equal measure. We conclude that the private sector may afford the luxury of being more focused and efficient in its information behavior mostly due to the clear and non-debatable ideological commitment. In a deliberative democracy this is rarely possible. It would be of great interest to check whether the same generalizations hold for jurisdictions that are free from the post-Soviet administrative tradition.
Introduction
The role of expertise has recently been gaining in importance both in the international and in the national policy making processes. For example, the European Commission in its Better Regulation and Smart Regulation agendas seeks to strengthen the role that policy appraisals, evaluations and estimations – data, in general – play in its deliberations (see Smismans, 2015). Van Aeken (2011: 45), more generally, highlights the very unavoidable need for systematic, science-based, rational evaluation. Policy makers are, indeed, elected, but in the age of practically unlimited data, the policy makers are to be guided, at least in addition to other considerations, by scientific reasoning. The number of institutionalized expert groups is growing. The variance in the different formats these institutes may take (“analysis center”, “think tank”, “advisory group” etc.) and in the competencies of the human resources they employ is likewise gaining in scope. So is the amount of produced analytical information, and the publicity this analytical information naturally or deliberately receives.
The graph (“timeline”) representing the “initiatives” in the public education system (the upper timeline), and the STRATA output (the lower timeline). The preparatory (red lines) and evaluative (the black lines) relationships between the analytical/research output and the “initiatives” are indicated
Citation: Hungarian Educational Research Journal 13, 2; 10.1556/063.2022.00150
The graph (“timeline”) representing the “initiatives” in the sample private education system (the upper timeline), and the relevant LFMI research output (the lower timeline). The preparatory (red lines) and evaluative (the black lines) relationships between the analytical/research output and the “initiatives” are indicated
Citation: Hungarian Educational Research Journal 13, 2; 10.1556/063.2022.00150
The role that these public or private institutions play has become the focus of much recently published research. First and foremost, it is the use of the analytical information that calls for most scrutiny. Additionally, an interesting aspect to be investigated is the information behavior of the expert groups themselves. The different funding models, organizational structure, the different methodologies and data gathering and processing approaches that these institutions adopt are of relevance. In particular, there are questions of focus and data control, and of timing control – the different agendas and timetables – that should be better understood for there is always a possibility that they may define the schedule of the political process in some particular jurisdiction. There may be several parameters that define the research timetable of an expert group: in addition to the institutional governance model, they are the frequencies of topics exploited in the public discourse and peaks of information in the public debate. These and other parameters – and the model itself that they constitute – are targeted in various investigations. This paper focuses on two of them: the (1) funding model (organizational structure; public vs. private) and the (2) analysis intention (ex-ante vs. ex-post approach).
Several aspects are already well known. First, as is the case in Smismans (2015), the research focuses on the link between ex-ante (prospective, preparatory, before the enactment of a policy) and ex-post (retrospective, estimative, after the enactment of a policy) evaluations: as an ex-ante prediction can later be appraised against the factual outcomes as were estimated in an ex-post analysis, and as the ex-post analysis could suggest improvements for the methodology of ex-ante appraisal, these two may, indeed, be argued to make up a “cycle”. Golen and Voorst (2016) find that out of timeliness, quality and focus it is timeliness that contributes the most to the effectiveness of the “cycle”.
Secondly, several researchers highlight that the “data”, or at least the use of the presented data, is not free from a political dimension. Mastenbroek, van Voorst, and Meuwese (2016: 1330) point out that an effective ex-post feedback is not necessarily desirable for it may lead to the “uncover[ing of] critical problems in the actual working of legislation, [it] may lead to calls for legislative repeal”. While Zwaan, van Voorst, and Mastenbroek (2016: 15) show that the ex-post appraisal does play a certain role, for example, it features in the questions of the members of the European Parliament to the Commission, in a later study Van Voorst and Zwaan (2019: 382) point out that, indeed, the resorting to ex-post evaluations may be politically motivated.
This then contradicts the ideal that the mechanism is intended to enhance the role played by science-based information, informing the legislating process in an ‘objective’ way. Olmedo (2013: 59) openly charges the think tanks operating in Spain in the domain of education policy as being engaged in mere neoliberal agenda setting and attempts at accusing the state of thwarting the actions and potentials of individuals. Lubienski, Brewer, and La Londe (2016: 69) are equally critical of the American education think tanks: they are claimed to be engaged in the promotion of a certain type of information, not in the production of information sui generis.
Similar findings are prevalent in research focusing on the Balto-Scandian realities. Kelstrup (2020: 130) highlights that the think tanks who provide the evaluations (at least in the Nordic countries) may be interpreted in two different ways: indeed, as providing rational information to the benefit of the public debate and, directly, to the benefit of policy makers, or else they may be judged as “lobbying vehicles” whose production is ultimately biased. Allern and Pollack (2020: 145) consider ‘most’ such organizations to be advocacy organizations.
Bublienė, Limantė, Ragauskas, and Vitkutė (2017: 96), focusing on state-funded analytical work in Lithuania (the Baltics), conclude that there are no general regulations that norm the timetable, the principles of the choosing of focus and topic of the ex-post analytical work, subsequently, there is no coherent planning; in contrast to the private think tanks, at least the publicly funded research is openly agenda-free and open to manipulation (see also Nordrum, 2013). Limantė (2017: 39), concurs that similar concerns (the lack of regulations that norm the evaluation process) were raised by the Organisation for Economic Co-operation and Development. It does make sense to look for a positive impact the expertise has on the quality of governance only if the expertise is itself impartial/objective, reliable, methodologically sound and consistent.
The rationale of this investigation arises out of these concerns with the formation of the timetables, the thematic schedules, at the Lithuanian think tanks, both public and private. A short survey of the strategic documents of several analysis centers operating in Lithuania1 reveals that, as far as the formal regulations go, there are several different approaches to the construction of the timetable of analytical work as employed by such institutions. Three main strategies could be identified:
- –(1) a weak and unfocused timetable arises out of an attempt to encompass as many as possible thematic fields, all of them treated as largely equally dense, equally important and equally specific (“formalism”);
- –(2) a strong and focused timetable arises if the particular center relies for the construction of its timetable on the monitoring of the public discourse, the events in the field dictating the topics (“bottom-up”);
- –(3) a strong and focused timetable arises if it is decided by the orders placed by information consumers (funding providers, management, politicians; “top-down”).
In this paper we focus on the timetable of analytic/expert work (in the field of education policy) as was the case for two centers of political analysis in 2017–2020 in Lithuania, a private one and a state-funded one, particularly on the relationship between (1) their factual research schedule and (2) the general activities of those interested in the outcomes of that particular work (usually, the order-placers, the customers, the information consumers). We focus on the differences in the expert information use cycle: how is the use (commissioning, resorting to) of the produced analytical work different depending on whether the customer (producer) is a public or a private institution? As was just indicated, here we target two parameters: organizational model (private vs. public) and analysis intention (temporal outlook; ex-ante vs. ex-post). Other authors (e.g., Brazinskaitė & Kėvišas, in press) have claimed that state-funded expert information, even if expensive and elaborately produced, features very little in the legislative and executive work of the institutions who place the orders for that particular information to be made available. Here we investigate whether the same is the case for what we argue is an equivalent in the private sector. In addition, as the research we report from here targets more the timetable and the chronological relation between the order and the use, we will attempt to delineate the effect the agenda of the customer has on the thematic schedule of the analytical center, and vice versa.
Methods
This paper, in terms of scope, focuses on the Lithuanian education sector because it is the field where the think tank activities, whether the think tanks are publicly or privately funded, are advanced the most and are of long standing. Indeed, the Research and Higher Education Monitoring and Analysis Centre, MOSTA, is the oldest Lithuanian state supported evidence-based governance institution. It was founded in 2007. The current Government Strategic Analysis Center, STRATA, an expert institution that provides the government and authorities with independent, research-based information required to make evidence-based public policy decisions, was established in 2019 following the most recent reorganization of MOSTA. STRATA still conducts studies – in addition to focusing on other sectors – in the fields of education, science, innovation, human capital and the labour market. STRATA is the first object of this study, together with the state education system treated as a case representative of the public sector.
For the purposes of comparison, we chose the Lithuanian Free Market Institute, LFMI. LFMI, a private sector think tank, is an independent, non-profit centre of excellence. Since its establishment in 1990, LFMI, one of the oldest privately-owned public policy research organizations, has been a firm advocate of free-market economics and was instrumental in reorienting the structure of the Lithuanian economy away from the statist Soviet model. LFMI houses a private “education system”, LFMI Education Centre.
These two institutions could be argued to be comparable within the framework of this investigation in the sense that they both contain (1) a body of personnel dedicated to analysis (STRATA vs. the LFMI analysts) as opposed to (2) a body that is dedicated to the implementation of the activities, the “running”, execution of actual policy initiatives. STRATA's analytical output ultimately does (at least ideally should) play a significant role in the regulation of the state school and university system. LFMI, in turn, since 2012, has been engaged in practical education activities via its own LFMI Education Centre oriented at teachers, students, pupils etc. LFMI Education Centre functions as a teacher training institution, textbook publisher, examination authority and educational programmes authority. Its commitment to the efforts targeting economic and financial literacy in the country are especially well known. Educational activities are an integral part of LFMI's work. – The two, however, differ in scale.
The two think tanks – the private one and the publicly funded one – are comparable only if they are comparable in their functioning and in their structure. These two, indeed, are. We are interested in the activities of the private think tank only inasmuch as it prepares or evaluates the activities of the private schooling system with which the think tank is associated. In this, the private think tank and the public think tank do the same thing (function), and they are positioned similarly vis-à-vis their environment, they are structured comparably (structure). In other words, we draw a parallel this way:
- –a private think tank vs. (its associated) private education system;
- –a publicly funded think tank vs. (its associated) public education system.
A comparison constructed this way allows one to detect differences that are due to the type, not due to the aims, goals and objectives.
This study covers the period of 2017–2020. We would contend that such a period, albeit short, is sufficient to assess the performance of the two organizations in question, and to evaluate the links between their analytical output and the associated initiatives implemented. In addition, it is important to note that the investigation period coincides (with a slight discrepancy) with the term of office of the 17th government of the Republic of Lithuania. This gives us an outlook on the key decisions executed by this government in the field of education (initiatives), and also lets us establish correlations between these particular decisions and STRATA's publications (analytical production).
In this paper, we perceive an “analysis” (“analytical production”) to be a publication that follows an explicitly stated data collection, processing and analysis methodology (or else such methodology follows from the functions of a particular institution, and is explicitly delineated elsewhere in the corpus of normative documents of that institute; the publication then (explicitly, implicitly) refers to the methodology employed). Further on, we comprehend as an “initiative” any deliberate action that is carried out within a clearly defined education system (the state education system, the LFMI Education Centre): a change in the regulations (access to services, remuneration to the service providers etc.), resources made available (textbooks, teaching platforms etc.), restructuring or an introduction of new functions (a new institutional body formed etc.), measures that address the competencies of service providers in the system etc. We treat all “initiatives” equally without ranking them.
We answered our research questions by amassing two self-constructed datasets. The first dataset contained 33 ex-post and ex-ante evaluations of (and the associated sets of recommendations for) regulations that pertained to initiatives in the education sector as defined above. Out of these, 26 made up the output of STRATA, and 7 made up the output of LFMI. For the criteria for inclusion of a particular document in the dataset we relied on its factual contents/format (the document had to be data-based, and be published (online; be publicly accessible) as a report, similar to a research paper in outline); we included analytical works with very different kinds of names (the most common ones being ‘report’ ‘study’, ‘overview’, etc.).
For the construction of the second dataset, which contained 39 initiatives as defined above (out of these, 18 pertained to the state education system, 21 pertained to the LFMI Education Centre), we (1) analyzed the public discourse during the research period (6975 summaries of press articles accumulated in the database2 had to be reviewed; the criterion for the inclusion of a particular item in the dataset was the keyword “education”, also, the item had to reflect events that took place during the research period; we also surveyed the press releases of LFMI), and (2) identified a number of major controversies in the Lithuanian media that were expected to be associated with education-related government initiatives or publicized LFMI activities. We identified as relevant (for our purposes) events within the domain of education only those that answered the following two conditions: (1) they were mentioned in the press at least ten times, and (2) they were related to a specific piece (or pieces) of legislation that went into effect during the investigation period; all of the LFMI activities that were publicized were taken into consideration as representative of LFMI Education Centre initiatives.
We dated each point/node in our dataset by resorting to the date of publication/release for analytical works, and by resorting to the (approximated to a calendar month) date of the change (or its associated enactment of legislation) for initiatives. In case of uncertainty (e.g., a certain analytical publication consisting of a series of releases that were protracted in time, when the middle time point was selected), we relied on the best available evidence (sometimes recovering the release time point from file metadata).
Having amassed the two datasets, we constructed four calendars. For the first two calendars, we graphically displayed the chronology of all education-related STRATA and LFMI publications (analytical productions), producing two parallel trajectories of analytical work. For the second two of calendars, we displayed the chronologies of initiatives as were the case for the state education system (government and parliamentary initiatives and the associated public discourse controversies) and as were the case for the LFMI Education Centre activities (projects implemented, courses arranged, textbooks released, exams organized etc.). Paired calendars (analytical works vs. initiatives) gave us two timelines.
We compared the events on the two matching timelines (STRATA analytical work <–> public sector initiatives; LFMI analytical work <–> the activities of the LFMI Education Centre) by linking the analytical work/publications and the initiatives (legislation) judging the contents of the nodes on the two sides of the timeline.
Results
Overall, the two resulting timelines have both features in common and pronounced differences. A clear common characteristic is
- –the relative comparability of timetable intensity.
Otherwise the operations in the private and the public sector differ in:
- –the temporal properties of the commission–use cycle;
- –the “productivity” (continued re-use) of any one single analytical work;
- –the tendency (or the lack thereof) to produce analytical works that are prospective (preparatory) as opposed to retrospective (estimative) or vice versa.
We will discuss these findings separately (see Figs 1 and 2).
The relative uniformity of intensity
The timetable of the parliament/government could be expected to be exhibitive of certain troughs and peaks. A common feature of the Lithuanian public discourse are comments that the beginning of a term is often associated with a slump in legislative productivity, whereas the end of the term is typically rather intensive. The slump is supposed to be due to the fact that the Lithuanian general elections take place in the fall, which means that the new parliamentary majority /government is not free to alter the budget or to fundamentally review other enacted legislation – due to the lack of time before the nominal beginning of the new fiscal year. Likewise, the end of the term is often hypothesized to call otherwise a rather inactive parliament for action, because the approaching general election intensifies the public debate in which the different parties showcase their attentiveness to the needs of various interest groups.
Our research does not indicate that such deliberate troughs or peaks are, indeed, the case. We did not detect any pronounced concentration of legislative or public discourse activity towards the end of the term, nor was there a slump in the beginning.
An episode of enhanced legislative intensity could only be observed in the second half-year of 2018 when the Seimas enacted laws and regulations pertaining to the full-day school (7–8 hours a day), voucher-per-class system, shift-based remuneration system, vouchers for non-formal education and the “Healthy Diet” programme. However, it couldn't be said that this was a deliberate attempt to “pull back” the legislative initiatives as far away as possible from the past and future general elections, for the legislation enacted was organically inextricably linked, the separate decisions could not have been made independent from one another. This particular cluster of intensified activity was the key point in the major reform of the field of education as promised by the coalition parties.
This relative uniformity in intensity is to be detected both in the activities of the parliament and the associated public debate, as well as in the analytical output of STRATA. Quite a number of new releases could be observed in the second half-year of 2020, but this is not of relevance in terms of the implications of this research: as the scope of responsibilities that STRATA is tasked with increases, and as their number of employees grows, the numerical estimation of the accomplished work increases. The timetable could be taken as additional evidence for the features of the state-funded information production that are reported from elsewhere (Brazinskaitė & Kėvišas, in press), namely, that quite a number of releases stand in no relation whatsoever with any legislative initiatives of public discourse topicalities (“waste”; out of 27, only 11 could be linked to legislative initiatives; more on that in the Discussion section).
In this regard, there is no pronounced difference between the public sphere timetable and its equivalent in the private sector either. In both cases, the schedules are characteristically uniform. In the private sector, neither the analytical production, nor the educational activities are exhibitive of clear periods of enhanced intensiveness. Quite the opposite: certain activities follow a consistent rhythm. The nation-wide economics exams (in March), the national conferences on civics course contents and pedagogy (in September-October) take place regularly, usually once a year. Other releases or events follow one another with roughly comparable lapses of time in between them. There are no explicit peaks of activity to be detected.
However, as we will see, this relative uniformity in intensity is the only regard in which the public sector scheduling is comparable to the practice as is the case for the private sector. When considering other aspects, clear differences become apparent.
The time-related characteristics of the commission–use cycle
In most of those 11 cases when some analytical publication released by STRATA could be linked to certain legislative initiatives, there is a pronounced temporal distance between the time point of the release and the associated discussion/enactment of the normative legal act in the Lithuanian parliament.
For example, as the problems of the national teacher workforce (the staff shortages) gained in notoriety, in November 2018 STRATA released an analysis of future demand for these particular skills in the labor market (STRATA indicated that the research focuses on the university admissions processes). However, the earliest government action in this domain (the introduction of scholarships for pedagogy students, a way to attract new young talent to the education field) was announced in June 2020 (an ex-ante case).
Another typical example could be the case of the attempt to reform the teacher remuneration system. In February 2020 STRATA made public a report focusing on the salary structure and dynamics of the public sector school employees. However, the actual changes in the payroll system were enacted by the parliament in November 2018. There is a clear 1 year gap (an ex-post case; see below).
Counterexamples are very few. In November 2018 STRATA released a report as to the results of the previously enacted reform package (see above). As the measures were highly controversial and led to somewhat unforeseen outcomes, the government attempted to adjust the regulations. The adjustments were debated and enacted in November 2018, too. Here the analytical output of STRATA was immediately made use of.
This particular trait, the temporal distance between the release of preparatory or evaluative report and its associated activities, is clearly out of character when compared to private sector operations. There is no pronounced detachment between the analytic/research output and the activities that stand in a relationship with the particular releases. An immediate application (if the production is an ex-ante one), evidence-based evaluations that follow the actions without delay (if the production is an ex-post one) are more typical of the private sector information behavior. For a typical example, the guidelines for civics courses (released in Fabruary 2017) were immediately followed by series of seminars for practicing teachers (the earliest one in March 2017).
This difference is not the only one.
The “productivity” (continued re-use) of a single analytical work
A striking feature of the public sector analytical works in regard to their subsequent use in the legislative process is their comparatively “low” productivity, or relatively limited “potential”. In other words, a single individual analysis or report is never taken as a point of departure for any legislative initiative or action in the system. A single investigation is never enough to launch, to precipitate – unilaterally and singlehandedly – any kind of change. In the public sector, inasmuch as our research licenses such a generalization, every observed initiative or action is usually informed by a series of conclusions and (repeated) recommendations that, equally importantly, are somewhat dispersed in time.
Two examples are typical. In June 2020 the Seimas enacted several amendments to the Law on Higher Education and Research and to the Lithuanian Vocational Training Act. To summarize, the government was tasked with constructing a scholarship allocation mechanism that takes into account the demand for specific skills in the current and future labor market, and then directs funds more or less automatically. The planning is to be consistent and to follow a system of guidelines: the choice a student makes when selecting the desired study programme should not be the only guiding principle, quite the contrary – the currently amassed data, future labor demand projections, expert recommendations and similar sources of input are also to play a certain role in allocating state funds for student scholarships. The so-called “target-oriented”, “purposive” scholarships should also stand in a relationship with the academic fields that the state judges to be a national “priority”. These “purposive” scholarships are to be offered even to self-funding students and those who are involved in the state underprivileged student aid scheme. – Chronologically, there are four STRATA reports that ground and prepare this “technocratic” reform. In March 2018 STRATA released the first paper on the structure and dynamics of the national labor force, paying a special attention to the tension between university programmes and vocational training programmes, and the differences in their attractiveness. Similar analyses were released in February 2019 and June 2019 (an ex-ante case).
In March 2018 Seimas, upon the proposal sent in by the President Grybauskaite, enacted certain changes in the state vocational training system. The “archaic and stagnant” institutions that provide such training ceased to be, legally, “state entities”, and became but “legal persons” who are created by the state, but who exist relatively independent from it (Lith. “viešoji įstaiga”, “autonomous public entity”; this should make schools more independent and encourage businesses to take part in the educational process – business entities were allowed to become co-owners of schools). The new regulations defined when and how many times one is allowed to pursue state-funded vocational training, they also promoted dual training and apprenticeships, introduced a more flexible system for the recognition of additional qualifications. They also stated that an external audit of the school is to be carried out every 5 years (a more elaborate quality control system). The outcomes of this particular reform were taken for the object in several subsequent STRATA investigations (immediately, in March 2018, then in December 2018, March 2020 and so on; an ex-post case).
This feature has an additional consequence in terms of STRATA planning for its activities. The fact that any actual legislative change has to be “well-informed” means that the investigations and research that STRATA engages in, make up complex series, cascades, continued projects, repeated analyses – all this being extended in time. This produces something of a consistency, even repetitiveness, in the research output that allows for an accumulation of evidence that eventually enables the evidence-based governance.
The reusability, the tendency to continuously exploit, repetitively and repeatedly, the same single information release, is, however, clear in the case of information behavior in the private sector. One information product can be – and is – applied, used, resorted to several times (this then means that the releases also tend to be preparatory, “programming”, ex-ante releases; see below). The cycle of information use is thus peculiar: due to the repeated re-use, “productivity” of information, its repeated exploitation, duplicated, multiplied application, the information consumption processes are extended in time. This characteristic reflects an inclination towards (economic) efficiency: not only is it the case that no information gets wasted, but the evidence-based information that is being created is sure to be of maximum productivity. The civics/ethics education programme “A Citizen in 31 Hours” may be taken here as an apt example: the release was taken for the basis not only for a published textbook (an internet-based platform), which later was made publicly available free of charge; it was also taken as a point of departure for a regular TV broadcast for children (a series) and a specialist conference for practicing teachers.
The tendency as to prospectiveness/retrospectiveness
In addition to the characteristics showcased above, the state information behavior – in regard to the evidence-based information input – is marked by a pronounced lack a dominant linear tendency: both ex-ante and ex-post research is commissioned and produced in almost equal measure. There is no tendency – neither a clear proclivity to preparatoriness or estimativeness – in the general strategy of the information circulation cycle. Both tactics are employed with equal intensity.
The previous examples were exhibitive of both of the strategies. A more detailed exposition follows.
The role STRATA played in the restructuring of the geographic distribution of state higher education institutions in the country (the “geography of institutions”) is an example of its ex-ante input. In 2017 (January-March) STRATA released a series of reports, one of which, for example, evaluated state universities of applied sciences (“polytechnics”), estimating their student numbers and the quality of the programmes of study on offer. In another paper from the same series STRATA focused on the demographic situation in the country as a whole and in the separate provinces within the country, on the supply-demand dynamics in the (provincial) labor market. The works relied heavily on amassed statistical information.
In its own annual activities overview for the fiscal year 2017 STRATA indicated that these reports inform discussions at the Lihuanian Ministry of Education and Science, where a possible reform of the geography of state tertiary education institutions is being considered.
The outline for legislation pertaining to the geographic distribution of state tertiary education institutions was promulgated in November 2017. The reorganizing itself was quick to follow: in 2018 the procedure of making Siauliai University a part of Vilnius University was initiated (completed in January 2021), in January 2019 the Lithuanian University of Educational Sciences and the Aleksandras Stulginskis University (in Kaunas, formerly the Lithuanian Academy of Agriculture) were incorporated into the Vytautas Magnus University in Kaunas.
The information STRATA provided on the economic situation within the separate provinces, labor market data, skills shortages data, demographic realities and dynamics, academic (research) capacity in the provinces, also, the conclusions it drew and the packages of recommendations that it submitted, played a significant part in delineating the conceptual framework of the future reform. STRATA reviewed the situation in a select number of Lithuanian districts regarding demographic tendencies, economic environment and the capacity to provide tertiary education as well as research.
Some STRATA releases were also relied upon by other branches of the government. For example, the Centre For Quality Assessment In Higher Education, later, when commenting on the functioning of the state higher education sector, relied upon STRATA's analysis of the (minimal criteria in) quality control systems in higher education.
Overall, the trajectory outlined above is a typical representative of ex-ante outlook: the reform enacted – especially in that it meant the closing down of a certain number of provincial tertiary education institutions, others being scaled-down – was highly controversial, it had many far-reaching outcomes. It is only natural that it had to be well-prepared, reasoned and discussed. It is equally natural that it took for its premise a wide array of prospective evaluations of the current situation and the expected dynamics – that were all done and published well in advance. These evaluations, employing various research methodologies, tried to project the likely outcomes.
The role STRATA played in reforming the teacher remuneration system could be returned to as a case of ex-post evaluation.
Shift-based work compensation scheme in the state education system was launched in June 2018. The regulation was very contentious, and the legislation had to be amended. In December 2018 President Grybauskaite tabled a set of urgent amendments that were supposed to remedy the greatest inadequacies. In essence, the proposal amounted to a new definition of work load for those employed by the public schools: a new set of principles that define the salary structure and pay calculation, measures that simplify the application of those principles, several measures that minimize the gap between schools of various sizes – those operating in small towns and large cities. The President highlighted the deficiencies and shortages of the newly enacted remuneration system, no longer speculated about, but observed and estimated.
The President clearly stated that her proposals are informed by the data provided and conclusions drawn by STRATA (then – MOSTA), especially on STRATA's estimation of the changes in actual net pay that teachers earn per month, contrasting the averages before and after the enactment of the reform.
The report referred to was a STRATA paper published in November 2018. (The intensity of the timetable (June 2018 – the law enacted, September 2018 – the system implemented, November 2018 – STRATA report, December 2018 – amendment) is out of character in the entirety of state information services history – in terms of the scope of this paper.)
In February 2020, continuing the monitoring of the outcomes of the reform, STRATA released a follow-up study targeting the teacher remuneration system. New data – as well as conclusions and recommendations – on the actual pay in the state secondary education system, non-formal education and vocational training institutions was collected, and both the status quo and the dynamics evaluated.
Overall, the preceding narrative showcases the ex-post outlook.
If in the public sector both tactics – both the ex-ante and the ex-post approach – are employed with equal intensity, in the private sector there is a pronounced tendency to mostly engage in the preparatory, perspective ex-ante mode of investigating. The data is being collected, the information processed, the conclusions and sets of recommendations, guidelines formulated and proposed only if a certain (desirable) set of future actions are being projected and programmed. It is atypical of the activities of the LFMI – at least within the confines of this research – to be tendentiously indicative of an ex-post, evaluative, retrospective outlook. This stands in a relationship with (literally, enables) the productivity of a single report, set of guidelines or information package of another kind (see above).
Discussion
This contrastive study has revealed three differences in customer information behavior between a publicly funded trajectory/“cycle” of analytical information commissioning and consumption, and the one that was determined to be the case for the private sector:
- –privately funded analytical work tends to be more prospective rather than retrospective;
- –privately funded cycle is of shorter duration, not as extended in time;
- –privately funded (prospective, “preparatory”) work is more productive, meaning here one release can be used to give rise to several initiatives; whereas in the public sector a single legislative or executive action usually takes a variety of analytical reports as the input preparing it.
- (1)These findings, for the most part (though see below), complement other conclusions usually made in evidence-based policy research. To begin with, they answer the claims made in Mastenbroek et al. (2016: 1330; highlighted in the introduction): an ex-post analysis may be particularly productive as a point of departure for political argument construction, precipitating a call to repeal a certain enacted norm; it is characteristic of political opposition. Therefore, it is only natural that commissioned ex-post type of analysis features prominently in the public sector (where it is routinely ordered by politicians), but not in the private sector where the activities are more focused, streamlined and not up for debate. Lindgard (2016: 18–19), indeed, defines think tanks as, say, hybrid organizations (boundary spanning organizations) engaging in many different formats of activities at the same time (publicizing of ideas, data-gathering, data-processing; educational activities etc.) that are nevertheless all of them goal-oriented: the various activities are subordinated to some pre-set general idea, which is not to be challenged. Bergström and Goldman in their guidelines for evaluating think tanks (2018: 18, 24) openly state that a think tank, here understood as a policy vehicle, is to set its intended goals (“overarching vision and/or mission”) before starting with any analytical or practical work, though these goals are to trickle down to every measure implemented when seeking “societal impact”.
The “efficiency” characteristic of the private sector is not only there, it can – only – be there for in the case of the private think tanks there is no uncertainty as to the ultimate goals, the ideological orientation, unlike in politics, is clear from the outset. The predefined ideological outlook critically limits, delineates the focus of attention and the spectrum of activities. Such a fixed ideological axis is visible in the outlook of LFMI, but not in the outlook of state institutions: LFMI focuses on training in economic behavior that is of clear liberal (even libertarian) persuasion, LFMI focuses on the drilling in a particular ethical outlook, it openly seeks to construct a particular ethical outlook in the pupils, it attempts to “indoctrinate” in a particular methodological paradigm in the social sciences.
- (2)Another expected finding is the idea that any kind of analytical, data-based, production is never pursued as an end in itself, but that it is invariably always oriented at a set of desired pragmatic consequences. We see that both the state-funded and privately-funded research in (ex-ante predicted, ex-post estimated) policy outcomes is pursued only inasmuch as it correlates with initiatives on the plane of executable actions. In the work of other researchers, it has been highlighted that such pragmatic orientation (desire for “societal impact”, being primarily motivated by societal realities) is not only visible in the operations of policy centers – be they privately or publicly funded, – but has recently become aggressively prevalent in scientific institutes as well (“academic steering”, Benneworth & Peñuela, 2019: 3). The intent to close the research-practice gap is thus very contextual.
Indeed, there is a certain amount of “waste” visible in the publicly-funded operations, meaning here reports generated that do not relate in any way to the actually introduced bills, debated proposals and the like, but this does not warrant a claim that this is, indeed, a wasted production. It is completely plausible that at the time when a particular report was being commissioned, it did have a legitimate legislative intent and context – that just failed to materialize later on. Lundin and Öberg (see Lundin & Öberg, 2014), mindful of the risks of the “waste” that is typical of the information behavior in a democracy, even suggest some countermeasures, first and foremost, publicity, an accompanying public controversy or a debate over a proposed bill. The amount of public attention usually increases the role played by expert information. (In other words, state administrators tend to “use expert information more when they operate in a context in which there are large political disagreements”.)
- (3)However, in a very clear sense, the findings we present here contradict, somewhat, or make more precise, the expectations delineated in some other recent papers, specifically those that deal with inter-institutional boundaries and “locations” of competencies. It has been claimed that research indicates a pronounced blurring in the distinction between the partisan “political” information (supplied by competence centers on the “outside”) and the administratively “technical” information (supplied by competence centers on the “inside”, government centers, state-funded centers) (see Craft & Howlett, 2012, 2013). The classical “separation” between the “state” (the “crown”, and its internal information processes, information behavior of its agents) and various “interest groups” that engage in attempts to steer the state in one or another direction is losing in its being identifiable. Our own research was methodologically limited, but, at least in terms of the information consumer behavior, we still see a difference between the state and interest groups. In their behavior, private actors are focused and streamline their activities so as to avoid all types of inefficiency. The information processes in a democratic state, internally, within the state narrowly defined, are characterized by the exact opposite: a slow, repetitive, effort-intensive information cycle where the information input and some particular output in terms of actions that rely on that information stand relatively apart. We do not contend that there is a difference in the information itself (the “contents”; in the data methodology, in the data itself, in the themes, focus etc.), but we contend that, still, there is a sharp distinction in information consumer behavior on the “inside” as opposed to the “outside”. Indeed, Kelstrup and Blach-Ørsten (2020: 173, 175) find, too, that there is a difference in the information behavior in that the production of publicly funded think tanks (“state technical information”) features more prominently in parliamentary activities for such information producers are “expected to be more heavily influenced by institutionalized norms”. Quite to the contrary, privately funded information producers/providers play a more visible role in the media because they naturally position themselves as institutes of “public advocacy” rather than “direct influence”.
However, this may only tend to be typical, as perhaps is the case here, of some particular democracy, here – of Lithuania. Indeed, Kelstrup (2017: 135) finds that the effectiveness of a policy think tank – as much as the general character of its operations, including its (and its agents' or consumers') information behavior – depends on the peculiarities of the context (jurisdiction) in which it operates. A liberal “market” with many competing think tanks (leading to excessive “contestation”), as is the case in the UK, may translate into markedly increased information dissemination activities, and the field itself may be exhibitive of a high degree of noise. Sweden, for example, pronouncedly lacks these sort of characteristics.
Conclusions
In this paper we investigated two parameters that may ultimately define the information behavior of institutional agents (specifically, the associated information consumer behavior): the (1) funding model (organizational structure; public vs. private) and the (2) analysis intention (temporal outlook; ex-ante vs. ex-post approach). We compare the information production and consumption cycle in two instances: (1) a private think tank vs. (its associated) private education system, (2) a publicly funded think tank vs. (its associated) public education system. A comparison constructed this way allows one to detect differences that are due to the type, not due to the aims, goals and objectives. We rely on a systematic survey of a pool of published ex-post and ex-ante evaluations, released and used by both public (STRATA and the Seimas/Government of Lithuania) and private (LFMI and its Education Centre) institutions in the field of education policy construction and education management in the years 2017–2020. Each release was dated, and its links to observed actions (“initiatives”: changes in the legal framework, changes in the organizational structure of a school system etc.) was established, four calendars were constructed, eventually enabling the authors to compare the events on the two matching timelines (STRATA analytical work <–> public sector initiatives; LFMI analytical work <–> the activities of the LFMI Education Centre).
Overall, comparing the private and the public sector, both features in common and pronounced differences are evident. A clear common characteristic is the relative comparability of timetable intensity.
Otherwise the operations in the private and the public sector differ in:
- –the temporal properties of the commission–use cycle (privately funded cycle is of shorter duration, not as extended in time; a release is often immediately followed by its associated initiative);
- –the “productivity” (continued re-use) of any one single analytical work (privately funded (prospective, “preparatory”) work is more productive, meaning here one release can be used to give rise to several initiatives; whereas in the public sector a single legislative or executive action usually takes a variety of analytical reports as the input preparing it);
- –the tendency (or the lack thereof) to produce analytical works that are prospective (preparatory) as opposed to retrospective (estimative) or vice versa (privately funded analytical work tends to be more prospective rather than retrospective).
These findings are rather contextual. The private sector may afford the luxury of being more focused and efficient in its information behavior due to the clear and non-debatable ideological commitment that the think tanks operating in it are exhibitive of. In a deliberative democracy this is rarely possible.
Future research is needed so as to not only nuance the conclusions drawn here by extending the research period and by focusing on domains other than education policy. It would undoubtedly be of great interest to check whether the same generalizations hold for other jurisdictions: Lithuania could be claimed to be a representative of a rather young, post-Soviet administrative tradition, and the tendencies we document in this paper may be somewhat slanted due to this particular circumstance.
A talk at the conference Political Leadership in a Parliamentary Democracy (23–24 November 2021) was based on an early draft of this paper.
Acknowledgments
Our gratitude goes to prof. Dr. Andrius Vaišnys for his most helpful suggestions as to the conceptual outlook of this investigation, and to Mr. Tomas Auškalnis and Mrs. Ernesta Keinienė for their technical assistance.
References
Allern, S., & Pollack, E. (2020). The role of think tanks in the Swedish political landscape. Scandinavian Political Studies, 43(3), 145–169. https://doi.org/10.1111/1467-9477.12180.
Benneworth, P., & Peñuela, J. O. (2019). Ex ante evaluation of societal impact in research: Towards a conceptually-based reflection. Online: https://hvlopen.brage.unit.no/hvlopen-xmlui/handle/11250/2631353.
Bergström, A., & Goldmann, M. (2018). What is a successful think tank? Best practice on quantifying results (p. 63). Stockholm: European Liberal Forum asbl. Online: https://liberalforum.eu/wp-content/uploads/2021/06/Publication-WSTT_book_Online-1.pdf.
Brazinskaitė, A., & Kėvišas, M. T. (forthcoming). Ekspertinės švietimo institucijos Lietuvoje: Vyriausybės strateginės analizės centro (STRATA) indėlis formuojant nacionalinę švietimo politiką. Aktualu Rytoj /Relevant Tomorrow, 1(20).
Bublienė, D., Limantė, A., Ragauskas, P., & Vitkutė, E. A. (2017). Teisinio reguliavimo ex post vertinimo perspektyvos Lietuvoje (p. 149). Vilnius: Lietuvos teisės institutas. Online: https://teise.org/wp-content/uploads/2018/01/Ex-post3.pdf.
Craft, J., & Howlett, M. (2012). Policy formulation, governance shifts and policy influence: Location and content in policy advisory systems. Journal of Public Policy, 32(2), 79–98. https://doi.org/10.1017/S0143814X12000049.
Craft, J., & Howlett, M. (2013). The dual dynamics of policy advisory systems: The impact of externalization and politicization on policy advice. Policy and Society, 32(2), 187–197. https://doi.org/10.1016/j.polsoc.2013.07.001.
Golen, T., & Voorst, S. (2016). Towards a regulatory cycle? The use of evaluative information in impact assessments and ex–post evaluations in the European Union. European Journal of Risk Regulation, 7(2), 388–403. https://doi.org/10.1017/S1867299X00005791.
Kelstrup, J. (2017). Quantitative differences in think tank dissemination activities in Germany, Denmark and the UK. Policy Sciences, 50(1), 125–137. https://doi.org/10.1007/s11077-016-9254-0.
Kelstrup, J. D. (2020). Introduction to special issue: Think tanks in the nordic countries. Scandinavian Political Studies, 43(3), 129–144. https://doi.org/10.1111/1467-9477.12179.
Kelstrup, J. D., & Blach-Ørsten, M. (2020). Public and private think tank visibility in the Danish parliament and media. Scandinavian Political Studies, 43(3), 170–186. https://doi.org/10.1111/1467-9477.12181.
Limantė, A. (2017). Teisėkūros tobulinimo keliu: teisės aktų vertinimas, jo rūšys bei plėtotė. Teisės Problemos, 2(94), 22–49. Online: https://teise.org/wp-content/uploads/2017/12/Limante-2017-2.pdf.
Lingard, B. (2016). Think tanks, ‘policy experts’ and ‘ideas for’ education policy making in Australia. Australian Educational Researcher, 43(1), 15–23. https://doi.org/10.1007/s13384-015-0193-0.
Lubienski, C., Brewer, T. J., & La Londe, P. G. (2016). Orchestrating policy ideas: Philanthropies and think tanks in US education policy advocacy networks. Australian Educational Researcher, 43, 55–73. https://doi.org/10.1007/s13384-015-0187-y.
Lundin, M., & Öberg, P. (2014). Expert knowledge use and deliberation in local policy making. Policy Sciences, 47(1), 25–49. https://doi.org/10.1007/s11077-013-9182-1.
Mastenbroek, E., van Voorst, S., & Meuwese, A. (2016). Closing the regulatory cycle? A meta evaluation of ex-post legislative evaluations by the European commission. Journal of European Public Policy, 23(9), 1329–1348. https://doi.org/10.1080/13501763.2015.1076874.
Nordrum, J. C. F. (2013). Evalueringer av lover. In A. Halvorsen, E. L. Madsen, & N. Jentoft (Eds.), Evaluering. Tradisjoner, praksis, mangfold (pp. 91–101). Bergen: Fagbokforlaget. ISBN 978-82-450-1331-3.
Olmedo, A. (2013). Policy-makers, market advocates and edu-businesses: New and renewed players in the Spanish education policy arena. Journal of Education Policy, 28(1), 55–76. https://doi.org/10.1080/02680939.2012.689011.
Smismans, S. (2015). Policy evaluation in the EU: The challenges of linking ex ante and ex post appraisal. European Journal of Risk Regulation, 6(1), 6–26. https://doi.org/10.1017/S1867299X00004244.
Van Aeken, K. (2011). From vision to reality: Ex post evaluation of legislation. Legisprudence, 5(1), 41–68. https://doi.org/10.5235/175214611796404859.
Van Voorst, S., & Zwaan, P. (2019). The (non-)use of ex post legislative evaluations by the European Commission. Journal of European Public Policy, 26(3), 366–385. https://doi.org/10.1080/13501763.2018.1449235.
Zwaan, P., van Voorst, S., & Mastenbroek, E. (2016). Ex post legislative evaluation in the European Union: Questioning the usage of evaluations as instruments for accountability. Psychological Science in the Public Interest, 82(4), 1–20. https://doi.org/10.1177/0020852315598389.
A short survey included most of the 24 of the Lithuanian analysis centres, however, few of them publicly comment on their timetable: the Government Strategic Analysis Center (STRATA; supplies the government and ministries with the independent, research-based information), Lithuanian Free Market Institute (LFMI; a private, non-profit, nominally non-partisan think tank), The Department for Parliamentary Information Services at the National Library of Lithuania (NLL; serves at the pleasure of the parliament), The Centre for Quality Assessment in Higher Education (SKVC; an independent public institution targeting the higher education quality), The Eastern Europe Studies Centre (EESC; an independent think tank focusing on analysing international policy processes and the role that Lithuania plays in them), Vilnius Institute for Policy Analysis (VIPA; an independent think tank that promotes participatory democracy and meaningful positive change in state governance) and the Research unit at the Information and Communication Department of the Office of the Seimas.
The database of Lithuanian press (national and local periodicals), thereunder also the articles on education, is maintained by the Department for Parliamentary Information Services at the National Library of Lithuania. The coverage spans contents from 1992 onwards. Each entry (record) in the database provides detailed bibliographical information and a comprehensive (analytical) summary of the relevant article. As a new version of Lithuanian Integral Information System of Libraries (LIBIS) was launched in 2021, all previously created records (as well as new daily additions) are accessible via external NLL's catalogue ibiblioteka.lt.
