Abstract
Taiwan government consolidated the kindergarten and daycare systems in 2012, and launched a new national curriculum framework, Early Childhood Education & Care Curriculum Framework (ECECCF), as a guidance for quality early childhood education programs. Research has shown that the effects of a new educational program highly depended on the fidelity of its implementation. It has thus been suggested that the degree of implantation of a program needs be evaluated before conducting further program evaluation.
Thus, the purpose of this study was to construct an Early Childhood Education Curriculum Framework Implementation Scale (ECECCF Implementation Scale) for Taiwanese preschool programs. 216 preschool classes in Taiwan were involved. The study consisted of two stages: In Stage 1, the exploratory factor analysis showed that the implementation of ECECCF could mostly be explained by four factors, and all the factors extracted had acceptable reliability and validity. In Stage 2, rubrics were constructed for each item and factor analysis was re-conducted, resulting in a 19-item scale of four factors (Awareness and Adjustment, Learning Centers Arrangement, Teaching Guidance and Curriculum Development). The reliability and validity tests shows that: (1) the final version of the ECECCF Implementation Scale is a valid and reliable instrument, explaining 62.9% of the total variance; (2) the criterion validity indicated that the ECECCF scale can not only be used for assessing the implementation of ECECCF, but also can be used for understanding teachers’ needs in instructional and operational curriculum for further professional development.
1 Introduction
As “Early Childhood Education and Care Act” was effective in Taiwan in 2012, all early childhood education programs for ages 2 to 6 have been under the jurisdiction of the Ministry of Education. To assure the quality of young children’s educational experience not to be undermined during the transformation process, early childhood education professors, researchers, and policy makers worked together to develop a new early childhood education curriculum framework -- “Early Childhood Education & Care Curriculum Framework (ECECCF).” The ECECCF was launched in 2016, marking the beginning of a new era of Taiwan’s early childhood education.
This ECECCF consists of six areas of learning to develop young children’s six core competencies, including awareness & identification, expression & communication, concern to others & collaboration with others, reasoning & appreciation, imagination & creativity, and motor development and self-regulation. The ECECCF differs from the 1987 “Standards for Kindergarten Curriculum”, not only in the content areas, but also in the fundamental beliefs of curriculum. The ECECCF emphasizes that young children do not learning by others’ telling, but from their active interacting with surrounding people and the environment. This interaction process in turn leads to young children’s own construction of knowledge of the world. The ECECCF further advocates the concept of scaffolding and guided participation for intentional teaching. Thus, child-centered, active interaction, and scaffolding are the main ideas of the ECECCF (MOE, 2017; Shin, 2015).
According to the ECECCF, the roles of early childhood education teachers are the observers, facilitators and learning companions of young children. Teachers should understand what their young students’ living experience, development and interests, and take these as the foundation of curriculum development. Through interacting with children within the curriculum, teachers can understand and facilitate children. After interacting with young children, teachers reflect and adjust the curriculum (MOE, 2017). These ideas of curriculum, children’ learning and teachers’ role of ECECCF are based on theories of early childhood education of high quality, which are promoted by the Organization for Economic Co-operation and Development (OECD) and most countries enhancing the quality of early childhood education (OECD, 2018).
The effectiveness of ECECCF launched the new era of early childhood education in Taiwan. The government of Taiwan invested many resources to promote the new national kindergarten curriculum framework, including in-service teachers training and on-site consulting program from 2013 (Shing & Chou, 2017). All the efforts put by the stakeholders were for fulfilling the ideas of ECECCF and enhancing the qualities of ECE in Taiwan. However, launching a national curriculum does not equal its implementation. Researchers found that teachers usually do not fully implement curriculum as its original design (Hume & Coll, 2010; Olsen, 1981; Snyder, Bolin, and Zumwalt, 1992 1 ). Alignment between curriculum standards, instruction and assessment is also low (Kurz, Elliott, Wehby, & Smithson, 2010; Stein, Remillard, & Smith, 2007). They urged that researchers should take the relationship between the formal, planned, and enacted curriculum into account when evaluating a curriculum or investigating how curriculum influences student learning.
Therefore, explication of the curriculum enactment process is essential to evaluating curriculum. John I. Goodlad (1979) proposed a five-level framework of curriculum inquiry, which provided the theoretical base for curriculum researches. The five levels were ideal curriculum, formal curriculum, instructional curriculum, operational curriculum, and experiential curriculum (J. I. Goodlad, Klein, & Tye, 1979). The formal curriculum was the written set of intended learning for students, e.g. a national/state curriculum framework, guideline or textbooks. Though formal curriculum was written by words, different teachers with different experience, beliefs or backgrounds might interpret it in a different way. That is the instructional curriculum, which means teachers’ interpretation of formal curriculum and their curriculum plans. Then teachers implement curriculum of their planning, operational curriculum, although teachers might not implement all what they perceived. The theoretical framework reminds us to put attention on the process of curriculum delivery. Teachers are central to whether a curriculum is delivered consistently and effectively (Lochner, Conrad, & Graham, 2015) and curriculum implementation (operational curriculum) is the most important variable in determining student learning (Zuzovsky & Aitkin, 1991). Therefore, if we want to consider whether a curriculum works or not we must consider the degree of its implementation or we may make misunderstandings.
By literature review, we found that researchers usually used three kinds of methods to investigate the curriculum implementation: observing teachers teaching, collecting students’ learning performance and investing content coverage index Observation may be the most accurate method, but it is time consuming and difficult to compare the differences between classrooms or institutions. In contrast, data gathered from content coverage index could be compared between different settings. However, some researches founded that self-rating of performance, may be caused by some personal or cultural factors, tend to show lenient (Kwan, John, Robins, & Kuang, 2008; Brown, 2010; Sinha, Mesmer-Magnus, & Viswesvaran, 2012). To overcome the problems, our team decided to integrate observational and index method and to develop an instrument with a clear index and used by trained observers. The mix method has the strength of accuracy of observation and can be compared more easily.
Researchers hope this scale can catch teachers’ interpretation and implementation of ECECCF and can be widely used by researches, institute leaders or coaches. Data gathered by this scale can be used to help teachers, school and government administrators for many purposes, like understanding what’s the actual situation of the perceived and operational curriculum of ECECCF, what teachers might need and knowing what might need to do next for improving the enactment of ECECCF.
2 Research purpose
In response to the empirical study needs and to make up the insufficiency of related tool for assessing the operational curriculum of national curriculum framework, this study aimed to develop a reliable and validity instrument to assess what teachers actually perceived and done with ECECCF in their classroom. A valid and reliable instrument could serve as a blueprint for developing criteria that can be used to define key elements of ECECCF implementation and explicit the process of operation in different ECE settings, that would assist teachers, curriculum leaders or coached for understanding the actual situation and planning the next implementation plans. Researchers and policy makers also can use the scale to evaluate the effects of ECECCF.
The development and validation of the tool for accessing ECECCF implementation are presented and discussed in this paper.
3 Method
The development and the validation of ECECCF Implementation Scale in this study followed the distinct procedure proposed by DeVellis (2012) and Boateng, Neilands, Frongillo, Melgar-Quiñonez, and L.Young (2018). The process can be divided into three main phases: item development and two stages of scale development and evaluation, as outline in Fig. 1.
Three main phases of developing the ECECCF implementation scale
Citation: Hungarian Educational Research Journal 11, 4; 10.1556/063.2021.00041
3.1 Phase I: Item development
Important works in this phase were clarifying and generating item pool and having an initial item pool reviewed by experts. First, researchers reviewed the publications and theories of ECECCF to develop the category and item pool to form a preliminary scale. Then we conducted two expert consulting meetings that recruited 16 early childhood professors to check the reasonability and importance of the items. The study invited professors from departments of early childhood education and early childhood care. This study refined the items based on experts’ comments and then 56 criteria were identified, which cover four dimensions: class management, learning centers design and practice, developing core competencies and reflection of teaching.
3.1.1 Phase II: Scale development and evaluation (1)
In this phase, we determined the format of scale and conducted a pilot study. A 4-point scale was applied: with “4” for high degree of and “1” for low degree of implementation (Larson, Day, Springer, Clark, & Vogel, 2003; Thompson, Clemmensen, & Ahn, 2013).
Researchers trained nine observers to make non-participated observations in 59 classrooms within 30 schools around Taiwan to collect the pilot study data. The background information of participated schools are shown in Table 1.
Participating kindergartens in phase I
| Area | Number of Schools | Number of Classes | Number of Teachers | Percentage of Classes (%) |
| Northern Taiwan | 12 | 2 | 5 | 40.6 |
| Central Taiwan | 5 | 2 | 4 | 16.9 |
| Southern Taiwan | 10 | 2 | 4 | 32.1 |
| Eastern Taiwan | 3 | 2 | 4 | 10.1 |
| Total | 30 | 59 | 122 | 100 |
The trained observers completed the scale by observing teaching and learning activities (operational curriculum). In addition to the observation, observers conducted semi-structured interviews with main teacher in each class. The interviews were inquiries of what teachers thought and did about their curriculum and what aspects of their guidance of students and reflection of their curriculum implementation (instructional curriculum). Observers integrated observing and interviewing data to complete the scale. The reliability between the observers was 0.86. All data were collected between May to June 2017.
3.1.2 Phase II: Scale evaluation (1)
This study used Item Analysis to check the validation of each item and Exploratory Factor Analysis to construct the components of the scale.
We chose those items that met the two qualifications: first, item discrimination index was above 0.5. Second, item correlation with total scale was above 0.3. Based on the results of the Item Analysis, we deleted 5 items. Exploratory Factor Analysis and oblique rotations were conducted on the remaining 51 items using SPSS version 24. Components with eigenvalue exceeding 1.0 are extracted into different components, and items with low component loadings (<0.4) were further deleted (Comrey & Lee, 1992; Tabachnick & Fidell, 2007). Finally, we got a three-factor scale with 28 items, explaining 66.310% of the total variance (as showed in Table 2). The Cronbach’s α value for the full scale was 0.966. It was valid and reliable.
Total variance explained (n = 59)
| Component | Extraction Sum of Loadings | ||
| Total | % of variance | Cumulative % | |
| 1 | 14.354 | 49.496 | 49.496 |
| 2 | 2.870 | 9.896 | 59.392 |
| 3 | 2.006 | 6.918 | 66.310 |
3.2 Phase III: Rating rubrics
After scale evaluation (1), our team conducted an observers’ meeting to gather their advice about data collection plan for the next step. Trained Observers recommended adding rubrics for each item for user friendly. Our team reviewed literature and found that scoring with rubrics tended more reliable than without one (Jonsson & Svingby, 2007). Therefore, we established rubrics and changed the format and conducted stage II validation of the scale in 2018.
There were 157 classes within 33 schools in the second stage, including 10 in the northern area, 11 in the central area, 7 in the southern area, and 5 in the eastern area (See Table 3). Fifteen trained observers took non-participated observation during May to June in 2018 2 .
Participating Kindergartens in phase II
| Area | Number of schools | Public School | Private School | Number of Classes | Number of Teachers | Number of Students |
| Northern Taiwan | 10 | 3 | 7 | 68 (43.31%) | 114 | 762 |
| Central Taiwan | 11 | 7 | 4 | 46 (29.30%) | 87 | 574 |
| Southern Taiwan | 7 | 4 | 3 | 27 (17.20%) | 48 | 397 |
| Eastern Taiwan | 5 | 2 | 3 | 16 (10.19%) | 31 | 223 |
| Total | 33 | 16 | 17 | 157 (100%) | 280 | 1956 |
Observers conducted 157 data collections from 33 schools, and researchers analyzed the data by using item analysis and exploratory factor analysis. After item analysis, we deleted 8 items with low discrimination index and then conducted the Exploratory Factor Analysis with the remaining 20 items. There were five factors with eigenvalue exceeding 1.0; however the fifth component had only one item. As the minimum number items of a factor should be three (Tabachnick & Fidell, 2007), we deleted that item, and ran exploratory factor analysis again. The EFA had extracted four components with 19 items and the total variance explained 62.905% (as showed in Table 4).
Total variance explained (n = 157)
| Component | Extraction Sum of Loadings | ||
| Total | % of variance | Cumulative % | |
| 1 | 7.668 | 40.356 | 40.356 |
| 2 | 1.792 | 9.431 | 49.786 |
| 3 | 1.355 | 7.129 | 56.916 |
| 4 | 1.138 | 5.989 | 62.905 |
4 Results
The final scale had two parts. The first part included demographic questions about the class (e.g., curriculum approach, class schedule, students’ age etc.). The second part was the 19-item scale.
4.1 The structure of ECECCF scale
The ECECCF Implementation Scale consists of 4 factors: Awareness and Adjustment, Learning Center Arrangement, Teaching Guidance, and Curriculum Development. Although the trends of curriculum emphasize learner centered and inquiry learning, ECECCF put emphasis on teachers playing critical roles in curriculum awareness and development at the same time. The factor of Awareness and Adjustment underlines that teachers should understand children’s abilities through carefully observing and the understanding should be the basis of adjustment of teaching practice and guidance. The factor of Learning Center Arrangement indicate that teachers should prepare sufficient and vary materials for learning centers, and should pay attention to the development appropriation of materials. The factor of Teaching Guidance emphasizes teachers involving and observing children’s learning center activities and giving the suitable guidance to deepen children’s learning. Finally, the factor of Curriculum Development assesses whether teachers use ECECCF in all their curriculum plans and if they can choose the appropriate goals for curriculum activities. In sum, ECECCF emphasize that teachers should be aware of what they were thinking and doing with the curriculum, and give appropriate guidance for children’s learning (MOE, 2017; Shin, 2015; Shing & Chou, 2017). These four factors represent the core spirits of ECECCF.
There are 8, 3, 4 and 4 items in factor 1 to 4, respectively. Every item has its rating rubrics to promote the evaluation reliability and user friendliness. The sample items are shown in Table 5.
Samples of scale item
|
Observers rated each item based on data gathered by observation or interviewing or both methods. Internal consistency coefficients of total scale, each item and four subscales are higher than 0.7, meeting the good reliability level set by DeVellis (2003) (showed in Table 6).
Reliability analysis for the scale
| Components of Scale | Items No. | Scale Mean if Deleted | Scale Variance if Item Deleted | Corrected Item-Total Correlation | Squared Multiple Correlation | Alpha if Item Deleted | Methodology | |
| Observation | Semi-structure Interview | |||||||
| Awareness and adjustment | 1 | 16.04 | 18.043 | 0.508 | 0.327 | 0.857 | V | |
| 11 | 15.53 | 17.174 | 0.635 | 0.431 | 0.843 | V | ||
| 12 | 16.08 | 17.917 | 0.649 | 0.441 | 0.843 | V | V | |
| 14 | 15.65 | 16.203 | 0.633 | 0.500 | 0.845 | V | ||
| 15 | 15.95 | 17.279 | 0.685 | 0.557 | 0.838 | V | ||
| 16 | 15.55 | 16.620 | 0.689 | 0.608 | 0.836 | V | ||
| 17 | 15.67 | 16.787 | 0.734 | 0.645 | 0.832 | V | ||
| 18 | 15.31 | 18.688 | 0.396 | 0.238 | 0.869 | V | ||
| Subscale 1 | 8 items | |||||||
| Cronbach α = 0.862 | Standardized item alpha = 0.865 | |||||||
| Learning center arrangement | 3 | 3.74 | 2.694 | 0.575 | 0.331 | 0.613 | V | |
| 4 | 4.13 | 2.958 | 0.550 | 0.305 | 0.643 | V | ||
| 5 | 4.18 | 3.023 | 0.530 | 0.281 | 0.666 | V | ||
| Subscale 2 | 3 items | |||||||
| Cronbach α = 0.729 | Standardized item alpha = 0.729 | |||||||
| Teaching guidance | 6 | 6.49 | 5.328 | 0.642 | 0.581 | 0.792 | V | V |
| 7 | 6.28 | 4.588 | 0.768 | 0.664 | 0.731 | V | V | |
| 13 | 6.55 | 5.120 | 0.694 | 0.504 | 0.769 | V | V | |
| 19 | 6.36 | 5.731 | 0.534 | 0.370 | 0.837 | V | ||
| Subscale | 4 items | |||||||
| Cronbach α = 0.830 | Standardized item alpha = 0.829 | |||||||
| Curriculum development | 2 | 6.62 | 5.007 | 0.617 | 0.383 | 0.764 | V | V |
| 8 | 6.79 | 4.693 | 0.644 | 0.423 | 0.752 | V | ||
| 9 | 6.82 | 5.532 | 0.590 | 0.363 | 0.776 | V | ||
| 10 | 7.19 | 5.284 | 0.661 | 0.443 | 0.745 | V | ||
| Subscale | 4 items | |||||||
| Cronbach α = 0.808 | Standardized item alpha = 0.811 | |||||||
| Total 19 items | ||||||||
| Cronbach α = 0.910 | Standardized item alpha = 0.913 | |||||||
The average of total scores of ECECCF Scale ranged from 1.58 to 4.00 with a mean of 2.81 and a standard deviation of 0.5497. The highest score of subscales is Learning Center Arrangement, while Curriculum Development is the lowest one. Descriptive statistics of four subscales were showed as Table 7.
Descriptive statistics of ECECCF Scale
| Subscale | N | Minimum | Maximum | Mean | Std. Deviation |
| Awareness and Adjustment | 157 | 1.25 | 4.00 | 2.75 | 0.5891 |
| Learning Center Arrangement | 157 | 1.00 | 4.00 | 2.99 | 0.7979 |
| Teaching Guidance | 157 | 1.00 | 4.00 | 2.86 | 0.7389 |
| Curriculum Development | 157 | 1.00 | 4.00 | 2.72 | 0.7317 |
| Total Scale | 157 | 1.58 | 4.00 | 2.81 | 0.5497 |
This study analyzed four subscale scores with different school types, public versus private owned school. Data analysis showed that public schools did better job on the Learning Center Arrangement, Teaching Guidance, and the total average of the ECECCF Scale than the private owned schools (as showed in Table 8). However, both public and private owned schools got the lowest scores in subscale of Curriculum Development and the Awareness and Adjustment came next. Researchers of curriculum coherence indicated that providing a coherent basis for building shared understandings of the goals of a curriculum is important for the reform to take root (Sullanmaa, Pyhältö, Pietarinen, & Soini, 2019). The main idea of the Curriculum Development subscale aims at understanding the interpretation and applying of the goals of ECECCF. It may need more time for teachers to get familiar with and to utilize them.
T-test of types of schools (N = 157)
| Subscale | Public (n = 56) | Private owned (n = 101) | t |
| Awareness and Adjustment | 2.84 (0.589) | 2.70 (0.586) | 1.425 |
| Learning Center Arrangement | 3.23 (0.705) | 2.86 (0.820) | 2.804** |
| Teaching Guidance | 3.10 (0.689) | 2.73 (0.736) | 3.150** |
| Curriculum Development | 2.83 (0.666) | 2.65 (0.761) | 1.535 |
| Average | 2.96 (0.490) | 2.72 (0.565) | 2.597** |
**P < 0.01.
In sum, the ECECCF Implementation Scale meets the high reliability standard and has construction validity based on the results of exploratory factor analysis. However, former researchers reminded us to be careful about threats to construction validity by multiple measurements for constructs (Abowitz & Toole, 2010; Farrington, 2003; Shadish, Cook, & Campbell, 2002). This study checked two more construction validity of ECECCF Implementation Scale to get further confirmation of the scale validity.
4.2 Construction validity with years of receiving ECECCF on-site consultant
For helping teachers understanding and implementing ECECCF, Ministry of Education ran the on-site consulting program since 2013. All experts were qualified by Ministry of Education to ensure their knowledge and ability to be a consultant of ECECCF. By participating in this program, kindergartens could invite an expert to work with them in the process of implementing ECECCF (Shing & Chou, 2017). Usually, the experts went to kindergarten once a month. They observed each class in the morning and discussed with teachers about their teaching at noon.
Teachers who participated in the consulting program were supposed to be more familiar with ECECCF and could implement ECECCF with more fidelity. This study tested the difference between different groups of years of participating in ECECCF consulting program. Classes were divided into two groups: “1” participating consulting program for less than three years, “2” participating consulting program for more than three years (including 3 years). The results of t-test showed that the average of ECECCF Implementation Scale was significantly different between these two groups (see Table 9).
t-test of years of ECECCF consulting program participating (N = 157)
| Subscale | Group 1 (n = 96) | Group 2 (n = 61) | t |
| Awareness and Adjustment | 2.63 (0.579) | 2.95 (0.554) | −3.452** |
| Learning Center Arrangement | 2.86 (0.844) | 3.20 (0.673) | −2.827** |
| Teaching Guidance | 2.73 (0.713) | 3.06 (0.741) | −2.724** |
| Curriculum Development | 2.55 (0.724) | 2.98 (0.670) | −3.696*** |
| Average | 2.67 (0.542) | 3.02 (0.485) | −4.048*** |
*P < 0.05. **P < 0.01. ***P < 0.001.
Classes that participated in ECECCF consulting program for more than 3 years got higher score in all four subscales and the overall scale. The overall average of group 1 was 2.67, and the average of group 2 was 3.02. Calculated t value in the data is -4.048, which is less than table value 2.67 at 0.01 level of significance. The difference existed in all four subscales. It meant that teachers who participated in ECECCF consulting program for more than 3 years got higher ECECCF Scale score than teachers participated in the program less than 3 years.
Garet, Porter, Desimone, Birman, and Yoon (2001) studied American teachers’ professional development program and indicated that sustained over time and involves a substantial number of hours were critical for higher quality of PD program. They indicated that “longer professional development activities tend to include substantially more opportunities for active learning, for planning classroom implementation, observe and be observed teaching and to promote connections to a teacher’s goals and experience alignment with standards (p. 933)”. Study of Kho Siaw, Thapanee, and Mohamed Ismail (2020) also proposed the importance of time spent in the coaching program. The relation of years of receiving on-site consulting and scores of ECECCF Implementation Scale showed that the scale could tell the differences between teachers with different years involving in ECECCF. Therefore, researchers in the study believed that the ECECCF Implementation Scale is valid to exam the degree of teachers’ curriculum implementation.
4.3 Construction validity with young children’s core competencies
Since the ECECCF means to enhance children’s six core competencies, it is important to see if the better implementation of ECECCF, the better children’s six core competencies were. We tested the correlation between the scale of the classrooms and the young children’s progress in six core competencies in each classroom.
Data came from Taiwan Early Childhood Education Panel Study (2nd year). The study sample consisted of 157 classes and 1926 children within 33 kindergartens. Children’s six core competencies were collected by Children’s Learning Outcomes Questionnaire (Lai, Liaw, & Chang, 2017). Learning Outcomes Questionnaire was designed to evaluate children’s performance of each core competency from age 3–6 and to be fulfilled by children’s kindergarten teachers. The test-retest reliability coefficients ranged between α = 0.947 to 0.781, for well-trained users and non-trained users on different subscales. The developer of this questionnaire also tested the alternate-form reliability. Pearson Correlation was conducted for comparing the result of Learning Outcomes Questionnaire and Learning Outcomes Checklist, which was well developed for evaluating children’s six core competencies (Liaw et al., 2016). Pearson Correlation between these two tools was from 0.811 to 0.909 (the results of Imagination & Creativity is 0.741, which was the only result that was below 0.80).
Children’s six core competencies were collected at the beginning of the first semester (pre-test) and the end of the second semester (post-test) in 2017 school year. The descriptive statistics of 3–6 year-old children’s six core competencies scores were shown in Table 10.
Descriptive statistics of children's six core competencies score (pre- and post-test)
| Core Competency | Age group | Pre-test | Post-test | ||||
| N | Mean | Std. Deviation | N | Mean | Std. Deviation | ||
| Awareness & Identification | |||||||
| 3 year-old | 103 | 1.20 | 0.436 | 119 | 1.30 | 0.452 | |
| 4 year-old | 437 | 1.51 | 0.626 | 455 | 1.81 | 0.694 | |
| 5 year-old | 637 | 1.87 | 0.723 | 650 | 2.20 | 0.752 | |
| 6 year-old | 691 | 2.45 | 0.870 | 696 | 2.72 | 0.822 | |
| Expression & Communication | |||||||
| 3 year-old | 103 | 1.60 | 0.657 | 119 | 1.73 | 0.684 | |
| 4 year-old | 437 | 1.93 | 0.832 | 455 | 2.38 | 0.877 | |
| 5 year-old | 637 | 2.52 | 0.869 | 650 | 2.84 | 0.830 | |
| 6 year-old | 691 | 2.95 | 0.819 | 696 | 3.20 | 0.761 | |
| Concern to Others & Collaboration with Others | |||||||
| 3 year-old | 103 | 1.48 | 0.739 | 119 | 1.70 | 0.671 | |
| 4 year-old | 437 | 1.74 | 0.774 | 455 | 2.22 | 0.872 | |
| 5 year-old | 637 | 2.14 | 0.866 | 650 | 2.50 | 0.873 | |
| 6 year-old | 691 | 2.59 | 0.874 | 696 | 2.86 | 0.893 | |
| Reasoning & Appreciation | |||||||
| 3 year-old | 103 | 1.23 | 0.596 | 119 | 1.37 | 0.608 | |
| 4 year-old | 437 | 1.66 | 0.835 | 455 | 2.03 | 0.899 | |
| 5 year-old | 637 | 2.16 | 0.895 | 650 | 2.51 | 0.889 | |
| 6 year-old | 691 | 2.70 | 0.951 | 696 | 3.01 | 0.848 | |
| Imagination & Creativity | |||||||
| 3 year-old | 103 | 1.13 | 0.416 | 119 | 1.36 | 0.500 | |
| 4 year-old | 437 | 1.54 | 0.720 | 455 | 1.89 | 0.859 | |
| 5 year-old | 637 | 1.95 | 0.833 | 650 | 2.22 | 0.916 | |
| 6 year-old | 691 | 2.35 | 0.960 | 696 | 2.66 | 0.943 | |
| Self Regulation | |||||||
| 3 year-old | 103 | 1.77 | 0.669 | 119 | 1.99 | 0.633 | |
| 4 year-old | 437 | 2.07 | 0.759 | 455 | 2.51 | 0.804 | |
| 5 year-old | 637 | 2.65 | 0.793 | 650 | 2.95 | 0.723 | |
| 6 year-old | 691 | 3.05 | 0.706 | 696 | 3.28 | 0.690 | |
We tested if the degree of ECECCF implementation affected children’s progress. For the ANOVA test, children were divided into three groups based on their classes’ scores of ECECCF Scale. The percentage of scores was used as a reference to divide the participants. Participants with top 27% scores were put in the High-score group, lower 27% were put in Low-score group and those between high and low were Middle-score group. Researchers conducted ANOVA to examine the difference between the progress of pre-post scores of children’s six core competencies among High, Middle and Low ECECCF groups. Table 11 displays the summary of the results.
ANOVA analysis of different score of ECECCF with young children's core competencies
| Low ECECCF score (1) (N = 543) |
Middle ECECCF score (2) (N = 819) |
High ECECCF Score (3) (N = 491) |
|||
| Post-Pre test Mean Gain | Post-Pre test Mean Gain | Post-Pre test Mean Gain | F | Post hoc | |
| Awareness & Identification | 0.178 (0.538) | 0.254 (0.594) | 0.423 (0.597) | 24.164*** | 3>1 & 2 |
| Expression & Communication | 0.193 (0.590) | 0.303 (0.596) | 0.454 (0.663) | 23.636*** | 3>2>1 |
| Concern to Others & Collaboration with Others | 0.195 (0.664) | 0.322 (0.742) | 0.546 (0.823) | 29.528*** | 3>2>1 |
| Reasoning & Appreciation | 0.241 (0.698) | 0.281 (0.715) | 0.485 (0.744) | 17.460*** | 3>2>1 |
| Imagination & Creativity | 0.164 (0.708) | 0.288 (0.743) | 0.460 (0.804) | 20.212*** | 3>2>1 |
| Self Regulation | 0.189 (0.543) | 0.303 (0.632) | 0.409 (0.654) | 16.697*** | 3>2>1 |
| Average of Six Core Competencies | 88.30 (11.220) | 92.41 (11.373) | 93.22 (10.840) | 38.931*** | 3>2>1 |
***P < 0.001.
The result indicated that there were statistically significant differences between the low, medium and high scores groups in the progress of children’s six core competencies. Researchers also ran the Scheffe’s Post Hoc Test to exam the differences among the three level groups. The results showed that children who were in the high ECECCF implemented classrooms outperformed significantly both the medium and low levels classrooms and children who were in the medium level classrooms progressed more than who were in the low-level classrooms. The only exception was that there was no significant difference between the progress of young children’s Awareness & Identification competency in the medium and low level ECECCF implemented classrooms.
Since the ECECCF aimed to develop children’s six core competencies from age 3–6, and children in high ECECCF implementation group indeed showed more progression in six core competencies. This result indicated that the scale of ECECCF could evaluate the implementation degree of ECECCF precisely, and the ECECCF Implementation Scale is a valid instrument.
5 Discussions and conclusion
5.1 General findings
The aim of this study was to develop the ECECCF Implementation Scale for assessing teachers’ implementation of Taiwan’s new national curriculum framework of early childhood education in class settings. Curriculum implementation contained both interpretational and operational elements, so the scale occupied observation and interview method. Every item has its rating rubrics for observers identifying the implementation degree. With the rigorous procedures, the scale has high reliability, construction, and construction validity. Reliability of total scale and all subscales are higher than 0.7, meeting the qualification of high reliability (DeVellis, 2012).
According to item analysis and exploratory factor analysis, the scale obtained four subscales with 19 items: Awareness and Adjustment, Learning Centers Arrangement, Teaching Guidance and Curriculum Development. The totally explained variation is 62.905%. Generally, the average total scores of ECECCF Scale ranged from 1.58 to 4.00 with a mean of 2.81 and a standard deviation of 0.5497. The highest score of subscales is Learning Center Arrangement, while Curriculum Development is the lowest one. Data analysis showed that public schools did better job on the Learning Center Arrangement, Teaching Guidance, and the total average of the ECECCF Scale than the private owned schools.
The study conducted criterion-validity of ECECCF Implementation Scale with years receiving of ECECCF on-site consulting and young children’s progress on six core competencies. Data showed that teachers who had participated in consulting program for more than 3 years got higher scores in all subscales and total scale. Moreover, there were statistically significant differences between the low, medium and high ECECCF Implementation Scale scores groups in the progress of children’s six core competencies. Generally, children who were in the high ECECCF implemented classrooms outperformed significantly both the medium and low levels classrooms and children who were in the medium level classrooms progressed more than those who were in the low-level classrooms.
Results of this study showed that the scale of ECECCF could evaluate the implementation degree of ECECCF precisely, and the ECECCF Implementation Scale is a valid instrument. It could be used by early childhood education researchers, school curriculum leaders and coaches to understand the operating curriculum of ECECCF. While with some extending, we believe that teachers for self-understanding and professional development could effectively use the observational scale, too.
5.2 ECECCF consulting program and implementation
In this study, we check the construction validity of The ECECCF Implementation Scale with the years of ECECCF on-site consultant and the six core competencies of young children.
Taiwan Ministry of Education invests in their continuous endeavor in producing high quality early childhood education and teachers. Taiwan MOE had initiated kindergarten on-site consulting program for ECECCF four years before ECECCF had been launched. The on-site consulting program keeps providing support and guidance in improving in-service teachers’ understanding and implementation of the ECECCF. Our study revealed that the degree of ECECCF implementation was related to the years of receiving consulting. This finding corresponded to the findings of earlier researches, which indicated that time span and contact hours have a substantial positive influence on the quality of professional development programs (Garet et al., 2001; Kho Siaw et al., 2020).
Researchers have consensus on the reasons of why time matters on the quality of professional development program, e.g. more opportunities for active learning, for planning classroom implementation, observe and be observed teaching and experience alignment with standards (Devine, Meyers, & Houssemand, 2013; Garet et al., 2001; Langon, Alexander, Farquhar, & Tesar, 2016). However, there is still no agreement on how much time is enough. Our study found that three years maybe the necessary time to understand and operate a national curriculum profoundly for teachers. This study discovered that teachers who participated in a consulting program for more than three years tended to implement ECECCF more profoundly. Although what actually happened or how teachers changed their understanding and practice by time, still need more studies to discover.
5.3 Teachers’ Implementation of ECECCF and progression of young Children’s six core competencies
Greater alignment between teachers’ instruction and national mathematics standards was directly and positively associated with higher student achievement scores, especially after students have been exposed to the instruction curriculum for a sustained period, for longer than 6 months (Kurz et al., 2010). Our study collected children’s six core competencies at the beginning and the end of 2017 school year and checked if children’s progression differed between different levels of implementation of ECECCF in classrooms. Results showed that young children who in high degree of classes got more six core competencies progression than those in medium and low implementation degree classes. The finding corresponded to the former researches and showed that The ECECCF Implementation Scale is valid.
Based on our research, researchers can evaluate the relationship of national curriculum implementation and its effects, e.g. teachers consulting or professional development programs, children’ learning and variety of policy investments. We especially recommend for longitudinal researches on issues we proposed above, because longitudinal researches not only tell us about what effects they are, but also the characteristics of the changing process both of the group and individual level. It can provide more detail information for stakeholders of national curriculum for further actions.
5.4 Limitations
Despite the potential applicability of this scale, there are several limitations that should be addressed in further studies. First, for preventing disturbing teachers and children’s daily life too much, this scale was developed to collect data in one day. We inquire teachers’ implementation by observing, interviewing and check the schedule of class without collecting documents of teachers’ curriculum plans, records or reflection. Though the results indicated that the scale was still valid. However, it might get more further understanding of the continuity and reflection of curriculum by collecting relatively curriculum documents, such as curriculum plans, records or reflection. We think this could be added in the future.
Secondly, observation and interviews allow for an in-depth understanding of teachers’ operating curriculum. However, the reliability and validity rely on valid training of observers. It takes time and concrete protocols. If we expect the scale would be used widely, then developing standard training procedure and training manual would be the essential work for next step.
Finally, although we conducted two stages of exploratory factor analysis (EFA) and criterion-validity to confirm the validity of ECECCF Implementation Scale, we have not checked the confirmatory factor analysis (CFA) of the scale yet. Confirmatory analysis usually is used to investigate valuable information regarding the fit of the data to theory-derived measurement model. It also can point to the potential weakness of specific item. Therefore, we recommend future researchers conduct CFA base on those were extracted in this study to test if the measurement fit to the construction structure.
Curriculum implementation is a complex and long journey. This study represents an attempt to clarify the gap between what is considered essential to the teaching of ECECCF with the reality of teaching in educational settings. In the future, the scale can be used to identify strengths and challenges in the implementation process for researchers, school leaders, coaches and teachers, and to make recommendations aimed at supporting future curriculum implementation.
About the authors
Ching-Wen Chang is an assistant professor in National Taitung University. Her research topics are about teachers’ professional development, practical knowledge, inquiry learning and assessment in early childhood education.
Shan-Shan Cheng is a post-doctoral researcher in National Chiayi University.
Yi-Wen Chang is a kindergarten teacher in Fuxing Elementary School.
Fong-Rui Liaw is a retired associate professor in Taiwan Normal University.
Ching-Ching Chen is a professor in National Chiayi University.
Acknowledgement
This study was funded by the Ministry of Education, Taiwan.
References
Abowitz, D. A. , & Toole, T. M. (2010). Mixed method research: Fundamental issues of design, validity and reliability in Construction. Journal of Construction Engineering and Management, 36(1), 108–116. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000026.
Boateng, G. O. , Neilands, T. B. , Frongillo, E. A. , Melgar-Quiñonez, H. R. , & Young, S. L. (2018). Best practices for developing and zalidating scales for health, social, and behavioral research: A primer. Front. Public Health, 6(149). https://doi.org/10.3389/fpubh.2018.00149.
Brown, J. D. (2010). Across the (not so) great divide: Cultural similarities in self-evaluative processes. Social and Personality Pyschology Compass, 4, 318–330. doi: https://doi.org/10.1111/j.1751-9004.2010.00267.
Comrey, A. L. , & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
De Vellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage Publications.
DeVellis, R. F. (2012). Scale development: Theory and pplications (3rd ed.). Thousand Oaks, Calif: Sage.
Devine, M. , Meyers, R. , & Houssemand, C. (2013). How can coaching make a positive impact within educational settings? Social and Behavioral Sciences, 93, 1382–1389.
Farrington, D. P. (2003). Methodological quality standards for evaluation research. Annals of the American Academy of Political and Social Science, 587, 49–68. https://doi.org/10.1177/0002716202250789.
Garet, M. S. , Porter, A. C. , Desimone, L. , Birman, B. F. , & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Education Research Journal, 38(4), 915–945.
Goodlad, J. I. (1979). Curriculum inquiry: The study of curriculum practice. New York: McGraw-Hill.
Goodlad, J. I. , Klein, F. , & Tye, K. A. (1979). The domains of curriculum and their study. In J. I. Goodlad (Ed.), Curriculum inquiry: The story of curriculum practice (pp. 43–76). New York: McGraw-Hill.
Hume, A. , & Coll, R. K. (2010). Authentic student inquiry: The mismatch between the intended curriculum and the student-experienced curriculum. Research in Science & Technological Education, 28(1), 43–62.
Jonsson, A. , & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Review, 2, 130–144.
Kho Siaw, H. , Thapanee, K. , & Mohamed Ismail, S. A. M. (2020). Keeping teachers aflfloat with instructional coaching: Coaching structure and implementation. Qualitative Report, 25(7), 1790–1816.
Kurz, A. , Elliott, S. N. , Wehby, J. H. , & Smithson, J. (2010). Alignment of the intended, planned, and enacted curriculum in general and special education and its relation to student achievement. The Journal of Special Education, 44(3), 131–145.
Kwan, V. S. , John, O. P. , Robins, R. W. , & Kuang, L. L. (2008). Conceptualizing and assessing self-enhancement bias: A componential approach. Journal of Personality and Social Psychology, 94(6), 1062–1077. https://doi.org/10.1037/0022-3514.94.6.1062.
Lai, W. F. , Liaw, F. R. , & Chang, C. W. (2017). Taiwan early childhood education Panel study (2nd year). Taipei: Ministry of Education.
Langon, F. J. , Alexander, P. A. , Farquhar, S. , & Tesar, M. (2016). Induction and mentoring in early childhood educational organizations: Embracing the complexity of teacher learning in contexts. Teaching and Teacher Education, 57, 150–160.
Larson, L. M. , Day, S. X. , Springer, S. H. , Clark, M. P. , & Vogel, D. L. (2003). Developing a supervisor feedback rating scale: A brief report. Measurement and Evaluation in Counseling and Development, 35(4), 230–238.
Liaw, F. R. , Lai, W. F. , Shing, M. L. , & Chang, C. W. (2016). Taiwan early childhood education panel study (1st year). Taipei: Ministry of Education.
Lochner, B. , Conrad, R. M. , & Graham, E. (2015). Secondary teachers’ concerns in adopting learning management systems: A U.S. Perspective. Tech Trends: Linking Research and Practice to Improve Learning, 59(5), 62–70.
MOE . (2017). The early childhood education & care curriculum framework. Taichung City: K-12 education administration.
OECD . (2018). Overview: Promoting quality early childhood education and care, child development and learning. In Engaging young children: Lessons from research about quality in early childhood education and care. Paris: OECD Publishing.
Olson, J.K. (1981). Teacher influence in the classroom: a context-for understanding curriculum translation. Instructional Science, 10(3), 259–275.
Shadish, W. R. , Cook, T. D. , & Campbell, D. T. (2002). Experimental and quasi- experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Shin, M.-L. (2015). Genaral outline. In M.-L. Shin , K.-B. Yang , C.-H. Q I , H.-W. Ke , M.-L. Tsi , R.-T. Kim , Z.-W. Kuo-Li , S.-Z. Jian , & M.-J. Lin (Eds.), What does the new framework want to say: The development of the early childhood education & care curriculum framework. Taipei City: Pyschological Publishing.
Shing, M.-L. , & Chou, Y.-P. (2017). Implementation of early childhood education & Care curriculum framework of preschool educators in Taiwan: From the perspective of a researcher developed during on-site visits. Journal of Textbook Research, 10(3), 101–131.
Sinha, N. , Mesmer-Magnus, J. , & Viswesvaran, C. (2012). Personality antecedents of self-other rating discrepancy. Revistade Psicologiadel Trabajoydelas Organizaciones, 28(1), 3–12. https://doi.org/10.5093/tr2012a1.
Snyder, J. , Bolin, F. , & Zumwalt, K. (1992). Curriculum implementation. In Jakson, W. P. (Ed.), Handbook of research on curriculum. New York: Macmillan Publishing Company.
Stein, M. , Remillard, J. , & Smith, M. S. (2007). How curriculum influences student learning. In F. K. L. Jr (Ed.), Second handbook of research on mathematics teaching and learning (pp. 319–369). Gweenwich, CT: Information Age.
Sullanmaa, J. , Pyhältö, K. , Pietarinen, J. , & Soini, T. (2019). Curriculum coherence as perceived by district-level stakeholders in large-scale national curriculum reform in Finland. Curriculum Journal, 30(3), 244–263.
Tabachnick, B. G. , & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson.
Thompson, M. K. , Clemmensen, L. K. H. , & Ahn, B.-u. (2013). The effect of rubric rating scale on the evaluation of engineering design projects. International Journal of Engineering Education, 29(6), 1490–1502.
Zuzovsky, R. , & Aitkin, M. (1991). Curricular changes and science achievement in Israeli elementary schools. In S. W. Raudenbush , & J. D. Willms (Eds.), School classrooms and pupils (pp. 25–36). San Diego: Academic Press.
Snyder, Bolin, and Zumwalt (1992) ‘‘a major modification to the curriculum implementation literature was made by researchers who discovered that, in reality, curriculum was never really implemented as planned, but rather adapted by local users’’ (p. 428).
We collected data at the end of school year to ensure teachers had implemented ECECCF for at least 6 months.
