Since the collapse of the Pol Pot regime until early 2000s the term Student Learning Achievement was occasionally mentioned. The government at that time essentially focused on how to gather children and adults to go to school to learn from the people who just had better knowledge than their students. As a result, just between 1998 and 2003 the percentage of children entering primary school expanded by almost 36 percent (EMIS, 2003).
But with the remarkable gains in numbers of student enrolment the education quality, in general, seem to be hard to improve.
MoEYS recognizes that careful attention must be given to ensure that policy-led decisions are consonant with improved learning achievement… Most schools do not fulfill expected minimum contact hours per year. Continuous classroom monitoring of individual students' mastery of essential learning competencies and skills, linked to ongoing remediation, and regular, independent 'auditing' of educational quality and standards have yet to be translated into programs that can be implemented nationwide (EFA National Plan 2003-2015, p.25).
If the quality issues had been asked for, specifically the student learning outcomes, the main mechanism that many people usually have been referred to, even up to now, is through national examination. But is it reasonable if national examination will be used for proving the student learning outcomes?
While analysis of examination results can provide insights into student achievement and can identify schools in which student performance is weak, its value is limited. One limitation is that public examinations usually test only narrow areas of a curriculum. This is partly because a limited number of subjects can be examined; it is also because within these subjects, the focus of the examination tends to be on curriculum content and competencies that will maximize discrimination between students who will be selected for further education and those who will not. The achievements of lower-performing students as a result may not be adequately represented (Kellaghan and Greaney, 2004, p. ?).
It is agreed with the statement of Kellghan and Greaney and is believed that examination is not the right way for assessing the student learning outcomes. Additionally, Bethell stated that: "The prime purpose of an examination system is to provide each candidate with a result which accurately reflects her/his level of achievement. The use of the result, e.g. for university selection or as an employment qualification, is aimed at the individual. In contrast, the purpose of a sample-based national assessment is to provide reliable data on the effectiveness of the system as a whole ( 2003, p. ?)." Moreover, Hernes, Director of International Institute for Educational Planning (IIEP), noted that "… Assessment has become over the years an important key to the improvement of the quality of education. It is one of the most reliable ways of identifying problems, whether these are the system level, of school level, or concern the individual student" (UNESCO, IIEP 2001, p. ?).
The concepts of Education Assessment have been applied in many countries/ organizations, although they have different purposes, frameworks and approaches, such as Southern and Eastern Africa Consortium for Monitoring Educational Quality (SAQMEC), Trends in International Mathematics and Science Study (TIMSS), Programme for International Student Assessment (PISA) and so forth.
TIMSS, for example, is an international assessment of the mathematics and science knowledge of fourth and eighth-grade students around the world. TIMSS is conducted every four year. The main goal of TIMMS is to provide comparative information about educational mathematical achievement across countries to improve teaching and learning in mathematics and science (TIMSS International Report, 2007). The TIMSS results then provide comparative perspectives on trends in academic achieving in the context of different educational systems, school organizational approaches, and instructional practices.
PISA, on the other hand, is a collaborative effort undertaken by all member countries of the Organization for Economic Co-operation and Development, (OECD) and a number of non-member partner countries to measure how well students, at age 15, are prepared to meet the challenges they may encounter in future life. The PISA assessment takes a broad approach to assessing knowledge, skills and attitudes that reflect current changes in curricula, moving beyond the school based approach towards the use of knowledge in everyday tasks and challenges. The skills acquired reflect the ability of students to continue learning throughout their lives by applying what they learn in school to non-school environments, evaluating their choices and making decisions. The assessment, jointly guided by the participating governments, brings together the policy interests of countries by applying scientific expertise at both national and international levels (PISA, 2006).
In particular the continent of Africa, the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) is an international non-profit developmental organization of 15 Ministries of Education in Southern and Eastern Africa. The country members decided to work together to share experiences and expertise in developing the capacities of education planners to apply scientific methods to monitor and evaluate the conditions of schooling and the quality of education, with technical assistance from UNESCO International Institute for Educational Planning (IIEP). The results of SACMEQ have been used extensively by various institutions/stakeholders, Ministries of Education (MOE's), international/bilateral organizations, universities, and individual educational planners and researchers, within the participating countries as a resource for education sector studies and as baseline information that can be employed in policy discussions and debates about the conditions of schooling and the quality of education (SACMEQ, 2010).
The Vietnamese Ministry of Education and Training (MoET) launched a large-scale monitoring study of primary education in 2000, the first of its kind in the country. The study tested pupils and their teachers in the last grade of primary education (Grade 5), using a sample cross sectional survey in two key subject areas, reading comprehension in Vietnamese and mathematics. The study tried to find the main issues and propose policy recommendations in order to raise the student achievement, to improve the system effectiveness, to close the gap between the top and bottom through compensatory funding, to counterbalance emerging social differences, and to measure the pupil achievement through regular testing (references?).
At the present day, there is a growing awareness of similar issues in Cambodian education policy documents such as the Education for All (EFA) plan 2003-2015, Education Strategic Plan (ESP) 2006-2010 and Education Sector Support Program ESSP) 2006-2010.The questions of quality, the measures to achieve the quality goals and targets have been received more and more attention. Additionally, in the Ministry of Education Youth and Sport (MoEYS) curriculum reform outline, the minimum standards for grades 3, 6 and 9 in subjects of Khmer, Social Study and Mathematics were established (MoEYS, 2006); and recently, a new institution that considers education quality (called the Quality Assurance Department) has been established (MoEYS, 2009).
Statement of the Problem
Although the strategies to achieve the outcomes and targets of the policy were set in almost the entire Ministry policy documents, for example, "Implement the minimum standards of student achievement for grades 3, 6 and 9 nationwide, assure a shared understanding of minimum standards amongst teachers, parents and other stakeholders and follow up on the results of the tests in the nationwide school report cards (ESP 2006-2010, p.14)," The minimum standards of above grades were illustrated, the information or indicators on actual achievement in quality and outcomes of learning have not been yet presented, except some proxy indicators showing about promotion and repetition rates. The above related information can be seen in some education partner studies such as UNICEF (Prak Phalla, 2005) and World Bank based projects, Education Quality Improvement Project (Marshall, 2004), Save the Children Norway (SCN,2007) and Cambodia Education Sector Support Project (CESSP, 2006, 2008, 2009). In the Education Strategic Plan (ESP) 2006-2010, the Sector Performance Milestones and Targets tables shows the targets for almost all indicators but there are blanks for indicators of percentage of students meeting curriculum standards in Khmer and Math based on standardized test at Grade 3, 6 and 9 (Appendix A, Table II: Quality and Efficiency, ESP 2006-2010). Furthermore, if the question about how well the students at grade 3 or 6 or 9 can achieve compared with the curriculum standards the answer will not be easy to find.
There is very little evidence in Cambodia related to test implementation. One of them is a report from implementation of a project called Education Quality Improvement Project (EQIP) which was supported by the World Bank from 1999 to 2003. One of the EQIP tasks was to test all Grade 4 students of the project coverage provinces (Kandal, Kampot and Takeo). Specifically, the main purpose of the test was to observe the impact from the implementation of project grants on various activities by the schools, i.e. to compare the student achievements in Numeracy and Literacy at the beginning of the project and end of the project period (EQIP Final Report, 2004).
In 2005, MoEYS/Pedagogical Research Department (PRD) with assistance from UNICEF started implementing a project on Grade 6 Learning Achievement by conducting the test on mathematics and Khmer. The objectives of the project were to provide information on students' learning achievement in the state school including the Child Friendly Schools (CFS) to MoEYS, UNICEF and relevant stakeholders; to establish a baseline level for Grade 6 learning achievement for the new curriculum and standard; to compare the student results between CFS and non-CFS schools; to collect data for use for a revision of the Grade 6 curriculum, textbooks and teachers' manuals and supplementary teaching/learning materials; and to find out the real situation of learning achievement in primary schools.
In his report, Prak (UNICEF, 2005) recommended that "more standard items should be put in. This is because in this test not all Grade 6 curriculum standards on Khmer and Math have been tested owing to a time and budget constraint (p.?)".
Purpose and Significance
The curriculum standards (minimum standards) for Grade 3, 6 and 9 had been established since 2006 (Appendix B) but until now there is no document or report describing how the students perform compared with the standards. This paper, therefore, attempts to illustrate the results of one of the above grade level (Grade 9), then the research findings will not only be used for fulfilling the missing data in the ESP document for the defined year but also to demonstrate how well the Grade 9 student can achieve comparing with the official curriculum standard. More importantly, the findings will explain why the students are possibly very good or poor in some content areas of the curriculum, for example, what possible reasons cause them to be good in solving problem by using graph or, contrastingly, they are poor in finding length of arc
Besides this, as we live in the global context and as the education systems are gradually developed we need to illustrate that kind of results to compare with other countries in the regions or/and well-known organizations such as TIMMS, PISA, etc.
Research Question
More specifically, this document tries to answer these research questions:
What is the overall average level of student learning achievement in Mathematics in sample schools in comparison to the national standard?
How do the averages in Mathematics vary by content and sub-content area?
Are there significant differences in the learning achievement by location, gender and socioeconomics by quintile?
Research Methods
With the support from the Cambodian Education Sector Support Project (CESSP)/ MoEYS, the National Assessment unit has conducted several tests in different grade levels, three, six and nine, since 2005. The overarching purpose of the MoEYS assessment work is to provide a system-wide diagnosis of school quality and student learning. This is done using standardized tests that measure the official, or intended, curriculum based on items created by MoEYS personnel. Since I am one of the members of this unit I have discussed with the team members to design some particular tasks for this study purpose along with the main tasks of the project.
Sampling Technique and Sample Size
This is a model of diagnostic assessment which requires only representative sample of schools to represent the country as the whole. With a full support from CESSP/World Bank a range of sampling technique and sample size was produced. The type of sample was defined as two-stage cluster sample. This was done in five steps: 1) defining the population: the grade nine population is restricted to lower secondary schools, EMIS data 2008, that have at least one grade nine class with 20 or more students; 2) deciding how many schools are required: 200 schools were selected by using sampling software produced by IIEP (called IIEPSAMP) with the Effective Sample Size (ESS) of 400 (Appendix C); 3) defining the strata: the strata here referred to the three main regions in the country: Urban, Rural and Remote; 4) selecting the schools: the software then produced 50 schools in urban, 139 in rural and 11 schools in remote areas (Appendix D); and, 5) selecting the students within each school: 30 students per school were randomly selected so the total of students were approximately 6,000. Limitations of the method adopted.
Research Tools
The scope of this study is actually seen as a very big study however, as above mentioned, the preparation for this study was carried out alongside with the MoEYS tasks which being involved by related ministry departments/staff therefore some of the related results/findings can be considered and used as the findings of this study.
With the purpose of comparing the level of student achievement in math the curriculum test blueprint (Appendix E), test booklets and official curriculum standards are essentially needed. In addition, in order to discover the implication to the student achievement the information from student and teacher interview is also required.
Data Analysis
The items produced by the MoEYS specialists covered the content and sub-content areas of the intended curriculum in general (Appendix E). For this study purpose the items related to curriculum standards will be defined.
Descriptive Statistics: This will summary the information especially the student achievement in average percentage correct (mean) and standard deviations are also used when appropriate.
Comparisons of Means: Since the sample is divided into three types of school locations (regions), male and female students participated and the students came from different socio-economic status t-test, ANOVA or/and chi-square could be used for estimating the differences among those variables.
By doing this we can show the differences existed in the content and sub-content areas, subsequently, we will explore for what areas have the highest relative scores, what is the lowest, etc. Then the results that relate to the curriculum standards can be explored, and therefore we can conclude about how the Grade 9 students achieved when they finished their class. The gaps between students like urban-rural, poor-rich, boys-girls, etc will also be demonstrated. More importantly, from results in numbers we will try to explore what do results mean in terms of what students can actually do and why they performed poorly in some particular content areas of the curriculum. The results, then, will become the first findings in such field and may probably essential for some stakeholders.
Appendix
Appendix A:
Table II: Quality and Efficiency (Promotion and repetition 2003-04 rate are used for baseline 2004-05)
Indicator
Baseline
Target
Target
Target
Target
Target
Source:
2004-05
2005-06
2006-07
2007-08
2008-09
2009-10
Pupil teacher ratio
EMIS
- Primary
53.5
51.0
50.0
50.0
50.0
50.0
- Lower Secondary
27.7
37
41
45
45
45
- Upper Secondary
29.4
35
38
40
40
40
Promotion rate
Total
Female
Male
Total
Female
Male
Total
Female
Total
Female
Total
Female
Total
Female
EMIS
- Grade 1
64.8%
65.4%
64.2%
92%
92%
92%
93%
93%
94%
94%
95%
95%
95%
95%
- Grade 3
77.3%
78.9%
75.7%
93%
93%
93%
93%
93%
94%
94%
95%
95%
95%
95%
- Grade 6
86.6%
86.3%
86.9%
93%
93%
93%
93%
93%
94%
94%
95%
95%
95%
95%
Repetition rate
Total
Female
Male
Total
Female
Male
Total
Female
Total
Female
Total
Female
Total
Female
EMIS
- Grade 1
23.6%
22.4%
24.9%
6%
6%
6%
5%
5%
4%
4%
4%
4%
4%
4%
- Grade 3
13.2%
11.3%
15.4%
6%
6%
6%
5%
5%
4%
4%
4%
4%
4%
4%
- Grade 6
2.6%
2.2%
3.1%
2%
2%
2%
2%
2%
2%
2%
1%
1%
1%
1%
Completion rate
Total
Female
Male
Total
Female
Male
Total
Female
Total
Female
Total
Female
Total
Female
EMIS
- Primary
46.8%
45.7%
47.9%
60%
60%
60%
70%
70%
80%
80%
90%
90%
100%
100%
- Lower secondary
20.6%
20.0%
21.2%
30%
30%
30%
40%
40%
50%
50%
60%
60%
75%
75%
Literacy rate
67.1%
60.3%
74.7%
70%
67%
73.1%
75%
73%
80%
78%
85%
84%
90%
90%
Number of students passing grade 12 examination
33,834
37,500
40,000
50,000
60,000
70,000
EMIS
% of students meeting curriculum standards in Khmer Language based on standardized test at:
PRD/DGE
- Grade 3
to be determined
to be determined
to be determined
to be determined
to be determined
- Grade 6
to be determined
to be determined
to be determined
to be determined
to be determined
- Grade 9
to be determined
to be determined
to be determined
to be determined
to be determined
% of students meeting curriculum standards in Mathematics based on standardized test at:
PRD/DGE
- Grade 3
to be determined
to be determined
to be determined
to be determined
to be determined
- Grade 6
to be determined
to be determined
to be determined
to be determined
to be determined
- Grade 9
to be determined
to be determined
to be determined
to be determined
to be determined
Appendix B: Grade 6 Curriculum Standard
STRAND
Grade 6
NUMBER
Read, count, write, order and compare whole numbers not exceeding 7 digits and numbers with decimal fractions to two decimal places
Read, write, order and compare fractions and mixed numbers.
Round decimal numbers to the nearest whole number.
Add, subtract, multiply and divide whole numbers using three types of brackets ( [, (, { )
Add and subtract fractions with the same denominator.
Add and subtract decimal numbers with two decimal places.
Multiply and divide numbers up to 4 digits by 2 digits.
Use estimation strategies to check multiplication, addition and division of whole numbers.
Rename common fractions (less than one) as decimals and percentages
Calculate simple ratios and direct proportions (eg 2 people need 4 cups of water so 6 people need 12).
Calculate average costs, profit and loss, and write and verify receipts.
MEASUREMENT
Use standard measuring instruments and read scales to the nearest gradation to determine:
Length from kilometres to millimetres (eg. wood, roads)
Capacity from litres to millilitres (eg. medicine, cooking oil)
Weight (mass) from kilograms to grams (eg vegetable, rice and meat)
Time from hours to seconds
Read and express accurately time in analogue, digital, 12 and 24 hours representations
Interpret a simple scale bar on a map and use the map to calculate distance between places
Calculate average travel times using given speeds and distances
GEOMETRY
Name types of angles (acute, right, obtuse and straight line) up to 180° and construct angles using rulers, protractors and compasses
Draw and label circles using the following terms: radius, centre, diameter and circumference
Make models of prisms, cones, pyramids, cylinders and spheres
Measure and find the perimeter and area of triangles, squares, and rectangle
Find the volume of solids made up of unit cubes.
STATISTICS
Construct and interpret data presented in tables, line graphs, bar chart and pie charts.
ALGEBRA
AND
PATTERNS
Find the value of simple algebraic expressions using substitution methods involving addition and subtraction. (eg. 3 + b =?, 8- b= ? where b is 4)
Simplify simple algebraic expressions using addition and subtraction methods. (eg. 4x + 2x = ?)
REASONING
Explain a short chain of reasoning used to approach and solve a problem that involves an analysis of data through the selection and use of mathematical techniques.
Appendix C. Sample Design Table
MCS:
Intraclass Correlation (ICC):
0.20
0.30
0.40
0.50
0.60
20 Schools
150
159
197
235
273
Students
3,000
3,180
3,940
4,700
5,460
25 Schools
150
151
190
228
266
Students
3,750
3,775
4,750
5,700
6,650
30 Schools
150
150
185
223
262
Students
4,500
4,500
5,550
6,690
7,860
35 Schools
150
150
181
220
259
Students
5,250
5,250
6,335
7,700
9,065
40 Schools
150
150
179
218
257
Students
6,000
6,000
7,160
8,720
10,280
Source: TIMSS 2004 Report, Chapter 5 (Table 5.2).
Notes: For this study a Minimum Cluster Size (MCS) of 30 was used, although in a small percentage of schools there were fewer than 30 grade nine students enrolled. Calculations of the ICC from previous student assessments in Cambodia suggest an average of roughly 0.30. The MoEYS grade nine sample of 200 schools and roughly 6,000 students is larger than the minimum sample predicted for an MCS of 30 and ICC of 0.30.
Appendix D: Comparison of School Strata in Population and Samples
Strata:
Schools in Population:
Number of Schools in Sample:
Raw Total
Percent*
(%)
Raw Total
Percent*
(%)
Weighted Percent (%)
1. Urban
140
28.0
50
25.0
28.0
2. Rural
776
70.7
139
69.9
70.8
3. Remote
28
1.3
11
5.0
1.2
Total:
944
100.0
200
100.0
100.0
Source: EMIS and MoEYS Assessment, 2007-2009
*Refers to percentage of total student population, not percentage of schools. The numbers in
the far right column (in bold) refer to the weighted sample that is used for the analysis.
Appendix E: Grade 9 Curriculum Blueprint and Item Analysis Summaries
No
Content Area
Sub-Content Area
Cognitive Skill
Item
%
Knowledge
Understanding
Application
Analyzing
1
Number
Operation in Integer
1
1
0
0
20
20%
Operations in Power
1
2
0
0
Taking number out of a root
1
2
0
0
Taking number into the root
1
2
0
0
Operations on a root
1
2
2
1
Finding a percentage of a number
1
1
1
0
2
Algebra
Linear equation
2
4
1
1
50
50%
A system of equations in two variables
2
3
1
1
Inequality and inequality systems
2
3
1
0
Quadratic equation
2
4
1
0
Distance between two points in Orthonormial grid
2
3
1
0
Equation of line
3
4
2
0
Solving a system equations and inequality by using graph
2
4
1
0
3
Statistics
Probability
1
2
1
1
10
10%
Representative statistics
1
1
0
0
Analysis statistics
1
2
0
0
4
Geometry
Thales theory
2
3
1
1
20
20%
Similar triangles
2
3
1
0
Trigonometry
1
2
1
0
Finding the length of arc and angle of circle
1
2
0
0
