RAMP midline survey findings
أَقرأ ُ الكتاب وأَفه ُم الحساب
ُمبادرة القراءة والحساب للصفوف المبكرة
Jordan RAMP initiative midline survey Aarnout Brombacher, Senior Education Program Specialist, RTI International, [email protected]
Background to RAMP
2012 2013/2014 2014/2015
National Survey (156 schools) Intervention Pilot & National Survey (42 treatment and 110 control schools) Remedial Pilot
2015 – 2019 Early Grade Reading and Mathematics Initiative (RAMP)
2
Background to RAMP
• A national EGRA/EGMA survey was conducted at the end of the 2013–2014 academic year (in May 2014) to measure the impact of the Intervention Pilot Research Activity. • Since RAMP is regarded, fundamentally, an extension/expansion of that activity, the MoE and USAID, felt that conducting a national baseline study for RAMP at the start of the initiative in 2015 was not necessary and that instead the performance of the students in the control group of the 2014 National Survey would be used to establish the baseline values for RAMP. This decision together with the cohort based approach to implementation has created severe limitations on the conclusions that can be drawn with regard to the impact of RAMP 18 months after the start of the in-service training.
Background to RAMP • The 2012 & 2014 National Literacy and Numeracy Surveys clearly demonstrated that the majority of Jordanian children in the early grades are not reading with comprehension or doing mathematics with understanding (application and reasoning).
Reading
18% Mathematics
14% 4
Background to RAMP: Reading • In the case of reading, the surveys found that children were not developing key foundational literacy skills largely because teachers had not been specifically trained in teaching early grade reading using an approach that develops phonemic awareness and provides deliberate instruction focused on phonics, fluency, vocabulary, and comprehension. • Early grade reading instruction in Jordan also does not take into account the important role that reading aloud to children (both at school and at home) plays.
5
Background to RAMP: Reading (2012 Survey) EGRA zero scores by grade 100% 90% 80% Percent of Students
70%
Grade 2
60%
Grade 3 49%
50%
45%
40% 28%
30% 20%
27%
21%
21% 20%
22% 15% 8%
10% 0% Letter Sounds
Invented words
Oral Reading Fluency
Reading Comprehension
Listening Comprehension 6
Background to RAMP: mathematics • In terms of mathematics, children’s underperformance on the surveys is explained by the fact that children memorize mathematical facts, rules, and formulas without understanding. • In general, early grade mathematics teachers also have low selfconfidence with respect to doing mathematics and lack skills in teaching early grade mathematics with a focus on understanding, application, and reasoning.
7
Background to RAMP: mathematics Percentage of zero scores by grade Background toEGMA: RAMP: Mathematics (2012 Survey) 100%
90%
Percentage of students
80% 70% 60% 50% 40%
30% 20% 10% 0% Num ID
Grade 2 Grade 3
1% 1%
Quant Disc 4% 3%
Miss Num 6% 5%
Add L1
Add L2
Subt L1
Subt L2
13% 11%
24% 18%
18% 15%
48% 36%
Word Prob 34% 22% 8
Background to RAMP: curriculum • The 2012 classroom observations revealed that daily lesson content was informed by the page in the textbook for the day. Teachers were teaching according to a schedule that determined what would be taught on each day with little regard for whether or not the children were developmentally ready for the lesson content. • An analysis of the curriculum in use in 2012, revealed that: – Despite grade 3 children not performing well on the letter-sounds reading task, this skill was not being addressed in the curriculum after grade 1. – In mathematics, the curriculum in grade 2 required students to add and subtract three and four-digit numbers and yet one-half of the grade 2 students in the survey could not correctly subtract 3 from 19.
• The 2012 National Survey revealed that students were not getting sufficient instruction in foundational reading and mathematics skills—in foundational skills that research indicates are predictive of future success in reading and mathematics—with little hope of having this insufficiency addressed by their teachers or the curriculum that was in use in 2012. 9
RAMP • The Early Grade Reading and Mathematics Initiative (RAMP) is an activity of the Jordanian MoE • RAMP will apply the methodologies researched in the intervention pilot, the remedial pilot and other activities of the MoE (CISLE, ERSP) on a national scale • The 5-year goal of RAMP is that:
By 2019, the majority of early grade students in Jordanian public schools will be reading with comprehension and doing mathematics with understanding
10
Background to RAMP •
RAMP is a five year nationwide initiative of the Ministry of Education designed to improve the reading and mathematics skills of students in Kindergarten 2 through G3.
• The initiative: 1. Has developed and distributed improved learning materials to every K2–G3 classroom in Jordan; 2. Is training teachers, principals, supervisors, and field directorate and MoE administrators to provide more effective research-based reading and mathematics instruction; 3. Promotes community participation in reading and mathematics education; and 4. Supports nationwide adoption of early grade reading and mathematics policies, standards, curricula, and assessments. 11
Background to RAMP: reading • The Simple View of Reading (Gough and Tunmer, 1986): Reading comprehension
=
Decoding
×
Language comprehension
•
Decoding = fast and accurate reading of familiar and unfamiliar words in both lists and connected text
•
Language comprehension = the ability to derive meaning from spoken words when they are part of sentences or other discourse
•
Reading comprehension = the ability to derive meaning from written words
It is possible to have strong language comprehension and still be a poor reader if there is difficulty with 12 decoding.
RAMP implementation
Cohort
Schools KG2 – Grade 2
Cohort 1
Teachers 2,651
623 (25%) Grade 3
Grade 3
(48%)
171,531 (51%)
(25%)
71,388 (21%)
1,942
KG2 – Grade 2
2,458 749 (30%)
Grade 3
90,349 (27%)
4,509
1,087 (44%)
Cohort 3
(27%) 1,067
KG2 – Grade 2
Cohort 2
Students
960
RAMP midline survey • To measure both the impact of the RAMP Initiative so far and the progress toward the RAMP indicator targets a midline study was conducted at the end of the 2016–2017 academic year in May 2017. • Instruments – Reading: Early Grade Reading Assessment (EGRA) – Mathematics: Early Grade Mathematics Assessment (EGMA)
14
Total
101 100 121 100 107 106 114 107 90 109 105 100 1,260
106 99 98 106 79 117 103 104 93 126 84 130 85 106 63 126 103 82 90 121 102 106 104 105 1,110 1,328
2,370
94 95 82 91 74 80 94 103 122 80 83 73 1,071
2,399
Schools
Teachers
Syrian
Total
Male
Ajloun Jerash Zarqa Karak Irbid Amman Tafilah Aqaba Mafraq Balqa Madaba Ma’an
Female
Governorate
North North Central South North Central South South North Central Central South
Male
Region
1 1 1 1 2 2 2 2 3 3 3 3
Students G3
G2
Female
Cohort
RAMP midline survey - sample
400 399 399 398 400 400 399 399 397 400 396 382
27 29 40 14 35 61 3 8 80 6 29 8
40 39 39 40 40 39 40 37 38 40 40 40
20 20 20 20 20 20 20 20 20 20 20 20
4,769
340
472
240
1 1 1 1 2 2 2 2 3 3 3 3
North Ajloun 101 106 99 94 400 27 40 North Jerash 100 98 106 95 399 29 39 The 2017 sample students for Central Zarqa 121of 4,800 79 117 82 allows 399 40 the 39 South Karakof point 100 estimates 103 104 at national, 91 398 grade, 14 40 calculation North Irbid 107 93 126 74 400 35 40 gender and governorate level. Central Amman 106 84 130 80 400 61 39 South Tafilah 114 85 106 94 399 3 40 South Aqaba The 2014 sample students for 107of 2,153 63 126 103 only 399 allows 8 37 North Mafraq 103estimates 82 122 397 80 grade 38 the calculation of90point at national, Central Balqa 109 90 121 80 400 6 40 and gender level. Central Madaba 105 102 106 83 396 29 40 South Ma’an 100 104 105 73 382 8 40 1,260 1,110 1,328 1,071 4,769 340 472 Total 2,370 2,399
Schools
Teachers
Syrian
Total
Male
Male
Female
Students G3
G2
Female
Governorate
Region
Cohort
RAMP midline survey - sample
20 20 20 20 20 20 20 20 20 20 20 20 240
RAMP midline survey – sample
Grade 2 students
Grade 3 students17
RAMP midline survey – sample
79% of the Grade 2 students in the midline survey have been exposed to RAMP methodologies for between 2 and 3 semesters 27% of the Grade 3 students in the midline survey have been exposed to RAMP methodologies for at most 3 semesters Grade 2 students
Grade 3 students18
أَقرأ ُ الكتاب وأَفه ُم الحساب
ُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: findings
On balance, the RAMP midline survey provides strong evidence that the RAMP initiative is making a positive impact, which if supported and sustained will, in the years to come, help Jordan achieve the early grade reading and mathematics outcomes that are critical for nation building.
20
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct syllables per min.) Syllable sound
✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
fluency (correct syllables per min.) Syllable sound
Invented words
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
26.6
26.4
% correct of items attempted
63.7%
60.4%
% of students with zero scores
9.1%
10.4%
fluency (correct syllables per min.) Syllable sound
Invented words
oral reading fluency (ORF) Oral reading
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
26.6
26.4
% correct of items attempted
63.7%
60.4%
% of students with zero scores
9.1%
10.4%
% correct of items attempted
45.3%
49.8%
% correct Reading comprehension % of students with 80% comp.
33.8%
36.9%
17.9%
21.3%
% of students with zero scores
34.1%
30.5%
Letter sound
fluency (correct syllables per min.) Syllable sound
Invented words
oral reading fluency (ORF) Oral reading
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
oral reading fluency (ORF)
26.6
26.4
% correct Reading comprehension % of students with 80% comp.
33.8%
36.9%
17.9%
21.3%
% of students with zero scores
34.1%
30.5%
✓ Letter sound % correct of items attempted 64.9% 76.1% *** ✓ % of students with zero scores 21.2% 7.3% *** ✓ fluency (correct syllables per min.) 25.2 31.7 *** ✓ Syllable sound % correct of items attempted 63.8% 72.5% *** ✓ % of students with zero scores 10.8% 4.8% *** ✓ fluency (correct words per min.) 8.9 12.2 *** ✓ Invented words % correct of items attempted 39.4% 49.3% *** ✓ % of students with zero scores 30.7% 15.1% *** ✓in There is a statistically significant improvement
performance on the foundational skills 60.4% for % correct of items attempted 63.7% reading. Reading with comprehension is a % of students with zero scores 9.1% 10.4% function ofof performance on foundational skills.✓ % correct items attempted 45.3% 49.8%
Oral reading
✓ ✓ ✓
RAMP midline survey findings: reading
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Letter sound
fluency
38.2
47.3
35.7
48.6
Syllable sound
fluency
22.1
29.8
28.7
33.8
fluency
7.1
10.8
10.9
13.7
34.5%
48.2%
44.9%
50.4%
19.1
21.4
35.0
31.7
56.7%
56.2%
71.6%
64.7
7.9%
11.4%
29.0%
31.5%
Invented words % correct
oral reading fluency (ORF) Oral reading % correct Reading comprehension
% of students with 80% comp.
RAMP midline survey findings: reading
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Letter sound
fluency
38.2
47.3
35.7
48.6
Syllable sound
fluency
22.1
29.8
28.7
33.8
fluency
7.1
10.8
10.9
13.7
34.5%
48.2%
44.9%
50.4%
19.1
21.4
35.0
31.7
56.7%
56.2%
71.6%
64.7
Invented words % correct
oral reading fluency (ORF) Oral reading % correct
On the foundational reading skills the 2017 % of students with 80% 7.9% 11.4% 29.0% Gradecomp. 2 students are performing at the same level that the Grade 3 students did in 2014.
Reading comprehension
31.5%
RAMP midline survey findings: mathematics Subtask Number Identification
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
✓ ✓
RAMP midline survey findings: mathematics Subtask Number Identification Quantity Comparison
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
% correct
78.9%
83.7% ***
✓ ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
Number Identification
✓ ✓ ✓ = = =
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
Number Identification
Addition and Subtraction L2
✓ ✓ ✓ = = = ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
% correct
60.3%
64.5% *
% of students with zero scores
3.1%
1.7% *
Number Identification
Addition and Subtraction L2
Missing Number
✓ ✓ ✓ = = = ✓ ✓ ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
% correct
60.3%
64.5% *
% of students with zero scores
3.1%
1.7% *
Number Identification
Addition and Subtraction L2
Missing Number
✓ ✓ ✓ = = = ✓ ✓ ✓ ✓
There is a statistically significant improvement in performance on the more conceptual mathematics tasks i.e. doing mathematics with understanding.
RAMP midline survey findings: mathematics
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Addition and Subtraction L1
% correct
82.6%
85.0%
85.5%
87.0%
Addition and Subtraction L2
% correct
36.8%
42.5%
47.5%
51.2%
Missing Number
% correct
54.3%
60.2%
66.9%
68.9%
RAMP midline survey findings: mathematics
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Addition and Subtraction L1
% correct
82.6%
85.0%
85.5%
87.0%
Addition and Subtraction L2
% correct
36.8%
42.5%
47.5%
51.2%
Missing Number
% correct
54.3%
60.2%
66.9%
68.9%
On the conceptual mathematics tasks the 2017 Grade 2 students are approaching the performance levels of the 2014 Grade 3 students.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
RAMP indicators Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do grade-level mathematics with understanding.
أَقرأ ُ الكتاب وأَفه ُم الحساب
ُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: discussion
RAMP midline survey: discussion • For reading there is a statistically significant improvement in performance on all of the foundational skills for reading. This is encouraging as reading with comprehension is a function of performance on foundational skills. – For comprehension there is a statistically significant
narrowing of the gender performance gap! • For mathematics there is a statistically significant improvement in performance on the more conceptual mathematics tasks i.e. doing mathematics with understanding.
44
RAMP midline survey: discussion • On the foundational reading skills the 2017 Grade 2 students are performing at the same level that the Grade 3 students did in 2014. • On the conceptual mathematics tasks the 2017 Grade 2 students are approaching the performance levels of the 2014 Grade 3 students.
45
RAMP midline survey: discussion • The impact of RAMP is not yet being felt at a statistically significant level in the Grade 3 performance.
46
RAMP midline survey: discussion • The RAMP midline survey has generated Governorate level performance data for both EGRA and EGMA. This is the first time that this level of data exists in Jordan. • This data provides the MoE with the data needed to make decisions about the effective deployment of scarce resources.
47
On balance, the RAMP midline survey provides powerful evidence that the RAMP initiative is making a positive impact, which if supported and sustained will, in the years to come, help Jordan achieve the early grade reading and mathematics outcomes that are critical for nation building.
48
49
ُمبادرة القراءة والحساب للصفوف المبكرة
Jordan RAMP initiative midline survey Aarnout Brombacher, Senior Education Program Specialist, RTI International, [email protected]
Background to RAMP
2012 2013/2014 2014/2015
National Survey (156 schools) Intervention Pilot & National Survey (42 treatment and 110 control schools) Remedial Pilot
2015 – 2019 Early Grade Reading and Mathematics Initiative (RAMP)
2
Background to RAMP
• A national EGRA/EGMA survey was conducted at the end of the 2013–2014 academic year (in May 2014) to measure the impact of the Intervention Pilot Research Activity. • Since RAMP is regarded, fundamentally, an extension/expansion of that activity, the MoE and USAID, felt that conducting a national baseline study for RAMP at the start of the initiative in 2015 was not necessary and that instead the performance of the students in the control group of the 2014 National Survey would be used to establish the baseline values for RAMP. This decision together with the cohort based approach to implementation has created severe limitations on the conclusions that can be drawn with regard to the impact of RAMP 18 months after the start of the in-service training.
Background to RAMP • The 2012 & 2014 National Literacy and Numeracy Surveys clearly demonstrated that the majority of Jordanian children in the early grades are not reading with comprehension or doing mathematics with understanding (application and reasoning).
Reading
18% Mathematics
14% 4
Background to RAMP: Reading • In the case of reading, the surveys found that children were not developing key foundational literacy skills largely because teachers had not been specifically trained in teaching early grade reading using an approach that develops phonemic awareness and provides deliberate instruction focused on phonics, fluency, vocabulary, and comprehension. • Early grade reading instruction in Jordan also does not take into account the important role that reading aloud to children (both at school and at home) plays.
5
Background to RAMP: Reading (2012 Survey) EGRA zero scores by grade 100% 90% 80% Percent of Students
70%
Grade 2
60%
Grade 3 49%
50%
45%
40% 28%
30% 20%
27%
21%
21% 20%
22% 15% 8%
10% 0% Letter Sounds
Invented words
Oral Reading Fluency
Reading Comprehension
Listening Comprehension 6
Background to RAMP: mathematics • In terms of mathematics, children’s underperformance on the surveys is explained by the fact that children memorize mathematical facts, rules, and formulas without understanding. • In general, early grade mathematics teachers also have low selfconfidence with respect to doing mathematics and lack skills in teaching early grade mathematics with a focus on understanding, application, and reasoning.
7
Background to RAMP: mathematics Percentage of zero scores by grade Background toEGMA: RAMP: Mathematics (2012 Survey) 100%
90%
Percentage of students
80% 70% 60% 50% 40%
30% 20% 10% 0% Num ID
Grade 2 Grade 3
1% 1%
Quant Disc 4% 3%
Miss Num 6% 5%
Add L1
Add L2
Subt L1
Subt L2
13% 11%
24% 18%
18% 15%
48% 36%
Word Prob 34% 22% 8
Background to RAMP: curriculum • The 2012 classroom observations revealed that daily lesson content was informed by the page in the textbook for the day. Teachers were teaching according to a schedule that determined what would be taught on each day with little regard for whether or not the children were developmentally ready for the lesson content. • An analysis of the curriculum in use in 2012, revealed that: – Despite grade 3 children not performing well on the letter-sounds reading task, this skill was not being addressed in the curriculum after grade 1. – In mathematics, the curriculum in grade 2 required students to add and subtract three and four-digit numbers and yet one-half of the grade 2 students in the survey could not correctly subtract 3 from 19.
• The 2012 National Survey revealed that students were not getting sufficient instruction in foundational reading and mathematics skills—in foundational skills that research indicates are predictive of future success in reading and mathematics—with little hope of having this insufficiency addressed by their teachers or the curriculum that was in use in 2012. 9
RAMP • The Early Grade Reading and Mathematics Initiative (RAMP) is an activity of the Jordanian MoE • RAMP will apply the methodologies researched in the intervention pilot, the remedial pilot and other activities of the MoE (CISLE, ERSP) on a national scale • The 5-year goal of RAMP is that:
By 2019, the majority of early grade students in Jordanian public schools will be reading with comprehension and doing mathematics with understanding
10
Background to RAMP •
RAMP is a five year nationwide initiative of the Ministry of Education designed to improve the reading and mathematics skills of students in Kindergarten 2 through G3.
• The initiative: 1. Has developed and distributed improved learning materials to every K2–G3 classroom in Jordan; 2. Is training teachers, principals, supervisors, and field directorate and MoE administrators to provide more effective research-based reading and mathematics instruction; 3. Promotes community participation in reading and mathematics education; and 4. Supports nationwide adoption of early grade reading and mathematics policies, standards, curricula, and assessments. 11
Background to RAMP: reading • The Simple View of Reading (Gough and Tunmer, 1986): Reading comprehension
=
Decoding
×
Language comprehension
•
Decoding = fast and accurate reading of familiar and unfamiliar words in both lists and connected text
•
Language comprehension = the ability to derive meaning from spoken words when they are part of sentences or other discourse
•
Reading comprehension = the ability to derive meaning from written words
It is possible to have strong language comprehension and still be a poor reader if there is difficulty with 12 decoding.
RAMP implementation
Cohort
Schools KG2 – Grade 2
Cohort 1
Teachers 2,651
623 (25%) Grade 3
Grade 3
(48%)
171,531 (51%)
(25%)
71,388 (21%)
1,942
KG2 – Grade 2
2,458 749 (30%)
Grade 3
90,349 (27%)
4,509
1,087 (44%)
Cohort 3
(27%) 1,067
KG2 – Grade 2
Cohort 2
Students
960
RAMP midline survey • To measure both the impact of the RAMP Initiative so far and the progress toward the RAMP indicator targets a midline study was conducted at the end of the 2016–2017 academic year in May 2017. • Instruments – Reading: Early Grade Reading Assessment (EGRA) – Mathematics: Early Grade Mathematics Assessment (EGMA)
14
Total
101 100 121 100 107 106 114 107 90 109 105 100 1,260
106 99 98 106 79 117 103 104 93 126 84 130 85 106 63 126 103 82 90 121 102 106 104 105 1,110 1,328
2,370
94 95 82 91 74 80 94 103 122 80 83 73 1,071
2,399
Schools
Teachers
Syrian
Total
Male
Ajloun Jerash Zarqa Karak Irbid Amman Tafilah Aqaba Mafraq Balqa Madaba Ma’an
Female
Governorate
North North Central South North Central South South North Central Central South
Male
Region
1 1 1 1 2 2 2 2 3 3 3 3
Students G3
G2
Female
Cohort
RAMP midline survey - sample
400 399 399 398 400 400 399 399 397 400 396 382
27 29 40 14 35 61 3 8 80 6 29 8
40 39 39 40 40 39 40 37 38 40 40 40
20 20 20 20 20 20 20 20 20 20 20 20
4,769
340
472
240
1 1 1 1 2 2 2 2 3 3 3 3
North Ajloun 101 106 99 94 400 27 40 North Jerash 100 98 106 95 399 29 39 The 2017 sample students for Central Zarqa 121of 4,800 79 117 82 allows 399 40 the 39 South Karakof point 100 estimates 103 104 at national, 91 398 grade, 14 40 calculation North Irbid 107 93 126 74 400 35 40 gender and governorate level. Central Amman 106 84 130 80 400 61 39 South Tafilah 114 85 106 94 399 3 40 South Aqaba The 2014 sample students for 107of 2,153 63 126 103 only 399 allows 8 37 North Mafraq 103estimates 82 122 397 80 grade 38 the calculation of90point at national, Central Balqa 109 90 121 80 400 6 40 and gender level. Central Madaba 105 102 106 83 396 29 40 South Ma’an 100 104 105 73 382 8 40 1,260 1,110 1,328 1,071 4,769 340 472 Total 2,370 2,399
Schools
Teachers
Syrian
Total
Male
Male
Female
Students G3
G2
Female
Governorate
Region
Cohort
RAMP midline survey - sample
20 20 20 20 20 20 20 20 20 20 20 20 240
RAMP midline survey – sample
Grade 2 students
Grade 3 students17
RAMP midline survey – sample
79% of the Grade 2 students in the midline survey have been exposed to RAMP methodologies for between 2 and 3 semesters 27% of the Grade 3 students in the midline survey have been exposed to RAMP methodologies for at most 3 semesters Grade 2 students
Grade 3 students18
أَقرأ ُ الكتاب وأَفه ُم الحساب
ُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: findings
On balance, the RAMP midline survey provides strong evidence that the RAMP initiative is making a positive impact, which if supported and sustained will, in the years to come, help Jordan achieve the early grade reading and mathematics outcomes that are critical for nation building.
20
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct syllables per min.) Syllable sound
✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
fluency (correct syllables per min.) Syllable sound
Invented words
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask Letter sound
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
26.6
26.4
% correct of items attempted
63.7%
60.4%
% of students with zero scores
9.1%
10.4%
fluency (correct syllables per min.) Syllable sound
Invented words
oral reading fluency (ORF) Oral reading
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
% correct of items attempted
64.9%
76.1% ***
% of students with zero scores
21.2%
7.3% ***
25.2
31.7 ***
% correct of items attempted
63.8%
72.5% ***
% of students with zero scores
10.8%
4.8% ***
fluency (correct words per min.)
8.9
12.2 ***
% correct of items attempted
39.4%
49.3% ***
% of students with zero scores
30.7%
15.1% ***
26.6
26.4
% correct of items attempted
63.7%
60.4%
% of students with zero scores
9.1%
10.4%
% correct of items attempted
45.3%
49.8%
% correct Reading comprehension % of students with 80% comp.
33.8%
36.9%
17.9%
21.3%
% of students with zero scores
34.1%
30.5%
Letter sound
fluency (correct syllables per min.) Syllable sound
Invented words
oral reading fluency (ORF) Oral reading
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓
RAMP midline survey findings: reading Subtask
Measure fluency (correct letters per min.)
2014 37.0
2017 47.9 ***
oral reading fluency (ORF)
26.6
26.4
% correct Reading comprehension % of students with 80% comp.
33.8%
36.9%
17.9%
21.3%
% of students with zero scores
34.1%
30.5%
✓ Letter sound % correct of items attempted 64.9% 76.1% *** ✓ % of students with zero scores 21.2% 7.3% *** ✓ fluency (correct syllables per min.) 25.2 31.7 *** ✓ Syllable sound % correct of items attempted 63.8% 72.5% *** ✓ % of students with zero scores 10.8% 4.8% *** ✓ fluency (correct words per min.) 8.9 12.2 *** ✓ Invented words % correct of items attempted 39.4% 49.3% *** ✓ % of students with zero scores 30.7% 15.1% *** ✓in There is a statistically significant improvement
performance on the foundational skills 60.4% for % correct of items attempted 63.7% reading. Reading with comprehension is a % of students with zero scores 9.1% 10.4% function ofof performance on foundational skills.✓ % correct items attempted 45.3% 49.8%
Oral reading
✓ ✓ ✓
RAMP midline survey findings: reading
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Letter sound
fluency
38.2
47.3
35.7
48.6
Syllable sound
fluency
22.1
29.8
28.7
33.8
fluency
7.1
10.8
10.9
13.7
34.5%
48.2%
44.9%
50.4%
19.1
21.4
35.0
31.7
56.7%
56.2%
71.6%
64.7
7.9%
11.4%
29.0%
31.5%
Invented words % correct
oral reading fluency (ORF) Oral reading % correct Reading comprehension
% of students with 80% comp.
RAMP midline survey findings: reading
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Letter sound
fluency
38.2
47.3
35.7
48.6
Syllable sound
fluency
22.1
29.8
28.7
33.8
fluency
7.1
10.8
10.9
13.7
34.5%
48.2%
44.9%
50.4%
19.1
21.4
35.0
31.7
56.7%
56.2%
71.6%
64.7
Invented words % correct
oral reading fluency (ORF) Oral reading % correct
On the foundational reading skills the 2017 % of students with 80% 7.9% 11.4% 29.0% Gradecomp. 2 students are performing at the same level that the Grade 3 students did in 2014.
Reading comprehension
31.5%
RAMP midline survey findings: mathematics Subtask Number Identification
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
✓ ✓
RAMP midline survey findings: mathematics Subtask Number Identification Quantity Comparison
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
% correct
78.9%
83.7% ***
✓ ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
Number Identification
✓ ✓ ✓ = = =
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
Number Identification
Addition and Subtraction L2
✓ ✓ ✓ = = = ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
% correct
60.3%
64.5% *
% of students with zero scores
3.1%
1.7% *
Number Identification
Addition and Subtraction L2
Missing Number
✓ ✓ ✓ = = = ✓ ✓ ✓ ✓
RAMP midline survey findings: mathematics Subtask
Measure fluency (correct items per min.)
2014 31.8
2017 36.8 ***
% correct of items attempted
88.1%
92.6% ***
Quantity Comparison
% correct
78.9%
83.7% ***
Addition L1
fluency (correct items per min.)
11.9
12.5
Subtraction L1
fluency (correct items per min.)
9.5
10.0
Addition and Subtraction L1
% correct
84.0%
86.0%
% correct
41.9%
46.8% **
% of students with zero scores
13.5%
7.4% ***
% correct
60.3%
64.5% *
% of students with zero scores
3.1%
1.7% *
Number Identification
Addition and Subtraction L2
Missing Number
✓ ✓ ✓ = = = ✓ ✓ ✓ ✓
There is a statistically significant improvement in performance on the more conceptual mathematics tasks i.e. doing mathematics with understanding.
RAMP midline survey findings: mathematics
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Addition and Subtraction L1
% correct
82.6%
85.0%
85.5%
87.0%
Addition and Subtraction L2
% correct
36.8%
42.5%
47.5%
51.2%
Missing Number
% correct
54.3%
60.2%
66.9%
68.9%
RAMP midline survey findings: mathematics
Subtask
Measure
Grade 2
Grade 3
2014
2017
2014
2017
Addition and Subtraction L1
% correct
82.6%
85.0%
85.5%
87.0%
Addition and Subtraction L2
% correct
36.8%
42.5%
47.5%
51.2%
Missing Number
% correct
54.3%
60.2%
66.9%
68.9%
On the conceptual mathematics tasks the 2017 Grade 2 students are approaching the performance levels of the 2014 Grade 3 students.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do gradelevel mathematics with understanding.
RAMP indicators Proportion of learners who, by the end of two grades of primary schooling, demonstrate reading fluency and comprehension of grade-level text.
RAMP indicators Proportion of students who, by the end of two grades of primary schooling, demonstrate that they can do grade-level mathematics with understanding.
أَقرأ ُ الكتاب وأَفه ُم الحساب
ُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: discussion
RAMP midline survey: discussion • For reading there is a statistically significant improvement in performance on all of the foundational skills for reading. This is encouraging as reading with comprehension is a function of performance on foundational skills. – For comprehension there is a statistically significant
narrowing of the gender performance gap! • For mathematics there is a statistically significant improvement in performance on the more conceptual mathematics tasks i.e. doing mathematics with understanding.
44
RAMP midline survey: discussion • On the foundational reading skills the 2017 Grade 2 students are performing at the same level that the Grade 3 students did in 2014. • On the conceptual mathematics tasks the 2017 Grade 2 students are approaching the performance levels of the 2014 Grade 3 students.
45
RAMP midline survey: discussion • The impact of RAMP is not yet being felt at a statistically significant level in the Grade 3 performance.
46
RAMP midline survey: discussion • The RAMP midline survey has generated Governorate level performance data for both EGRA and EGMA. This is the first time that this level of data exists in Jordan. • This data provides the MoE with the data needed to make decisions about the effective deployment of scarce resources.
47
On balance, the RAMP midline survey provides powerful evidence that the RAMP initiative is making a positive impact, which if supported and sustained will, in the years to come, help Jordan achieve the early grade reading and mathematics outcomes that are critical for nation building.
48
49
