smarter balanced - Carnegie Learning
National Assessments: A Closer Look at Smarter Balanced
Liz Storey
KEYNOTES
Liz Storey Director of Educational Partnerships Carnegie Learning, Inc.
KEYNOTES
Kasey Bratcher Director of Educational Partnerships Carnegie Learning, Inc.
Most Recent Updates Spring 2014 – Field Testing • Smarter Balanced will conduct Field Test of summative and interim assessment items and performance tasks from March 18-June 6 • In this large scale effort, Smarter Balanced will present all assessment items to students across all member states (in approximately 15-20% of consortium schools) • Most students will complete a test in either ELA or mathematics • Each subject area is expected to take 3-4 hours (intended to be untimed) • Schools may administer Field Test over multiple days; will be administered only via computer
SPRING 2014 – FIELD TESTING • Some states will administer in place of their state accountability test; others in addition to their state’s test (decision made by states) • Most students will receive a fixed form (not an adaptive form) with questions of varying difficulty. Will collect the statistics necessary to determine item difficulty levels. • All 23 Governing States will participate in grades 3-8 and 11 (and a small sample of students in grades 9 and 10) • Is a trial run to help ensure assessments are valid, reliable and fair for all students • Will help schools gauge their readiness in advance of the first operational assessment in spring 2015
SEPTEMBER 10, 2013 Smarter Balanced Governing States voted
to move forward with the development of a student data privacy policy.
Governing States unanimously approved the following motion: • Each member state retains control of its student-level data. • Smarter Balanced will develop a data privacy policy consistent with this principle
September 2013 Usability, Accessibility, and Accommodations Guidelines adopted to ensure assessment system meets the needs of English language learners and students with disabilities.
May 29, 2013 Smarter Balanced released sets of example test questions for grades 3–8 and 11 in both English language arts/literacy and mathematics.
Test includes
selected-response items constructed-response items technology-enhanced items performance tasks
April 25, 2013 Grandview Elementary School was one of 1,000 schools in California selected to take the Smarter Balanced pilot test. It also was a test of Grandview's computers, operating systems, Internet connectivity and bandwidth to see if the new t system would run on the district’s computer infrastructure.
For Grandview's 3rd-graders, mostly 8-yearolds, the Smarter Balanced Pilot Test required a computer, a keyboard, a headset, and a connection to the Internet.
“It was a very worthwhile experience because it showed that our technology will be able to handle the requirements of the (Smarter Balanced) test, and it also showed us where we are going to need to work on the computer skills we're going to need to give to our children to be successful.” Grandview Principal Cheri Singer
February 20, 2013 Smarter Balanced began administering a Pilot Test of assessment Items and performance tasks in schools across the country Research sample included 10 percent of public school students in member States First of two large-scale tryouts of the assessments prior to implementation in 2014-15
Gathering information about the performance of assessment items and the test delivery system under real-world conditions
February 1, 2013 State assessment director, Gloria Turner, confirmed that Alabama had bowed out of both PARCC and SMARTER BALANCED consortia
Indicated the state had decided to go “in another direction," but didn't offer any more detail
November 2012 Smarter Balanced adopted assessment blueprints and accompanying materials Blueprints include critical information about the number of items, score points, and depth of knowledge for items associated with each assessment target.
In addition to guiding the development of Pilot and Field tests, they will further guide score reporting and standard setting They will likely continue to be revised with the analysis of Pilot and Field tests
SMARTER BALANCED 25 STATES + U.S. VIRGIN ISLANDS (AFFILIATE MEMBER)
GOVERNING STATES California Delaware Idaho Kansas Michigan Montana
Connecticut Hawaii Iowa Maine Missouri Nevada
New Hampshire Oregon South Dakota Washington Wisconsin North Dakota
North Carolina South Carolina Vermont West Virginia Wyoming
PARTICIPATING STATES Alaska Pennsylvania
U.S Virgin Islands (Affiliate State)
KEY DIFFERENCE PARCC - Fixed-Form Delivery Will be computer-assisted. Students take one of several fixed, equated sets of items and tasks.
Smarter Balanced - Adaptive Delivery Students will get an individually tailored set of items and tasks depending on their responses. Performance-based tests will be taken on computers but will not be computer adapted.
SMARTER BALANCED WILL . . . Indicate students’ progress and ultimate readiness for career and college Inform instruction, guide interventions, and help target professional development Be taken on computers ELA/Literary and Math (Grades 3-8 and 11) Include selected-response, constructed response, technology-enhanced, and complex performance tasks Cost less on average according to projections than states are currently paying for assessments (est. cost for full suite of summative, interim, and formative assessment is $27.30 per student)
Provide sample assessment items to inform Common Core implementation
SMARTER BALANCED WILL . . . Have two required components, both given during the last 12 weeks of the school year Use electronic and human scoring with results available within two weeks Utilize technology enhancements including
creating a line moving one or more objects producing geometric shapes selecting one or more points on a graphic dragging and dropping a graphic selecting blocks of text marking a visual field
SMARTER BALANCED WILL HAVE . . . optional interim assessments, which will also be computer-adaptive a Digital Library of high quality resources and tools for teachers formative items and tasks for classroom use a secure online reporting system
Performance tasks to challenge students to respond to real-world problems (will take1-2 class periods to complete
a pencil/paper version for a three-year transition period
ADDITIONAL SMARTER BALANCED DETAILS System will draw from a large bank of items with students receiving different questions based on their responses – SB claims test will be more secure and able to be used for a longer period of time Will not replace ACT or SAT but states are working with higher ed toward a goal of the scores being accepted as a College-and-Career Ready Determination Optional Interim assessments will involve a large teacher role in developing and scoring items and performance tasks Will offer a retake opportunity on the CAT portion of the summative assessment SB indicates “. . . a 600-student middle school will be able to test its students using only one 30-computer lab.”
• PARCC will have one optional diagnostic and one optional mid-year assessment. Mid-year assessment will contain mostly tasks similar to the summative performance-based tasks
• Smarter Balanced will have optional interim assessments for grades 3-12 which will be computer adaptive with multiple item types, including performance tasks. The number of tests, timing, and scope will be locally determined.
SMARTER BALANCED MATHEMATICS DESIGN Overall Claim: Students can demonstrate progress toward college and career readiness in mathematics (grades 3-8) and can demonstrate college and career readiness in mathematics (grade 11).
Claim #1: Concepts & Procedures: Students can explain and apply mathematical concepts Can interpret and carry out mathematical procedures with precision and fluency
Claim #2: Problem Solving: Students can solve a range of complex well-posed problems in pure and applied mathematics Can make productive use of knowledge and problem-solving strategies .
Claim #3: Communicating Reasoning: Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others
Claim #4: Modeling & Data Analysis: Students can analyze complex, real-world scenarios Can construct and use mathematical models to interpret and solve problems
A Look at Sample Items
Shifts in Instruction Preparing Students for Smarter Balanced Tasks
“The sample items and performance tasks are intended to help teachers, administrators, and policymakers implementing the CCSS and preparing for next-generation assessments. They provide an early look into the depth of understanding of the CCSS that will be measured by the Smarter Balanced assessment system. While the items and tasks are not intended to be used as sample tests, educators can use them to begin planning the shifts in instruction that will be required to help students meet the demands of the new assessments.”
“These assessments will go beyond multiple choice to include performance tasks that allow students to demonstrate research, writing, and analytical skills.”
5th Grade Sample Item
3RD Grade Practice Test Items
Instructional Significance
Students MUST be given opportunities to look for and make use of structure.
8th Grade Online Performance Task
8th Grade Online Performance Task
8th Grade Online Performance Task
Instructional Significance
Students MUST be given opportunities to contextualize and decontextualize.
Instructional Significance
# of student tickets
$ raised from student tickets
tickets
dollars
1250
3000
Students MUST be given opportunities to contextualize and decontextualize.
6th Grade Sample Item
Instructional Significance
Students MUST be given opportunities based on their needs!!
6th Grade Online Performance Task
6th Grade Online Performance Task
Instructional Significance
Students MUST be given opportunities to construct viable arguments and critique the reasoning of others.
6th Grade Sample Performance Task Your class and your teacher are going on a field trip. There are three possible choices for the field trip: an aquarium, a science museum, or a zoo. Students were asked to write down their first and second choices. In this task, you will determine where the class should go on the field trip based on the survey results and the cost per student.
6th Grade Sample Performance Task Based only the results of the class votes, where would you recommend the class go on the field trip?
Show your work or explain how you found your answer.
6th Grade Sample Performance Task The teacher and parent helpers do not pay an entrance fee. There are 30 students in the class. Only 1 bus is needed. The bus charge is for the entire busload of students (not for each student). Each student will pay the same amount. The school fund will pay the first $200 of the trip.
Now we will think about the costs of the trip. How much will each student pay to go on each trip?
Daniel thinks that it will cost less to go to the zoo because the entrance fee is only $2.50 per person. Explain why you agree or disagree. Write a short note to your teacher…
Instructional Significance Kay Merseth on productive struggle…
“…it’s not about guessing what the teacher wants to hear or about finding a particular answer. It is about the process of thinking, making sense, and persevering in the face of not knowing exactly how to proceed or whether a particular approach will work. Exploring, investigating one or multiple approaches, and articulating a chain of reasoning behind the approaches characterize productive struggle.”
Students MUST be given opportunities to make sense of problems and persevere!!
Overall Instructional Implications We must give students ample opportunities to experience and practice ALL EIGHT of the Standards for Mathematical Practice.
We must use mathematical tasks with real-world contexts on a DAILY basis!!! We must give students ample opportunities to interact with online tools and interactive software. We must move away from the teacher-centered classroom and allow students opportunities to makes sense of the mathematics together with their peers by posing good tasks, asking good questions and promoting productive struggle.
For More Information, Contact me:
Kasey Bratcher, Director of Professional Development 888.851.7094 ext 416 270-405-7323 [email protected] www.carnegielearning.com
@kbratcher31 Kasey Bratcher
Liz Storey
KEYNOTES
Liz Storey Director of Educational Partnerships Carnegie Learning, Inc.
KEYNOTES
Kasey Bratcher Director of Educational Partnerships Carnegie Learning, Inc.
Most Recent Updates Spring 2014 – Field Testing • Smarter Balanced will conduct Field Test of summative and interim assessment items and performance tasks from March 18-June 6 • In this large scale effort, Smarter Balanced will present all assessment items to students across all member states (in approximately 15-20% of consortium schools) • Most students will complete a test in either ELA or mathematics • Each subject area is expected to take 3-4 hours (intended to be untimed) • Schools may administer Field Test over multiple days; will be administered only via computer
SPRING 2014 – FIELD TESTING • Some states will administer in place of their state accountability test; others in addition to their state’s test (decision made by states) • Most students will receive a fixed form (not an adaptive form) with questions of varying difficulty. Will collect the statistics necessary to determine item difficulty levels. • All 23 Governing States will participate in grades 3-8 and 11 (and a small sample of students in grades 9 and 10) • Is a trial run to help ensure assessments are valid, reliable and fair for all students • Will help schools gauge their readiness in advance of the first operational assessment in spring 2015
SEPTEMBER 10, 2013 Smarter Balanced Governing States voted
to move forward with the development of a student data privacy policy.
Governing States unanimously approved the following motion: • Each member state retains control of its student-level data. • Smarter Balanced will develop a data privacy policy consistent with this principle
September 2013 Usability, Accessibility, and Accommodations Guidelines adopted to ensure assessment system meets the needs of English language learners and students with disabilities.
May 29, 2013 Smarter Balanced released sets of example test questions for grades 3–8 and 11 in both English language arts/literacy and mathematics.
Test includes
selected-response items constructed-response items technology-enhanced items performance tasks
April 25, 2013 Grandview Elementary School was one of 1,000 schools in California selected to take the Smarter Balanced pilot test. It also was a test of Grandview's computers, operating systems, Internet connectivity and bandwidth to see if the new t system would run on the district’s computer infrastructure.
For Grandview's 3rd-graders, mostly 8-yearolds, the Smarter Balanced Pilot Test required a computer, a keyboard, a headset, and a connection to the Internet.
“It was a very worthwhile experience because it showed that our technology will be able to handle the requirements of the (Smarter Balanced) test, and it also showed us where we are going to need to work on the computer skills we're going to need to give to our children to be successful.” Grandview Principal Cheri Singer
February 20, 2013 Smarter Balanced began administering a Pilot Test of assessment Items and performance tasks in schools across the country Research sample included 10 percent of public school students in member States First of two large-scale tryouts of the assessments prior to implementation in 2014-15
Gathering information about the performance of assessment items and the test delivery system under real-world conditions
February 1, 2013 State assessment director, Gloria Turner, confirmed that Alabama had bowed out of both PARCC and SMARTER BALANCED consortia
Indicated the state had decided to go “in another direction," but didn't offer any more detail
November 2012 Smarter Balanced adopted assessment blueprints and accompanying materials Blueprints include critical information about the number of items, score points, and depth of knowledge for items associated with each assessment target.
In addition to guiding the development of Pilot and Field tests, they will further guide score reporting and standard setting They will likely continue to be revised with the analysis of Pilot and Field tests
SMARTER BALANCED 25 STATES + U.S. VIRGIN ISLANDS (AFFILIATE MEMBER)
GOVERNING STATES California Delaware Idaho Kansas Michigan Montana
Connecticut Hawaii Iowa Maine Missouri Nevada
New Hampshire Oregon South Dakota Washington Wisconsin North Dakota
North Carolina South Carolina Vermont West Virginia Wyoming
PARTICIPATING STATES Alaska Pennsylvania
U.S Virgin Islands (Affiliate State)
KEY DIFFERENCE PARCC - Fixed-Form Delivery Will be computer-assisted. Students take one of several fixed, equated sets of items and tasks.
Smarter Balanced - Adaptive Delivery Students will get an individually tailored set of items and tasks depending on their responses. Performance-based tests will be taken on computers but will not be computer adapted.
SMARTER BALANCED WILL . . . Indicate students’ progress and ultimate readiness for career and college Inform instruction, guide interventions, and help target professional development Be taken on computers ELA/Literary and Math (Grades 3-8 and 11) Include selected-response, constructed response, technology-enhanced, and complex performance tasks Cost less on average according to projections than states are currently paying for assessments (est. cost for full suite of summative, interim, and formative assessment is $27.30 per student)
Provide sample assessment items to inform Common Core implementation
SMARTER BALANCED WILL . . . Have two required components, both given during the last 12 weeks of the school year Use electronic and human scoring with results available within two weeks Utilize technology enhancements including
creating a line moving one or more objects producing geometric shapes selecting one or more points on a graphic dragging and dropping a graphic selecting blocks of text marking a visual field
SMARTER BALANCED WILL HAVE . . . optional interim assessments, which will also be computer-adaptive a Digital Library of high quality resources and tools for teachers formative items and tasks for classroom use a secure online reporting system
Performance tasks to challenge students to respond to real-world problems (will take1-2 class periods to complete
a pencil/paper version for a three-year transition period
ADDITIONAL SMARTER BALANCED DETAILS System will draw from a large bank of items with students receiving different questions based on their responses – SB claims test will be more secure and able to be used for a longer period of time Will not replace ACT or SAT but states are working with higher ed toward a goal of the scores being accepted as a College-and-Career Ready Determination Optional Interim assessments will involve a large teacher role in developing and scoring items and performance tasks Will offer a retake opportunity on the CAT portion of the summative assessment SB indicates “. . . a 600-student middle school will be able to test its students using only one 30-computer lab.”
• PARCC will have one optional diagnostic and one optional mid-year assessment. Mid-year assessment will contain mostly tasks similar to the summative performance-based tasks
• Smarter Balanced will have optional interim assessments for grades 3-12 which will be computer adaptive with multiple item types, including performance tasks. The number of tests, timing, and scope will be locally determined.
SMARTER BALANCED MATHEMATICS DESIGN Overall Claim: Students can demonstrate progress toward college and career readiness in mathematics (grades 3-8) and can demonstrate college and career readiness in mathematics (grade 11).
Claim #1: Concepts & Procedures: Students can explain and apply mathematical concepts Can interpret and carry out mathematical procedures with precision and fluency
Claim #2: Problem Solving: Students can solve a range of complex well-posed problems in pure and applied mathematics Can make productive use of knowledge and problem-solving strategies .
Claim #3: Communicating Reasoning: Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others
Claim #4: Modeling & Data Analysis: Students can analyze complex, real-world scenarios Can construct and use mathematical models to interpret and solve problems
A Look at Sample Items
Shifts in Instruction Preparing Students for Smarter Balanced Tasks
“The sample items and performance tasks are intended to help teachers, administrators, and policymakers implementing the CCSS and preparing for next-generation assessments. They provide an early look into the depth of understanding of the CCSS that will be measured by the Smarter Balanced assessment system. While the items and tasks are not intended to be used as sample tests, educators can use them to begin planning the shifts in instruction that will be required to help students meet the demands of the new assessments.”
“These assessments will go beyond multiple choice to include performance tasks that allow students to demonstrate research, writing, and analytical skills.”
5th Grade Sample Item
3RD Grade Practice Test Items
Instructional Significance
Students MUST be given opportunities to look for and make use of structure.
8th Grade Online Performance Task
8th Grade Online Performance Task
8th Grade Online Performance Task
Instructional Significance
Students MUST be given opportunities to contextualize and decontextualize.
Instructional Significance
# of student tickets
$ raised from student tickets
tickets
dollars
1250
3000
Students MUST be given opportunities to contextualize and decontextualize.
6th Grade Sample Item
Instructional Significance
Students MUST be given opportunities based on their needs!!
6th Grade Online Performance Task
6th Grade Online Performance Task
Instructional Significance
Students MUST be given opportunities to construct viable arguments and critique the reasoning of others.
6th Grade Sample Performance Task Your class and your teacher are going on a field trip. There are three possible choices for the field trip: an aquarium, a science museum, or a zoo. Students were asked to write down their first and second choices. In this task, you will determine where the class should go on the field trip based on the survey results and the cost per student.
6th Grade Sample Performance Task Based only the results of the class votes, where would you recommend the class go on the field trip?
Show your work or explain how you found your answer.
6th Grade Sample Performance Task The teacher and parent helpers do not pay an entrance fee. There are 30 students in the class. Only 1 bus is needed. The bus charge is for the entire busload of students (not for each student). Each student will pay the same amount. The school fund will pay the first $200 of the trip.
Now we will think about the costs of the trip. How much will each student pay to go on each trip?
Daniel thinks that it will cost less to go to the zoo because the entrance fee is only $2.50 per person. Explain why you agree or disagree. Write a short note to your teacher…
Instructional Significance Kay Merseth on productive struggle…
“…it’s not about guessing what the teacher wants to hear or about finding a particular answer. It is about the process of thinking, making sense, and persevering in the face of not knowing exactly how to proceed or whether a particular approach will work. Exploring, investigating one or multiple approaches, and articulating a chain of reasoning behind the approaches characterize productive struggle.”
Students MUST be given opportunities to make sense of problems and persevere!!
Overall Instructional Implications We must give students ample opportunities to experience and practice ALL EIGHT of the Standards for Mathematical Practice.
We must use mathematical tasks with real-world contexts on a DAILY basis!!! We must give students ample opportunities to interact with online tools and interactive software. We must move away from the teacher-centered classroom and allow students opportunities to makes sense of the mathematics together with their peers by posing good tasks, asking good questions and promoting productive struggle.
For More Information, Contact me:
Kasey Bratcher, Director of Professional Development 888.851.7094 ext 416 270-405-7323 [email protected] www.carnegielearning.com
@kbratcher31 Kasey Bratcher
