Using Data to Catalyze Continuous School Improvement
Using Data to Catalyze Continuous School Improvement How Prince George’s County Public Schools uses data inquiry cycles to drive school and district improvement.
About Prince George’s County Public Schools • Serves 128,000 students
Prince George’s County Public Schools (PGCPS) in Maryland is one of the largest public school districts in the United States. More students and teachers means more data— and more possible ways to look at and think about it. David Rease Jr., Ed.L.D., Executive Director of the Office of Continuous Systemic Improvement at PGCPS, shares how he and his fellow administrators use data to make districtwide improvements. David leads PGCPS’s data-driven improvement efforts using a framework developed at the Harvard Graduate School of Education called the Data Wise Improvement Process. Dr. Rease identified five themes to consider as you start your own improvement cycle after receiving data.
• 22,000 teachers and staff
Organize Your Team
• 208 schools and academic centers • Located in the greater DC-Baltimore area • One of the top 25 largest districts in the US
1. Develop a Shared Mindset “We want all central office staff to use the same process of looking at data so that we have coherence around efforts, which is an important enabling condition for successful collaboration.” Sharing a mindset about how to approach looking at data is a critical first step in the process of leveraging data to make evidence-based decisions. The most effective group mindsets tend to be those focused on what is going to make the greatest difference for student outcomes in the long- and short-term.
Panorama Education
|
The Office of Systemic Improvement in PGCPS works intentionally in groups to use comprehensive data sources in order to drive school improvement across the district.
www.panoramaed.com
1
2. Clarify Group Norms “We set agendas, times, and norms so that we all know how to collaborate well together.” So that meetings are as productive as possible, all participants in the data inquiry process should agree on concrete norms for engaging with each other. Doing so will create a working environment that is productive and safe, so that participants can have discussions with each other from a place of equality and comfort.
Look at (and Learn from) Data Together 1. Work on Data Literacy “We don’t like to make any assumptions around what people know and understand about the data that are most germane to their improvement efforts. We spend a great deal of time helping people to understand how to read reports, how to make appropriate inferences from the data sets that they’re looking at, so that we’re very clear about what the data are saying before we begin to study what the data tell us about how to make improvements.”
PGCPS has worked to build data literacy on many of its teams, starting with understanding where the data come from.
Building data literacy levels the playing field for improvement teams, and means that everyone can get on the same page before starting to dig into the data. Confirming that everyone on your team knows how to look at the data also means that you will be able to maximize your time together, because everyone will begin from a shared starting place. 2. Understand Where the Data Come From “We needed background information that was relevant to our analysis before we could actually begin to look at what students said in our Panorama data.” Making sure that everyone understands how the data was collected and from whom gives participants in the data inquiry cycle confidence in the quality and reliability of the data. When David and his team began to look at their data, they brought in experts to tell them about how it was collected and from whom, as well as other relevant pieces of context.
Panorama Education
|
www.panoramaed.com
2
3. Avoid Jumping to Conclusions “When you put student data in front of people, whether it’s written comments or assessment scores or survey data, and it doesn’t look the way that you want it to, it’s very important to actually see what’s there instead of becoming emotional or drawing quick conclusions or explaining it away. One of my favorite tools in the world is Chris Argyris’s ladder of inference, which helps avoid that problem. When we stay low on the ladder of inference, we find that there’s a lot more consensus around what the data are saying before moving to action.” The ladder of inference has everyone looking at the data begin by noticing trends in the data without making inferences or conclusions. By following a set of steps (called “climbing the ladder”), team members can better understand which assumptions and interpretations they are making about the data along the way. At the lowest rung of the ladder, participants simply describe what they notice before moving up the ladder to interpreting what the data mean, to arriving at conclusions, and eventually taking action. This approach draws upon evidence to arrive at conclusions, which helps facilitate participants’ arriving at the same conclusions and taking action together.
Identify a Manageable Problem 1. Set a General Focus Area “We use protocols to try to determine which of the problems we’ve identified that we should focus on. Our criteria is: which problem, if we tried to solve it, would have the greatest impact on improving the conditions for our students? Deciding which problem to tackle becomes a really rich conversation in which lots of people bring in opinions, research, and past experiences to help us define the problem, its scope, and extent.” Protocols are helpful to ensure that everyone has the chance to give input into selecting the area of focus. You and your team have a sense of what matters to the staff, families, and students in your school or district, so work together to determine which priority area you want to focus on. The more you can encourage and facilitate stakeholder participation in this process, the more your team and other stakeholders will share a sense of urgency and clarity that motivates them to action. 2. Pick a Problem “The data from Panorama and other sources was alarming— we saw very clearly what some of the strengths were in classroom instruction, and what some areas of improvement might be. We started trying to figure out how schools are using or leveraging data to really impact student performance, so we became most interested in thinking about how principals and assistant principals were getting involved in the work of helping students to learn better. ”
Panorama Education
|
www.panoramaed.com
3
Seeing where the most negative trends or patterns are—as well as the most positive—can help narrow your focus. Once you’ve picked some data points, brainstorm what the data suggest and what the implications might be for students, families, and staff. 3. Determine Who the Problem Affects “It’s very important to be clear about whether the problem is a learner-centered problem or a problem of practice or both because when you move into the act phase, you have to be very intentional about the action steps that you take so that you can really move forward.” A learner-centered problem refers to a problem your stakeholder struggles with, often phrased as “Our students struggle with…” A problem of practice refers to our contribution to that problem, often phrased as “As teachers, we struggle to…” Determining who the problem affects and who needs to take steps for improvement will allow you to establish clear metrics for success. In the case of PGCPS, the learners in the problem were the administrators who needed to learn how to use data better to drive student improvement and the district team was the “teachers” determining how to support those administrators. 4. Collect Additional Data “We sought out folks in our research and evaluation departments and asked them to help us design focus group questions so we could dig a little deeper into the student perception data that we got from Panorama. Teams of several people each from the central office went to high schools in the system and had a Q and A with students where we learned even more about what was happening to impact their performance.” Oftentimes, gathering more information about the problem can help define its scope and extent. Engaging with students helped to clarify the problem and to suggest next steps for improvement, while also validating the students’ experiences and the importance of their voices. 5. Self-Reflect “We turned the mirror on ourselves, and asked: what is our role as a central office in this? What did we do to contribute to the successes that the students were speaking to, and what have we done that contributed to some of the things that students wanted to change about their learning environment?” Acknowledging your team’s contribution to the problem is a powerful way to model a commitment to improvement.
Panorama Education
|
www.panoramaed.com
Taking opportunities for group reflection and discussion is an important part of the data inquiry process.
4
Being honest about your role in helping to create and solve problems can help you take action and create additional buy-in from other stakeholders. Most problems have many root causes, and large groups have to be mobilized to work to resolve them.
Take Deliberate Action 1. Engage in Professional Learning “You want to consult the education research or other research as it relates to your field. We want to act and move with intentionality, so we are taking the best action that we possibly can. It’s important to remind people to slow down, be intentional, and engage with the research in order to figure out how to craft the action plan.” Once you have clearly defined your problem, think about what else you want to learn or need to know to take action. Often the desire to fix the problem leads to a hasty movement towards action. Instead, using research to come up with an action plan allows you to draw upon the best practices of others who struggle with similar problems.
Giving educators the opportunity to engage in further professional growth and development helps them get the most out of the data they receive.
Reflect and Iterate 1. Define Your Metrics for Success “You have to be intentional about planning what success is going to look like, which often means going back to the same data sets that you looked at earlier. We have to keep asking students about their experiences in their classes to know if the change in practice that we’ve made is truly helping students to be successful.”
Panorama Education
|
www.panoramaed.com
5
Once an action plan has been put into place, it is essential to be clear on what the metrics for assessing success will be, and at what intervals progress will be measured. Many teams get stuck in the process of data-driven improvement because they forget to establish metrics and intervals for measuring progress, and so have nothing to reflect and iterate upon in future cycles. 2. Share Best Practices “We share best practices with each other in monthly system-wide meetings.” Finding positive outliers— individuals, schools, or teams who are doing successful work— will help your school or district define what success looks like at a larger scale. Sharing best practices will also allow you to learn from everyone in your school or district and build feedback cycles around what is and isn’t working. It’s valuable to host regular convening of
3. Recognize the Overall Benefits faculty and staff to share best practices. “The learning and the collaboration that’s happening as a result of people learning how to look at data together and becoming stronger colleagues impacts practice.”
The benefits of doing data-driven inquiry in a thoughtful, collaborative way can be transformative for teams, schools, and districts. While it may feel like it takes a long time to get to action, moving through the process helps cultivate a datadriven growth mindset for teachers and staff, builds relationships, and encourage participants to reflect and take action on their own. 4. Stay With It “Inquiry is a process. If you stay with it, you will see the dividends.” Improvement cycles take time and sustained energy. Committing to moving through cycles of data inquiry on a regular basis will help your school or district get better continuously— and hold you and your team accountable for measuring progress and making change.
Panorama Education
|
www.panoramaed.com
6
About the Authors
Dr. David Rease, Jr., Ed.L.D. - Dr. Rease is the executive director of the Office of Systemic Improvement in Prince George’s County, MD. Before joining the team at PGCPS, Dr. Rease worked as a high school social studies teacher in Durham, NC. He then worked for the North Carolina Department of Public Instruction as an instructional facilitator for seventeen turnaround high schools. Dr. Rease has also worked at the Mid-Continent for Research in Education and Learning and facilitated their “Power of Data” workshop. Dr. Rease holds a BA from Columbia University, a M.A.T. from Duke University, and an Ed.L.D. from the Harvard Graduate School of Education.
Panorama Education - Panorama Education partners with schools, districts, charter networks, and state departments of education to design and implement survey programs for students, families, and teachers. Panorama offers a technology platform to support survey administration and create reports that are clear, actionable, and, most importantly, help teachers and administrators improve their schools. Panorama currently runs survey programs in over 6,500 schools in 35 states, including those in the Connecticut State Department of Education, San Francisco Unified School District, Dallas Independent School District and Teach for America.
Data Wise Project - The Data Wise Project supports educators in using collaborative data inquiry to drive continuous improvement of teaching and learning for all students. The core of the Data Wise Project is the Data Wise Improvement Process, which helps teams and individuals meaningfully engage with data to catalyze school and district improvement. Data Wise is committed to providing educators with high-quality resources and processes to drive improvement through data.
Panorama Education
|
www.panoramaed.com
7
About Prince George’s County Public Schools • Serves 128,000 students
Prince George’s County Public Schools (PGCPS) in Maryland is one of the largest public school districts in the United States. More students and teachers means more data— and more possible ways to look at and think about it. David Rease Jr., Ed.L.D., Executive Director of the Office of Continuous Systemic Improvement at PGCPS, shares how he and his fellow administrators use data to make districtwide improvements. David leads PGCPS’s data-driven improvement efforts using a framework developed at the Harvard Graduate School of Education called the Data Wise Improvement Process. Dr. Rease identified five themes to consider as you start your own improvement cycle after receiving data.
• 22,000 teachers and staff
Organize Your Team
• 208 schools and academic centers • Located in the greater DC-Baltimore area • One of the top 25 largest districts in the US
1. Develop a Shared Mindset “We want all central office staff to use the same process of looking at data so that we have coherence around efforts, which is an important enabling condition for successful collaboration.” Sharing a mindset about how to approach looking at data is a critical first step in the process of leveraging data to make evidence-based decisions. The most effective group mindsets tend to be those focused on what is going to make the greatest difference for student outcomes in the long- and short-term.
Panorama Education
|
The Office of Systemic Improvement in PGCPS works intentionally in groups to use comprehensive data sources in order to drive school improvement across the district.
www.panoramaed.com
1
2. Clarify Group Norms “We set agendas, times, and norms so that we all know how to collaborate well together.” So that meetings are as productive as possible, all participants in the data inquiry process should agree on concrete norms for engaging with each other. Doing so will create a working environment that is productive and safe, so that participants can have discussions with each other from a place of equality and comfort.
Look at (and Learn from) Data Together 1. Work on Data Literacy “We don’t like to make any assumptions around what people know and understand about the data that are most germane to their improvement efforts. We spend a great deal of time helping people to understand how to read reports, how to make appropriate inferences from the data sets that they’re looking at, so that we’re very clear about what the data are saying before we begin to study what the data tell us about how to make improvements.”
PGCPS has worked to build data literacy on many of its teams, starting with understanding where the data come from.
Building data literacy levels the playing field for improvement teams, and means that everyone can get on the same page before starting to dig into the data. Confirming that everyone on your team knows how to look at the data also means that you will be able to maximize your time together, because everyone will begin from a shared starting place. 2. Understand Where the Data Come From “We needed background information that was relevant to our analysis before we could actually begin to look at what students said in our Panorama data.” Making sure that everyone understands how the data was collected and from whom gives participants in the data inquiry cycle confidence in the quality and reliability of the data. When David and his team began to look at their data, they brought in experts to tell them about how it was collected and from whom, as well as other relevant pieces of context.
Panorama Education
|
www.panoramaed.com
2
3. Avoid Jumping to Conclusions “When you put student data in front of people, whether it’s written comments or assessment scores or survey data, and it doesn’t look the way that you want it to, it’s very important to actually see what’s there instead of becoming emotional or drawing quick conclusions or explaining it away. One of my favorite tools in the world is Chris Argyris’s ladder of inference, which helps avoid that problem. When we stay low on the ladder of inference, we find that there’s a lot more consensus around what the data are saying before moving to action.” The ladder of inference has everyone looking at the data begin by noticing trends in the data without making inferences or conclusions. By following a set of steps (called “climbing the ladder”), team members can better understand which assumptions and interpretations they are making about the data along the way. At the lowest rung of the ladder, participants simply describe what they notice before moving up the ladder to interpreting what the data mean, to arriving at conclusions, and eventually taking action. This approach draws upon evidence to arrive at conclusions, which helps facilitate participants’ arriving at the same conclusions and taking action together.
Identify a Manageable Problem 1. Set a General Focus Area “We use protocols to try to determine which of the problems we’ve identified that we should focus on. Our criteria is: which problem, if we tried to solve it, would have the greatest impact on improving the conditions for our students? Deciding which problem to tackle becomes a really rich conversation in which lots of people bring in opinions, research, and past experiences to help us define the problem, its scope, and extent.” Protocols are helpful to ensure that everyone has the chance to give input into selecting the area of focus. You and your team have a sense of what matters to the staff, families, and students in your school or district, so work together to determine which priority area you want to focus on. The more you can encourage and facilitate stakeholder participation in this process, the more your team and other stakeholders will share a sense of urgency and clarity that motivates them to action. 2. Pick a Problem “The data from Panorama and other sources was alarming— we saw very clearly what some of the strengths were in classroom instruction, and what some areas of improvement might be. We started trying to figure out how schools are using or leveraging data to really impact student performance, so we became most interested in thinking about how principals and assistant principals were getting involved in the work of helping students to learn better. ”
Panorama Education
|
www.panoramaed.com
3
Seeing where the most negative trends or patterns are—as well as the most positive—can help narrow your focus. Once you’ve picked some data points, brainstorm what the data suggest and what the implications might be for students, families, and staff. 3. Determine Who the Problem Affects “It’s very important to be clear about whether the problem is a learner-centered problem or a problem of practice or both because when you move into the act phase, you have to be very intentional about the action steps that you take so that you can really move forward.” A learner-centered problem refers to a problem your stakeholder struggles with, often phrased as “Our students struggle with…” A problem of practice refers to our contribution to that problem, often phrased as “As teachers, we struggle to…” Determining who the problem affects and who needs to take steps for improvement will allow you to establish clear metrics for success. In the case of PGCPS, the learners in the problem were the administrators who needed to learn how to use data better to drive student improvement and the district team was the “teachers” determining how to support those administrators. 4. Collect Additional Data “We sought out folks in our research and evaluation departments and asked them to help us design focus group questions so we could dig a little deeper into the student perception data that we got from Panorama. Teams of several people each from the central office went to high schools in the system and had a Q and A with students where we learned even more about what was happening to impact their performance.” Oftentimes, gathering more information about the problem can help define its scope and extent. Engaging with students helped to clarify the problem and to suggest next steps for improvement, while also validating the students’ experiences and the importance of their voices. 5. Self-Reflect “We turned the mirror on ourselves, and asked: what is our role as a central office in this? What did we do to contribute to the successes that the students were speaking to, and what have we done that contributed to some of the things that students wanted to change about their learning environment?” Acknowledging your team’s contribution to the problem is a powerful way to model a commitment to improvement.
Panorama Education
|
www.panoramaed.com
Taking opportunities for group reflection and discussion is an important part of the data inquiry process.
4
Being honest about your role in helping to create and solve problems can help you take action and create additional buy-in from other stakeholders. Most problems have many root causes, and large groups have to be mobilized to work to resolve them.
Take Deliberate Action 1. Engage in Professional Learning “You want to consult the education research or other research as it relates to your field. We want to act and move with intentionality, so we are taking the best action that we possibly can. It’s important to remind people to slow down, be intentional, and engage with the research in order to figure out how to craft the action plan.” Once you have clearly defined your problem, think about what else you want to learn or need to know to take action. Often the desire to fix the problem leads to a hasty movement towards action. Instead, using research to come up with an action plan allows you to draw upon the best practices of others who struggle with similar problems.
Giving educators the opportunity to engage in further professional growth and development helps them get the most out of the data they receive.
Reflect and Iterate 1. Define Your Metrics for Success “You have to be intentional about planning what success is going to look like, which often means going back to the same data sets that you looked at earlier. We have to keep asking students about their experiences in their classes to know if the change in practice that we’ve made is truly helping students to be successful.”
Panorama Education
|
www.panoramaed.com
5
Once an action plan has been put into place, it is essential to be clear on what the metrics for assessing success will be, and at what intervals progress will be measured. Many teams get stuck in the process of data-driven improvement because they forget to establish metrics and intervals for measuring progress, and so have nothing to reflect and iterate upon in future cycles. 2. Share Best Practices “We share best practices with each other in monthly system-wide meetings.” Finding positive outliers— individuals, schools, or teams who are doing successful work— will help your school or district define what success looks like at a larger scale. Sharing best practices will also allow you to learn from everyone in your school or district and build feedback cycles around what is and isn’t working. It’s valuable to host regular convening of
3. Recognize the Overall Benefits faculty and staff to share best practices. “The learning and the collaboration that’s happening as a result of people learning how to look at data together and becoming stronger colleagues impacts practice.”
The benefits of doing data-driven inquiry in a thoughtful, collaborative way can be transformative for teams, schools, and districts. While it may feel like it takes a long time to get to action, moving through the process helps cultivate a datadriven growth mindset for teachers and staff, builds relationships, and encourage participants to reflect and take action on their own. 4. Stay With It “Inquiry is a process. If you stay with it, you will see the dividends.” Improvement cycles take time and sustained energy. Committing to moving through cycles of data inquiry on a regular basis will help your school or district get better continuously— and hold you and your team accountable for measuring progress and making change.
Panorama Education
|
www.panoramaed.com
6
About the Authors
Dr. David Rease, Jr., Ed.L.D. - Dr. Rease is the executive director of the Office of Systemic Improvement in Prince George’s County, MD. Before joining the team at PGCPS, Dr. Rease worked as a high school social studies teacher in Durham, NC. He then worked for the North Carolina Department of Public Instruction as an instructional facilitator for seventeen turnaround high schools. Dr. Rease has also worked at the Mid-Continent for Research in Education and Learning and facilitated their “Power of Data” workshop. Dr. Rease holds a BA from Columbia University, a M.A.T. from Duke University, and an Ed.L.D. from the Harvard Graduate School of Education.
Panorama Education - Panorama Education partners with schools, districts, charter networks, and state departments of education to design and implement survey programs for students, families, and teachers. Panorama offers a technology platform to support survey administration and create reports that are clear, actionable, and, most importantly, help teachers and administrators improve their schools. Panorama currently runs survey programs in over 6,500 schools in 35 states, including those in the Connecticut State Department of Education, San Francisco Unified School District, Dallas Independent School District and Teach for America.
Data Wise Project - The Data Wise Project supports educators in using collaborative data inquiry to drive continuous improvement of teaching and learning for all students. The core of the Data Wise Project is the Data Wise Improvement Process, which helps teams and individuals meaningfully engage with data to catalyze school and district improvement. Data Wise is committed to providing educators with high-quality resources and processes to drive improvement through data.
Panorama Education
|
www.panoramaed.com
7
