Data Analysis for Continuous School Improvement
Data Literacy and Perceptual Data
https://goo.gl/r8fQSc 2015 ADMINISTRATORS’ DAYS ► NDE DAY
About this Session 1. Data Literacies Overview The importance of : data literacy and value of data use. Goal: Utilize established data protocols for decision-making.
2. Perceptual Data The value of perceptual data and access options. Pursue field testing and use perceptual data for improvement purposes.
Data Literacies: Purpose 1. Understand and use the Data Literacies in districts and schools 2. Recognize the connection between the Data Literacies and the Continuous Improvement Process 3. Recognize the connection between the Data Literacies and the AQuESTT tenets 4. Identify resources available through NDE & ESUs for analyzing data through the Continuous Improvement Process
Goal 2 SLDS Grant Provide a statewide system of data analysis training that reaches every district and is sustainable once the grant funds have been expended.
Tiered Training Delivery Statewide Data Cadre ✔ ESUs and NDE Staff ✔ School District Leadership and Staff ✔
Data Cadre • Create data use and literacy capacity and culture in Nebraska school districts. • Collaborative effort between NDE and ESUCC: – 8 ESU consultants from 7 service units and the ESUCC – 11 NDE representatives from 8 departments – 1 representative from higher education
Understanding AQuESTT HAVE TO DOs GET TO DOs
Nebraska Data Literacies Data Comprehension Evaluation Did our response produce results?
What do the data show?
Data Use How should we respond?
Data Interpretation Why might this be?
Data Literacies Format
1.
Literacy a.
Concept i.
Indicators
Literacy 1 Data Comprehension: What do the data show? Concepts: a. Continuous Improvement Process b. District/School Profile c. Data Tools and Skills
Data Literacies
Data Literacies Definitions
Data Use Curriculum
Nebraska Data Literacies
Bernhardt, V.L. (2013)
Data Analysis for Continuous School Improvement (Third Edition) New York, NY: Routledge
Data Literacy 1 What do the data show?
Where are we now? Data Literacy 2 Why might this be? Data Literacy 2 Why might this be?
Where do we want to be?
Data Literacy 4 Did our response produce results?
Is what we are doing making a difference?
How did we get to where we are?
Data Literacy 3 How should we respond?
How are we going to get to where we want to be?
Using Data for Continuous Improvement Using Results Continuous School Improvement
Continuous Improvement Models NEBRASKA MODEL FOR CONTINUOUS IMPROVEMENT
Nebraska Frameworks
AdvancED
Rule 10: Regulations and Procedures for the Accreditation of Schools 009 Continuous School Improvement • 009.01A2 Collection and analysis of data about student performance, demographics, learning climate, and former high school students.
AdvancED • Standard 5: Using Results for Continuous Improvement
NDE Continuous School Improvement Workshops • Learn about the Nebraska Data Literacies • Learn data analysis techniques • Define appropriate action steps based on data analysis • Retrieve student data submitted to NDE via DRS
NDE Continuous School Improvement Workshops 2015-2016 Workshops • September 22 & 23 – Norfolk • September 28 & 29 – Gering • October 15 & 16 – Lincoln • October 19 & 20 – Omaha • October 26 & 27 – Kearney
Each school will receive a copy of Data Analysis for Continuous School Improvement
Perceptual Data: Purpose 1.
Understand the role that perceptual data plays
2.
Establish a process for perceptual data collection
3.
Share existing data collection resources
Role of Perceptual Data Why Survey? It is REQUIRED! • AdvancED requires the administration of ASSIST surveys within 1 year of the External Review • Nebraska Frameworks should utilize perceptual data to be a part of the district profile for the external visit
It is One of the Four Lenses Multiple Measures of Data • • • •
Demographics School Processes Student Learning Perceptions
It Answers the Question ‘How do we do business?’ Where we are now? What do students, staff, and parents think about the learning environment?
Perceptions impact beliefs, behaviors, and, to an extent, outcomes.
Collection Process 1. 2. 3. 4. 5. 6. 7. 8. 9.
Identify the Purpose Determine the Content* Develop or Select the Instrument* Pilot the Instrument* Field the Instrument/Collect the Data Analyze Results Report Results* - data displays Share and Review Results Use Results
*NDE/ESU Perceptual Survey Service
Identify the Purpose 1.
What do you want to learn?
2.
Will it be important to monitor changes in perceptions over time? How often? For how long?
3.
How will you use your survey results?
Determine the Content • What do you need to know? • What do you want to know? • Which stakeholders’ perspectives are relevant and should be considered?
Select/Develop and Pilot the Instrument 1. Who is the pilot group? 2. Is the instrument working the way you want it to work?
Collect the Data ●
How will the survey be administered? (paper/pencil, web-based)
●
When will the survey be administered?
Available Resources What instruments have already been developed? 1.
ASSIST Surveys (AdvancED)
1.
State Surveys (NDE)
Example Survey Instruments Education For The Future Victoria Bernhardt
Example Survey Instruments
Survey Constructs What is measured? Some constructs that are measured by perceptual surveys include the following: For Students
For Staff
For Parents
Safety
School Environment
Children’s Safety
Freedom of Choice
Administrative Support
School Staff
Relationship with Teachers
Staff Morale
School Assessment
Relationship with Other Students
Relationship with Students Learning Environment
School Performance
Student Achievement
Family Support
NDE/ESU Request Form ▼Click to view
How is the Data Displayed and Organized? • The data is graphed to objectively and fairly demonstrate the results.
• The goal of data display is to inform the audience and create consistent understanding.
Displaying Perceptual Data Results Page 248
Data Interpretation 1. How can the results be analyzed to show the information gleaned from the survey? 2. What did we learn about our learning environment? 3. What connections can be made when correlated with other data sources (i.e., demographic, school processes, student learning)?
Using the Results • How do these results help the CIP process? • What can you learn from these results? • How do we change perceptions?
Reporting the Results • How are the results going to be shared? • When are the results going to be shared? • When/how do you report changes that were made due to these results?
Analyzing Text Data • Responses to the two open-ended questions are also provided as part of the NDE/ESU Perceptual Survey Service. • Open source data analysis software is available. Contact your local ESU for more information.
Analyzing Text Data Protocol
Pages 242-243
NDE/ESU Perceptual Surveys • Ability to administer surveys will continue to be available in the future. • Perceptual data survey services will continue to improve and expand. • A vision is in place to develop minisurveys by constructs (main ideas being measured).
NDE/ESU Data Cadre Assistance The NDE Data, Research, and Evaluation (DRE) Team or ESU Data Cadre can assist you with ● ● ● ●
New Surveys Quality Instrument review Survey data analysis Survey management process assistance
● ● ●
Sources of survey error Survey software recommendations Survey question and design development
THINK BROADER Holistic view of each student Equal access to information
BE BETTER Continuous improvement Sharing of best practices Focused on every student, every day
Collective impact
EXPERIENCE BOLDER New system for measurement Customized for Nebraska Outcome-based approach
Thank you!
https://goo.gl/r8fQSc 2015 ADMINISTRATORS’ DAYS ► NDE DAY
About this Session 1. Data Literacies Overview The importance of : data literacy and value of data use. Goal: Utilize established data protocols for decision-making.
2. Perceptual Data The value of perceptual data and access options. Pursue field testing and use perceptual data for improvement purposes.
Data Literacies: Purpose 1. Understand and use the Data Literacies in districts and schools 2. Recognize the connection between the Data Literacies and the Continuous Improvement Process 3. Recognize the connection between the Data Literacies and the AQuESTT tenets 4. Identify resources available through NDE & ESUs for analyzing data through the Continuous Improvement Process
Goal 2 SLDS Grant Provide a statewide system of data analysis training that reaches every district and is sustainable once the grant funds have been expended.
Tiered Training Delivery Statewide Data Cadre ✔ ESUs and NDE Staff ✔ School District Leadership and Staff ✔
Data Cadre • Create data use and literacy capacity and culture in Nebraska school districts. • Collaborative effort between NDE and ESUCC: – 8 ESU consultants from 7 service units and the ESUCC – 11 NDE representatives from 8 departments – 1 representative from higher education
Understanding AQuESTT HAVE TO DOs GET TO DOs
Nebraska Data Literacies Data Comprehension Evaluation Did our response produce results?
What do the data show?
Data Use How should we respond?
Data Interpretation Why might this be?
Data Literacies Format
1.
Literacy a.
Concept i.
Indicators
Literacy 1 Data Comprehension: What do the data show? Concepts: a. Continuous Improvement Process b. District/School Profile c. Data Tools and Skills
Data Literacies
Data Literacies Definitions
Data Use Curriculum
Nebraska Data Literacies
Bernhardt, V.L. (2013)
Data Analysis for Continuous School Improvement (Third Edition) New York, NY: Routledge
Data Literacy 1 What do the data show?
Where are we now? Data Literacy 2 Why might this be? Data Literacy 2 Why might this be?
Where do we want to be?
Data Literacy 4 Did our response produce results?
Is what we are doing making a difference?
How did we get to where we are?
Data Literacy 3 How should we respond?
How are we going to get to where we want to be?
Using Data for Continuous Improvement Using Results Continuous School Improvement
Continuous Improvement Models NEBRASKA MODEL FOR CONTINUOUS IMPROVEMENT
Nebraska Frameworks
AdvancED
Rule 10: Regulations and Procedures for the Accreditation of Schools 009 Continuous School Improvement • 009.01A2 Collection and analysis of data about student performance, demographics, learning climate, and former high school students.
AdvancED • Standard 5: Using Results for Continuous Improvement
NDE Continuous School Improvement Workshops • Learn about the Nebraska Data Literacies • Learn data analysis techniques • Define appropriate action steps based on data analysis • Retrieve student data submitted to NDE via DRS
NDE Continuous School Improvement Workshops 2015-2016 Workshops • September 22 & 23 – Norfolk • September 28 & 29 – Gering • October 15 & 16 – Lincoln • October 19 & 20 – Omaha • October 26 & 27 – Kearney
Each school will receive a copy of Data Analysis for Continuous School Improvement
Perceptual Data: Purpose 1.
Understand the role that perceptual data plays
2.
Establish a process for perceptual data collection
3.
Share existing data collection resources
Role of Perceptual Data Why Survey? It is REQUIRED! • AdvancED requires the administration of ASSIST surveys within 1 year of the External Review • Nebraska Frameworks should utilize perceptual data to be a part of the district profile for the external visit
It is One of the Four Lenses Multiple Measures of Data • • • •
Demographics School Processes Student Learning Perceptions
It Answers the Question ‘How do we do business?’ Where we are now? What do students, staff, and parents think about the learning environment?
Perceptions impact beliefs, behaviors, and, to an extent, outcomes.
Collection Process 1. 2. 3. 4. 5. 6. 7. 8. 9.
Identify the Purpose Determine the Content* Develop or Select the Instrument* Pilot the Instrument* Field the Instrument/Collect the Data Analyze Results Report Results* - data displays Share and Review Results Use Results
*NDE/ESU Perceptual Survey Service
Identify the Purpose 1.
What do you want to learn?
2.
Will it be important to monitor changes in perceptions over time? How often? For how long?
3.
How will you use your survey results?
Determine the Content • What do you need to know? • What do you want to know? • Which stakeholders’ perspectives are relevant and should be considered?
Select/Develop and Pilot the Instrument 1. Who is the pilot group? 2. Is the instrument working the way you want it to work?
Collect the Data ●
How will the survey be administered? (paper/pencil, web-based)
●
When will the survey be administered?
Available Resources What instruments have already been developed? 1.
ASSIST Surveys (AdvancED)
1.
State Surveys (NDE)
Example Survey Instruments Education For The Future Victoria Bernhardt
Example Survey Instruments
Survey Constructs What is measured? Some constructs that are measured by perceptual surveys include the following: For Students
For Staff
For Parents
Safety
School Environment
Children’s Safety
Freedom of Choice
Administrative Support
School Staff
Relationship with Teachers
Staff Morale
School Assessment
Relationship with Other Students
Relationship with Students Learning Environment
School Performance
Student Achievement
Family Support
NDE/ESU Request Form ▼Click to view
How is the Data Displayed and Organized? • The data is graphed to objectively and fairly demonstrate the results.
• The goal of data display is to inform the audience and create consistent understanding.
Displaying Perceptual Data Results Page 248
Data Interpretation 1. How can the results be analyzed to show the information gleaned from the survey? 2. What did we learn about our learning environment? 3. What connections can be made when correlated with other data sources (i.e., demographic, school processes, student learning)?
Using the Results • How do these results help the CIP process? • What can you learn from these results? • How do we change perceptions?
Reporting the Results • How are the results going to be shared? • When are the results going to be shared? • When/how do you report changes that were made due to these results?
Analyzing Text Data • Responses to the two open-ended questions are also provided as part of the NDE/ESU Perceptual Survey Service. • Open source data analysis software is available. Contact your local ESU for more information.
Analyzing Text Data Protocol
Pages 242-243
NDE/ESU Perceptual Surveys • Ability to administer surveys will continue to be available in the future. • Perceptual data survey services will continue to improve and expand. • A vision is in place to develop minisurveys by constructs (main ideas being measured).
NDE/ESU Data Cadre Assistance The NDE Data, Research, and Evaluation (DRE) Team or ESU Data Cadre can assist you with ● ● ● ●
New Surveys Quality Instrument review Survey data analysis Survey management process assistance
● ● ●
Sources of survey error Survey software recommendations Survey question and design development
THINK BROADER Holistic view of each student Equal access to information
BE BETTER Continuous improvement Sharing of best practices Focused on every student, every day
Collective impact
EXPERIENCE BOLDER New system for measurement Customized for Nebraska Outcome-based approach
Thank you!
