Overview of CAEP Accreditation Pathway
Overview of the CAEP Accreditation Process Nate Thomas, Accreditation Associate
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Session Outline CAEP process—general framework with common elements Accreditation Pathways Review of Assessments Self-study Site visit Accreditation Decisions Annual Reporting Process Resources * Note that this presentation is specific to visits using the 2013 CAEP Standards CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Three Pathways • Continuous Improvement (CI) • Transformation Initiative (TI) • Inquiry Brief (IB) Pathways differ chiefly in what is submitted for the selfstudy: • Format • Addressing the Standards • Demonstrating quality assurance
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive characteristics: CI • Format: structured report • Addressing the standards: EPPs write directly to the standards with evidence and supporting narrative • Demonstrating quality assurance: The EPP develops and implements a data-driven Continuous Improvement Plan that focuses on improvement with respect to a selected Standard, Standard component, or cross-cutting theme
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive characteristics: TI • Format: structured report • Addressing the standards: The EPP writes directly to the standards with evidence and supporting narrative • Demonstrating quality assurance: The EPP develops a Transformation Initiative Plan (sometimes in consortium with states, schools, or other collaborators) for a rigorous research investigation of an aspect of educator preparation that will inform the profession and/or offer research-proven models for replication of promising practices CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive Characteristics: IB • Format: research monograph • Addressing the standards: The EPP makes claims consistent with its own goals and mission about the competence of its completers and align the claims to the Standards • Demonstrating quality assurance: The EPP describes its quality assurance system and conducts an internal audit to determine whether the system is functioning as designed
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Review of Assessments Up to Three Years Prior • Improve the quality of assessments used by EPPs to evaluate and report candidate/completer performance • EPP-wide assessments/surveys reviewed by CAEP up to three years prior to submitting the Self-Study Report • The intent is to allow EPPs
time to improve their assessments/surveys and scoring guides provide more precise feedback to candidates improve the program’s ability to analyze data for evidence leading to continuous improvement, and ensure consistency among evaluators using the same assessment. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Proprietary Assessments • Licensure test, edTPA, PPAT, VAM, and other standardized state or national level assessments • Proprietary assessment instruments are exempt from being reviewed three years prior • List proprietary assessments with validity and reliability information, when administered, and justification for selection
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
EPP Created Assessments • Specific assessments created or modified by the EPP and used across all discipline specific content areas in the EPP • Student teaching observation instruments, exit surveys, teacher work samples, portfolios, etc. • Submit the assessment instrument up to three years prior to the visit
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Steps to Assessment Review Request shell up to 3 years prior to scheduled visit
Attach table identifying all assessments & surveys
Receive feedback on each assessment/survey
Answer three questions regarding each assessment/survey
Attach each assessment & survey with scoring guide
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Questions • For each assessment or survey, the EPP will answer the following three questions: How was the assessment developed? How will the quality of the assessment/evidence be determined or assured including trustworthiness/validity and consistency/reliability? What criteria of success have been established or measured for scoring guides and survey data?
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Reviewers • CAEP assigns a lead reviewer and two additional reviewers for each EPP submitting a report. • Reviewers are specifically trained on the criteria for quality assessments, scoring guides, surveys, portfolios, etc • Reviewers provide feedback to EPPs on assessments.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Feedback Report • Reviewers provide feedback on assessments/surveys/portfolios the alignment of assessments, surveys, and portfolios to state standards the quality of the responses to the three questions answered for each survey or assessment.
• The state and CAEP site visitors receive copies of the feedback report and CAEP site visitors have access to the report when reviewing the Self-Study Report.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Revised Assessment Data • If EPPs revise or create new assessments or surveys based on the feedback, the data requirements are reduced for those new or revised assessments. • New or revised assessments require at least one cycle of data. • One cycle of data is defined as one collection of data using the new or revised assessment.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • EPP created assessments must align with the state standards and provide evidence that candidates are meeting the standard. • Assessment items need to align with the standard, and address the range of knowledge, skills, and dispositions delineated in the standard. • The scoring guide and assessment must reflect the level of difficulty, degree of challenge, and the language of the standard. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • Assessments and scoring guides must be specific and provide meaningful feedback to candidates’ on their performance. Distinct level of candidate performance must be defined. • Assessments are free of bias. • Assessments are more than mere checklists or rating scales where evaluators circle or assign a number for each item CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • Assessments must be capable of yielding approximately the same values across raters. • Assessments must measure what they purport to measure.
Note: Detailed criteria, guidance, and review guides will be made available to EPPs. CAEP views this process as capacity building CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: Common Elements • EPP context • Evidence • Characterization of the quality of the evidence • Discussion of results and their implications • Demonstration of quality assurance • Evidence of integration of cross-cutting themes of diversity and technology
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: CI/TI • EPP context • Evidence uploaded for each standard • Questions/prompts specific to the standard about the source of evidence – questions or prompts are specific to the type of evidence and the standard • Response to previous Area(s) for Improvement • Submission of CI or TI plan
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The CI Plan • A description of the focal area for continuous improvement - standard(s)/component(s)/themes • Rationale for selecting the focal area – Why? What are the baseline data? What are the goals • Plan for Continuous Improvement – what you are going to improve on and how • Evidence of success – emphasis on data quality • Progress will be reported annually by the EPP and evaluated during the subsequent accreditation visit to determine if components 5.3 and 5.4 of Standard 5 are satisfied CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The TI Proposal • Significance of the Project • Quality of the Project Design • Quality of the Research Design • Capacity to Conduct the Initiative • Progress will be reported annually by the EPP and evaluated during the subsequent accreditation visit to determine if components 5.3 and 5.4 of Standard 5 are satisfied
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: IB • EPP context • Evidence identified for each claim • Discussion of rationale and reliability/validity of each source of evidence with respect to the claim it is supporting what is it what evidence is available regarding its quality what criteria has been established for successful performance (and why)
• Presentation of results • Discussion of results and their implications what do the reported results mean how are results used in improvement
• Report on internal audit CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The IB Internal Audit • Description of the quality assurance system
Curriculum Faculty Facilities/Resources Candidates Clinical Partnerships
• Description of the procedure followed in conducting the internal audit • Presentation of the findings, the conclusions that faculty draws from the findings, and a discussion of the implications for the program. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Site Visit • Activities: site visitors will look at documentation and conduct interviews of various parties involved in the EPP • Purpose: site visitors will evaluate the accuracy and quality of the evidence • Result: site visit report and EPP response • Site visitors will recommend Areas for Improvement (AFI) and Stipulations • CI/TI: examination of progress on the development and implementation of the CI Plan or TI Plan • IB: examination and verification of internal audit CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
AFIs & Stipulations • Area for Improvement: Identifies a weakness in the evidence for a component or a standard. A single AFI is usually not of sufficient severity that it leads to an unmet standard. • Stipulation: Deficiency related to one or more components or a CAEP standard. A stipulation is of sufficient severity that a standard is determined to be unmet. For EPPs seeking to continue their accreditation, a stipulation must be corrected within two years to retain accreditation. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Accreditation Decision • Accreditation Commissions: Continuous Improvement Commission Inquiry Brief Commission Transformation Initiative Commission
• 1st review by commission panel; 2nd review by joint panel • Entire Accreditation Council determines accreditation decision • EPP representative is invited to observe initial review of case (in-person or virtually) CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Levels of Accreditation • Full Accreditation - awarded to providers that meet CAEP guidelines for all five standards • Probationary Accreditation - awarded to providers that meet or surpass CAEP guidelines in four standards, but fall below in one of the standards • Denial of Accreditation - for providers that fall below CAEP guidelines in two or more standards
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Annual Reporting • Due in April each year • Data reporting period aligned with AACTE PEDS Report • Sections: AIMS Profile Program Completers Substantive Changes Display of Candidate Performance Data Candidate and Program Measures (Initial Licensure Programs; some measures required and others optional)
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Resources • CAEP Accreditation Community – online learning community to support EPPs implementing CAEP Standards; full launch in 2015 • Program Review staff – available to support preparing for Assessments Review and SPA Reviews • Accreditation staff – available to support for the accreditation visit, Self-Study submission, Annual Report CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Other Resources • Updated Guide to Accreditation available in 2015 • Spring 2015 CAEP Conference – Denver, Colorado April 9-10, 2015 Consultations available with staff
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Contact [email protected] Telephone: (202) 223-0077
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Questions?
Q& A
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Session Outline CAEP process—general framework with common elements Accreditation Pathways Review of Assessments Self-study Site visit Accreditation Decisions Annual Reporting Process Resources * Note that this presentation is specific to visits using the 2013 CAEP Standards CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Three Pathways • Continuous Improvement (CI) • Transformation Initiative (TI) • Inquiry Brief (IB) Pathways differ chiefly in what is submitted for the selfstudy: • Format • Addressing the Standards • Demonstrating quality assurance
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive characteristics: CI • Format: structured report • Addressing the standards: EPPs write directly to the standards with evidence and supporting narrative • Demonstrating quality assurance: The EPP develops and implements a data-driven Continuous Improvement Plan that focuses on improvement with respect to a selected Standard, Standard component, or cross-cutting theme
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive characteristics: TI • Format: structured report • Addressing the standards: The EPP writes directly to the standards with evidence and supporting narrative • Demonstrating quality assurance: The EPP develops a Transformation Initiative Plan (sometimes in consortium with states, schools, or other collaborators) for a rigorous research investigation of an aspect of educator preparation that will inform the profession and/or offer research-proven models for replication of promising practices CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Distinctive Characteristics: IB • Format: research monograph • Addressing the standards: The EPP makes claims consistent with its own goals and mission about the competence of its completers and align the claims to the Standards • Demonstrating quality assurance: The EPP describes its quality assurance system and conducts an internal audit to determine whether the system is functioning as designed
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Review of Assessments Up to Three Years Prior • Improve the quality of assessments used by EPPs to evaluate and report candidate/completer performance • EPP-wide assessments/surveys reviewed by CAEP up to three years prior to submitting the Self-Study Report • The intent is to allow EPPs
time to improve their assessments/surveys and scoring guides provide more precise feedback to candidates improve the program’s ability to analyze data for evidence leading to continuous improvement, and ensure consistency among evaluators using the same assessment. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Proprietary Assessments • Licensure test, edTPA, PPAT, VAM, and other standardized state or national level assessments • Proprietary assessment instruments are exempt from being reviewed three years prior • List proprietary assessments with validity and reliability information, when administered, and justification for selection
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
EPP Created Assessments • Specific assessments created or modified by the EPP and used across all discipline specific content areas in the EPP • Student teaching observation instruments, exit surveys, teacher work samples, portfolios, etc. • Submit the assessment instrument up to three years prior to the visit
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Steps to Assessment Review Request shell up to 3 years prior to scheduled visit
Attach table identifying all assessments & surveys
Receive feedback on each assessment/survey
Answer three questions regarding each assessment/survey
Attach each assessment & survey with scoring guide
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Questions • For each assessment or survey, the EPP will answer the following three questions: How was the assessment developed? How will the quality of the assessment/evidence be determined or assured including trustworthiness/validity and consistency/reliability? What criteria of success have been established or measured for scoring guides and survey data?
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Reviewers • CAEP assigns a lead reviewer and two additional reviewers for each EPP submitting a report. • Reviewers are specifically trained on the criteria for quality assessments, scoring guides, surveys, portfolios, etc • Reviewers provide feedback to EPPs on assessments.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Assessment Feedback Report • Reviewers provide feedback on assessments/surveys/portfolios the alignment of assessments, surveys, and portfolios to state standards the quality of the responses to the three questions answered for each survey or assessment.
• The state and CAEP site visitors receive copies of the feedback report and CAEP site visitors have access to the report when reviewing the Self-Study Report.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Revised Assessment Data • If EPPs revise or create new assessments or surveys based on the feedback, the data requirements are reduced for those new or revised assessments. • New or revised assessments require at least one cycle of data. • One cycle of data is defined as one collection of data using the new or revised assessment.
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • EPP created assessments must align with the state standards and provide evidence that candidates are meeting the standard. • Assessment items need to align with the standard, and address the range of knowledge, skills, and dispositions delineated in the standard. • The scoring guide and assessment must reflect the level of difficulty, degree of challenge, and the language of the standard. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • Assessments and scoring guides must be specific and provide meaningful feedback to candidates’ on their performance. Distinct level of candidate performance must be defined. • Assessments are free of bias. • Assessments are more than mere checklists or rating scales where evaluators circle or assign a number for each item CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Criteria for Assessments & Scoring Guides • Assessments must be capable of yielding approximately the same values across raters. • Assessments must measure what they purport to measure.
Note: Detailed criteria, guidance, and review guides will be made available to EPPs. CAEP views this process as capacity building CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: Common Elements • EPP context • Evidence • Characterization of the quality of the evidence • Discussion of results and their implications • Demonstration of quality assurance • Evidence of integration of cross-cutting themes of diversity and technology
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: CI/TI • EPP context • Evidence uploaded for each standard • Questions/prompts specific to the standard about the source of evidence – questions or prompts are specific to the type of evidence and the standard • Response to previous Area(s) for Improvement • Submission of CI or TI plan
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The CI Plan • A description of the focal area for continuous improvement - standard(s)/component(s)/themes • Rationale for selecting the focal area – Why? What are the baseline data? What are the goals • Plan for Continuous Improvement – what you are going to improve on and how • Evidence of success – emphasis on data quality • Progress will be reported annually by the EPP and evaluated during the subsequent accreditation visit to determine if components 5.3 and 5.4 of Standard 5 are satisfied CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The TI Proposal • Significance of the Project • Quality of the Project Design • Quality of the Research Design • Capacity to Conduct the Initiative • Progress will be reported annually by the EPP and evaluated during the subsequent accreditation visit to determine if components 5.3 and 5.4 of Standard 5 are satisfied
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: IB • EPP context • Evidence identified for each claim • Discussion of rationale and reliability/validity of each source of evidence with respect to the claim it is supporting what is it what evidence is available regarding its quality what criteria has been established for successful performance (and why)
• Presentation of results • Discussion of results and their implications what do the reported results mean how are results used in improvement
• Report on internal audit CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Writing the Self-Study: The IB Internal Audit • Description of the quality assurance system
Curriculum Faculty Facilities/Resources Candidates Clinical Partnerships
• Description of the procedure followed in conducting the internal audit • Presentation of the findings, the conclusions that faculty draws from the findings, and a discussion of the implications for the program. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Site Visit • Activities: site visitors will look at documentation and conduct interviews of various parties involved in the EPP • Purpose: site visitors will evaluate the accuracy and quality of the evidence • Result: site visit report and EPP response • Site visitors will recommend Areas for Improvement (AFI) and Stipulations • CI/TI: examination of progress on the development and implementation of the CI Plan or TI Plan • IB: examination and verification of internal audit CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
AFIs & Stipulations • Area for Improvement: Identifies a weakness in the evidence for a component or a standard. A single AFI is usually not of sufficient severity that it leads to an unmet standard. • Stipulation: Deficiency related to one or more components or a CAEP standard. A stipulation is of sufficient severity that a standard is determined to be unmet. For EPPs seeking to continue their accreditation, a stipulation must be corrected within two years to retain accreditation. CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
The Accreditation Decision • Accreditation Commissions: Continuous Improvement Commission Inquiry Brief Commission Transformation Initiative Commission
• 1st review by commission panel; 2nd review by joint panel • Entire Accreditation Council determines accreditation decision • EPP representative is invited to observe initial review of case (in-person or virtually) CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Levels of Accreditation • Full Accreditation - awarded to providers that meet CAEP guidelines for all five standards • Probationary Accreditation - awarded to providers that meet or surpass CAEP guidelines in four standards, but fall below in one of the standards • Denial of Accreditation - for providers that fall below CAEP guidelines in two or more standards
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Annual Reporting • Due in April each year • Data reporting period aligned with AACTE PEDS Report • Sections: AIMS Profile Program Completers Substantive Changes Display of Candidate Performance Data Candidate and Program Measures (Initial Licensure Programs; some measures required and others optional)
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Resources • CAEP Accreditation Community – online learning community to support EPPs implementing CAEP Standards; full launch in 2015 • Program Review staff – available to support preparing for Assessments Review and SPA Reviews • Accreditation staff – available to support for the accreditation visit, Self-Study submission, Annual Report CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Other Resources • Updated Guide to Accreditation available in 2015 • Spring 2015 CAEP Conference – Denver, Colorado April 9-10, 2015 Consultations available with staff
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Contact [email protected] Telephone: (202) 223-0077
CONNECT WITH CAEP |www.CAEPnet.org| Twitter: @CAEPupdates
Questions?
Q& A
