Continuing Cycles of Improvement
Continuing Cycles of Improvement Improving Student Performance Across Time Dr. Ann Saurbier Walsh College Troy MI
Continuing Cycles of Improvement Improving Student Performance across Time • Four-year time span – 2011 – 2014 • MBA Capstone course • Specifically targeted competency • Written Communication • Actions taken • Online course content and activities • In-class activities • Results of actions • Closing – but continuing - the loop activities
@ACBSPAccredited #ACBSP2015
Walsh College • Established 1922 • Professional accounting school • Night courses for working adults • Legacy • ‘practical’ business education for working adults
• Upper division undergraduate / graduate programs • Headcount < 4,000 • 91% Part-time attendance
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Walsh College Mission • Provide a relevant business education that integrates theory and application to prepare graduates for successful careers through: • Faculty who bring meaningful scholarship, academic credentials and professional work experience to the learning process • Close ties with the business community • A supportive environment that inspires student achievement
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Walsh College Core Competencies • Problem Solving • Research • Written Communication • Oral Communication • Master a Business Discipline
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Core Competency • Organization • Tone, word choice, and style • Sources and evidence • Grammar and mechanics • Format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Assessment of Written Communication – MBA Program • Cycle • Annually - End of the Fall term • Assessment Course • MBA Capstone – Strategic Management • Assignment • Final Exam / Final Paper • Case-based Organizational Analysis
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement MBA Capstone Assignment Final Exam / Researched Paper • Select a Fortune 500 company • Analyze the organization’s environment • Analyze the organization’s financial position • Determine the strategic issues facing the company • As a consultant hired by that organization • Propose strategic recommendations to the Board of Directors
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2011 – 2012 Identifiable Areas for Improvement • Category Gap • Sources/Evidence • Formatting @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement
• Modality Gap Online performance vs. On Ground performance
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Identifiable Areas for Improvement Written Communication Common Struggles? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement
‘Bad’ news = ‘Good’ news Data = Actionable Areas for Improvement • Plan for Improvement • Target specific categories • Target modality differences @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2011 and 2012 Written Communication Results Criteria
Issue
Sources are relevant, verifiable, and authoritative.
Poor source quality
Sources are incorporated and attributed properly.
Improper/no attribution
Compelling evidence is provided to support claims and perspectives.
Limited number of sources Low quality sources
Statements of fact and opinion are clearly differentiated.
Poor attribution Limited analysis
Document has a professional appearance and is appropriate for the intended audience.
Poor section headings Not written for ‘audience’
Format meets the assignment requirements.
‘Book report’ rather than organizational analysis
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Issue Disconnection • Assignment instructions/outcomes • Targeted Student Learning Outcome Not ‘bad’ assignment Not improper SLO Missing • direct tie of Assignment to SLO
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Driven Improvement Plan Action Taken – Assignment Revision 1. Specified Connections between Problem Solving and Written Communication 2. Revised Assignment Instructions 3. Provided Guided Instruction
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #1 Clarify – and specify – the connection between the assignment’s focus on Problem Solving and Written Communication • Retitled Assignment • From – Research Paper • To – Analytical Organizational Assessment
• Focused Attention Directly on the ‘Author’ and ‘Audience’ • YOUR Recommendations to the Board of Directors
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #2 Revised Assignment Instructions 1. Required page minimums/maximums by section 2. Provided specific requirements on ‘allowable’ sources • High quality sources • Number /type of websites allowed • Highlighted use of APA format for source referencing/citations
3. Embedded Core Competency Rubric INTO instructions
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #3
Guided Instruction
1. Sources / Evidence • Additional Online Content • Formatting Conventions/Examples 2. Format • Do / Don’t areas of focus - by paper section
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Identifiable Methods Of Improvement For Issues Identified Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Results - 2013
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Analysis – 2013 Results Differences Sources/Evidence • On ground vs. Online Format • Online vs. On ground
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2013 Identifiable Areas for Improvement Modality Gap • Sources/Evidence
Sources/ Evidence Format
• On ground sections outperformed Online
• Formatting • Online sections outperformed On ground
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Differences On Ground • In class student questions • Appropriate sources • Source formatting
Online • Writing proficiency of Online students • WRITTEN ‘practice’ cases • vs. Presented practice cases
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Driven Improvement Plan – Phase II Action Taken – Assignment Revision 1. Revised formative assignments – all modalities 2. Scaffolded content mastery AND written communication mastery 3. Fundamentally revised summative assessment assignment
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Actions #1 and #2 Revised formative assignments / Scaffolded mastery • Semester long in-depth case analysis • Weekly concept application writing assignments – all modalities • Structured ‘discussion’ of concept application – all modalities
• Scenario-based weekly concept application • Mirroring exam format
• Executive Summary of weekly concept analysis • Mirroring exam format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #3
Fundamental Summative Assessment Revision – Performance Task 1. Connected Mid-term / Final papers
• Scaffolded writing – two part written assignment • Mid-term conclusions become basis for Final
2. Feedback loops
• Feedback from Mid-term exam applied to Final
3. Personal inclusion scenario
• Student in the direct role of decision-maker • ‘Personal’ contact with the CEO
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improving on Good Ideas For Solutions Identified Phase II Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Results - 2014
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Analysis – 2014 Results Closed the Gap Sources/Evidence • On ground vs. Online Format • Online vs. On ground
Sources/ Evidence
Format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance - 2011/12 to 2014 Written Communication Sources / Evidence • 57% increase Format • 13% increase @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance across Time
Scaffold Learning
Rebuild Assignment/SLO Connections @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance across Time
Personalized Scenario Revise Assignment Instructions
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing
the Loop
Phase III Plans • Revise Institutional Written Communication Rubric • better articulate/target results desired • Continue assignment revisions • Continue online content revisions • Mirror information shared in the classroom
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing the Loop Building Successes Forward Next Step Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing the Loop Criterion 4.4 use results to improve educational processes in the interest of continuously improving student learning outcomes
• Identify the problem • Target the issue • Experiment • Assess results
Repeat Assess Experiment Target Identify
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Dr. Ann Saurbier Walsh College
[email protected]
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improving Student Performance across Time • Four-year time span – 2011 – 2014 • MBA Capstone course • Specifically targeted competency • Written Communication • Actions taken • Online course content and activities • In-class activities • Results of actions • Closing – but continuing - the loop activities
@ACBSPAccredited #ACBSP2015
Walsh College • Established 1922 • Professional accounting school • Night courses for working adults • Legacy • ‘practical’ business education for working adults
• Upper division undergraduate / graduate programs • Headcount < 4,000 • 91% Part-time attendance
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Walsh College Mission • Provide a relevant business education that integrates theory and application to prepare graduates for successful careers through: • Faculty who bring meaningful scholarship, academic credentials and professional work experience to the learning process • Close ties with the business community • A supportive environment that inspires student achievement
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Walsh College Core Competencies • Problem Solving • Research • Written Communication • Oral Communication • Master a Business Discipline
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Core Competency • Organization • Tone, word choice, and style • Sources and evidence • Grammar and mechanics • Format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Assessment of Written Communication – MBA Program • Cycle • Annually - End of the Fall term • Assessment Course • MBA Capstone – Strategic Management • Assignment • Final Exam / Final Paper • Case-based Organizational Analysis
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement MBA Capstone Assignment Final Exam / Researched Paper • Select a Fortune 500 company • Analyze the organization’s environment • Analyze the organization’s financial position • Determine the strategic issues facing the company • As a consultant hired by that organization • Propose strategic recommendations to the Board of Directors
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2011 – 2012 Identifiable Areas for Improvement • Category Gap • Sources/Evidence • Formatting @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement
• Modality Gap Online performance vs. On Ground performance
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Identifiable Areas for Improvement Written Communication Common Struggles? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement
‘Bad’ news = ‘Good’ news Data = Actionable Areas for Improvement • Plan for Improvement • Target specific categories • Target modality differences @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2011 and 2012 Written Communication Results Criteria
Issue
Sources are relevant, verifiable, and authoritative.
Poor source quality
Sources are incorporated and attributed properly.
Improper/no attribution
Compelling evidence is provided to support claims and perspectives.
Limited number of sources Low quality sources
Statements of fact and opinion are clearly differentiated.
Poor attribution Limited analysis
Document has a professional appearance and is appropriate for the intended audience.
Poor section headings Not written for ‘audience’
Format meets the assignment requirements.
‘Book report’ rather than organizational analysis
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Issue Disconnection • Assignment instructions/outcomes • Targeted Student Learning Outcome Not ‘bad’ assignment Not improper SLO Missing • direct tie of Assignment to SLO
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Driven Improvement Plan Action Taken – Assignment Revision 1. Specified Connections between Problem Solving and Written Communication 2. Revised Assignment Instructions 3. Provided Guided Instruction
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #1 Clarify – and specify – the connection between the assignment’s focus on Problem Solving and Written Communication • Retitled Assignment • From – Research Paper • To – Analytical Organizational Assessment
• Focused Attention Directly on the ‘Author’ and ‘Audience’ • YOUR Recommendations to the Board of Directors
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #2 Revised Assignment Instructions 1. Required page minimums/maximums by section 2. Provided specific requirements on ‘allowable’ sources • High quality sources • Number /type of websites allowed • Highlighted use of APA format for source referencing/citations
3. Embedded Core Competency Rubric INTO instructions
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #3
Guided Instruction
1. Sources / Evidence • Additional Online Content • Formatting Conventions/Examples 2. Format • Do / Don’t areas of focus - by paper section
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Identifiable Methods Of Improvement For Issues Identified Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Results - 2013
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Analysis – 2013 Results Differences Sources/Evidence • On ground vs. Online Format • Online vs. On ground
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement 2013 Identifiable Areas for Improvement Modality Gap • Sources/Evidence
Sources/ Evidence Format
• On ground sections outperformed Online
• Formatting • Online sections outperformed On ground
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Differences On Ground • In class student questions • Appropriate sources • Source formatting
Online • Writing proficiency of Online students • WRITTEN ‘practice’ cases • vs. Presented practice cases
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Driven Improvement Plan – Phase II Action Taken – Assignment Revision 1. Revised formative assignments – all modalities 2. Scaffolded content mastery AND written communication mastery 3. Fundamentally revised summative assessment assignment
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Actions #1 and #2 Revised formative assignments / Scaffolded mastery • Semester long in-depth case analysis • Weekly concept application writing assignments – all modalities • Structured ‘discussion’ of concept application – all modalities
• Scenario-based weekly concept application • Mirroring exam format
• Executive Summary of weekly concept analysis • Mirroring exam format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Action #3
Fundamental Summative Assessment Revision – Performance Task 1. Connected Mid-term / Final papers
• Scaffolded writing – two part written assignment • Mid-term conclusions become basis for Final
2. Feedback loops
• Feedback from Mid-term exam applied to Final
3. Personal inclusion scenario
• Student in the direct role of decision-maker • ‘Personal’ contact with the CEO
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improving on Good Ideas For Solutions Identified Phase II Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Written Communication Results - 2014
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Data Analysis – 2014 Results Closed the Gap Sources/Evidence • On ground vs. Online Format • Online vs. On ground
Sources/ Evidence
Format
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance - 2011/12 to 2014 Written Communication Sources / Evidence • 57% increase Format • 13% increase @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance across Time
Scaffold Learning
Rebuild Assignment/SLO Connections @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Improved Student Performance across Time
Personalized Scenario Revise Assignment Instructions
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing
the Loop
Phase III Plans • Revise Institutional Written Communication Rubric • better articulate/target results desired • Continue assignment revisions • Continue online content revisions • Mirror information shared in the classroom
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing the Loop Building Successes Forward Next Step Ideas? @ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Continuing the Loop Criterion 4.4 use results to improve educational processes in the interest of continuously improving student learning outcomes
• Identify the problem • Target the issue • Experiment • Assess results
Repeat Assess Experiment Target Identify
@ACBSPAccredited #ACBSP2015
Continuing Cycles of Improvement Dr. Ann Saurbier Walsh College
[email protected]
@ACBSPAccredited #ACBSP2015
