Improving the Mathematics Placement Process
Improving the Mathematics Placement Process Helen E. Burn and Shannon Waits, Highline College Annual Conference of the American Mathematical Association of Two-Year Colleges November 19-22, 2015 New Orleans, Louisiana Session S131, Saturday, November 21, 10:45-11:35 AM Abstract Ensuring that students are accurately placed into their initial mathematics course is a priority given that placement into developmental mathematics reduces a student’s probability of completing a degree. This presentation describes components of a research-based redesigned mathematics placement process developed through a collaboration with mathematics faculty and placement center staff. The goals of this session are: I. Examine research on institutional placement practices and challenges II. Explain components of comprehensive redesign effort undertaken at Highline College III. Describe evaluation plan for the redesign
I.
Relevant research on institutional placement practices and challenges
After a decade-long developmental mathematics reform movement that resulted in technologymediated acceleration models that shorten the mathematics sequences or pathways models that tailor course outcomes to students’ educational or career goals, concerns persist about the expense of remediation compared to the benefit to students in terms of outcomes (Bailey, Jeong, & Cho, 2008; Cullinane & Treisman, 2010). Consequently, the national conversation has broadened to include institutional policy and practice around assessment and placement (A&P) of students into their initial mathematics course (Couturier & Cullinane, 2015; Hughes & Scott-Clayton, 2011; Melguizo, Kosiewicz, Prather, & Bos, 2014; Rodriguez, Bowden, Belfield, & Scott-Clayton, 2014). Placing into developmental mathematics reduces a student’s probability of completing college-level math and thus hinders national efforts to boost student degree attainment (Adelman, 1999, 2004; Attewell, Lavin, Domina, & Levey, 2006; Bahr, 2010; Bettinger & Long, 2005; Cullinane & Treisman, 2010). For example, Melguizo et al.’s (2014) study of nearly 80,000 transcripts of students in the Los Angeles Community College District revealed that 45% of the students who placed five levels below college-level math never enrolled in a math class compared to 18% of students who placed one level below. Thus, ensuring that students are accurately placed into their initial mathematics course is an institutional imperative. As a result, research has shifted to examining the validity and reliability of A&P instruments and to overall institutional A&P policies and practices, including student awareness of the impact of initial placement (Fay, Bickerstaff, & Hodara, 2013; Rodriguez, et al., 2014; Hughes and ScottClayton, 2011). The components of Highline College’s math placement redesign drew from research recommending that math placement should be an educative process that takes a team approach and that multiple measures should be used for student placement (e.g., Couturier & Cullinane, 2015; Fay et al., 2013; Hughes & Scott-Clayton, 2011; Melguizo et al., 2014). Research suggests students tend to take placement tests without understanding their purpose or high-stakes nature (Fay et al., 2013; Hughes & Scott-Clayton, 2011). Because of this, a key component of Highline College’s math placement redesign is to educate students about the purpose and implications of math placement and to bolster students’ tendency to study prior to taking a math placement test. Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
II.
Components of comprehensive math placement redesign at Highline College1
Provide an educational and participatory math placement process that is focused on students’ educational pathway Our goal is that students are actively involved in their placement experience and have sufficient information to make informed decisions about their approach to the process. When students are involved and communicated with, it allows us to contextualize the assessment based on their program of interest, their math pathway. This ensures students’ goals are at the center of the placement process (Couturier & Cullinane, 2015). The Assessment and Placement Advisor is a new position created to improve communication with students about placement options, results, and next steps. A second new project position is an embedded math faculty position, to deliver brush-up workshops and math-specific advising for students before and after placement in the Placement and Testing Center (PTC). Use multiple measures, rather than single test score, to place students into math classes The recommendation to use multiple measures to evaluate students’ knowledge—including talking to students about their prior coursework or time commitment—emerged from concerns about the predictive validity of the primary tests currently used for placement (Hughes & Scott-Clayton, 2011). At Highline, once each student’s educational pathway is determined, multiple measures are used to assess their math skills, including: high school transcripts, GED scores, courses from other colleges, and other institutions’ placement test scores. Overall, we hope to reduce the number of students requiring a test. Use a math placement test that aligns with your math curriculum After a decade of redesigning Highline’s developmental math sequence, the math department opted to replace COMPASS with an instrument (MyMathTest: MMT) that was better aligned with the scope and sequence of mathematics topics in the curriculum. The college adopted MMT and created customized mathematics placement tests starting in 2013. After one year, we examined placement data and modified cut scores and test items. We have found it essential to have mathematics faculty involved with determining, maintaining, and evaluating assessment measures. Provide students with ways to “brush-up” prior to or after taking the placement test Students that place into developmental math upon initial assessment are routed to one of two brush-up workshop interventions: face-to-face or online. Both forms of brush-up leverage the individualized study plan created by MMT based on test results. The face-to-face brush-up workshops are offered 5 times per week. The online brush-up is an open-enrollment, Canvas mediated course currently being piloted. Fay et al. (2013) identified barriers to students studying, including lack of awareness of existing study materials, knowledge of how to study effectively, math confidence, and a belief that studying was a form of cheating. Further, in trying to allay students’ anxiety about testing, staff gave the false impression that there was no need to study. The redesign provides us with a unique opportunity to explore methods for bolstering students’ interest and willingness to engage in brush-up. Educate campus stakeholders about the redesign Campus advisers and faculty receive training and information about placement redesign so that students receive a unified message from staff and faculty they encounter during admissions and enrollment. The training includes face-to-face meetings as well as promotional material. It is noteworthy that Melguizo et al. (2014) found that students, educational advisers, and faculty tended to be unfamiliar with how multiple measures worked in practice, resulting in continued heavy reliance on the results of a single, standardized placement test.
1
This project is funded in part by a grant from College Spark Washington http://www.collegespark.org/
Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
III.
Project evaluation
Project Outcomes Increase the percentage of students who earn college-level math credit within two years of enrollment at Highline from 41% to 44%. Increase the percentage of eligible students who attain college level math credit within the current year from 30% to 33%.
Increase the number of students who transition from developmental math to college-level math within one year from 47% to 50% Decrease the percentage of enrolling students who place into developmental math courses from 83% to 75%.
Measurement Method College enrollment data tracking a cohort of students with Fall/Summer start Student Achievement Initiative data from the Washington State Board for Community and Technical Colleges http://www.sbctc.ctc.edu/college/e_studentachi evement.aspx Student Achievement Initiative data from the Washington State Board for Community and Technical Colleges Student Placement Data, for each fall cohort of enrolling students with placement data over the life of the grant.
In addition, the project evaluation will answer the following key questions: 1. 2. 3.
What impact does an educational and participatory placement process have on student behavior? Which elements of the placement process are working effectively? Is this process equitable?
Additional Measures Placement Testing Center (PTC) database. Any student visiting the PTC will answer three questions at intake. This tool is used to document the “assessment path” that is determined for the student. Students requiring MMT will be tracked in terms of whether they took the test or decided to study first. Data from educational planning staff. Surveys (pre- and post) and focus groups will be used to measure increases in educational planning staff awareness of the components of the redesign and shifts in student advising related to placement. Baseline surveys were conducted in summer of2013. Pre-post surveys of students. Students in math classes will be surveyed about their experience with assessment and placement. Baseline surveys were conducted in fall of 2013. Student focus groups. Throughout the project we will conduct focus groups to hear from students about their experiences with the placement process. MMT Takers. For the subgroup of students required to take MMT for placement, we will measure engagement and track activities through MMT Analytics and tools developed related to the Canvas Brush-up.
Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
References Adelman, C. (1999). Answers in the toolbox: Academic intensity, attendance patterns, and bachelor’s degree attainment. Washington, D.C.: Office of Educational Research and Improvement, U.S. Department of Education. Retrieved from http://www.ed/gov/pubs/Toolbox/toolbox.html Adelman, C. (2004). Principal indicators of student academic histories in post-secondary education, 1972-2000. Washington, DC: Institute of Education Sciences, U.S. Department of Education. Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. The Journal of Higher Education, 77(5), 886-924. Bailey, T., Jeong, D. W., & Cho, S.W. (2008). Referral, enrollment, and completion in developmental education sequence in community colleges (CCRC Research Brief No. 45). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/student-progression-through-developmentalsequences-brief.pdf Bahr, P. R. (2010). Revisiting the efficacy of postsecondary remediation: The moderating effects of depth/breath of deficiency. The Review of Higher Education, 233(2), 177-205. Bettinger, E. P., & Long, B. T. (2005). Addressing the needs of under-prepared students in higher education: Does college remediation work? (NBER Working Paper No. 11325). Retrieved from National Bureau of Economic Research Website: www.nber.org/papers/w11325 Couturier, L., & Cullinane, J. (2015) A Call to Action to Improve Math Placement Policies and Processes. Retrieved from http://www.jff.org/publications/call-action-improve-math-placement-policies-and-processes Cullinane, J., & Treisman, P. U. (2010). Improving developmental mathematics education in community colleges: A prospectus and early progress report on the Statway Initiative. Paper presented at the National Center for Postsecondary Research Conference, New York, N.Y. Retrieved from National Center for Postsecondary Research Wedsite: http://www.postsecondaryresearch.org/conference/downloads.html Fay, M.P., Bickerstaff, S., & Hodara, M. (2013). Why students do not prepare for math placement exams: Student perspectives (CCRC Research Brief No. 57). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/publications/whystudents-do-not-prepare.html Hughes, K. L. & Scott-Clayton, J. (2011). Assessing developmental assessment in community colleges. Community College Review, 39(4), 327-351, DOI 10.1177/0091552111426898 Melguizo, T., Kosiewicz, H., Prather, G., & Bos, J. (2014). How are community college students assessed and placed in developmental math? Grounding our understanding in reality. Journal of Higher Education, 85(5), 691722. Rodriguez, O., Bowden, B., Belfield, C., & Scott-Clayton, J. (September, 2014). Remedial placement testing in community colleges: What resources are required and what does it cost? (CCRC Working Paper No. 73). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/remedial-placement-testing-resources.pdf
Helen Burn ([email protected]) is an instructor of mathematics at Highline College (1994 to present), director of the Curriculum Research Group (2009 to present), and adjunct faculty in the College of Education at Seattle University (2009 to present). She has served as math department coordinator (1998-2000) and chair of the Pure and Applied Sciences division (2006-2009) at Highline. She holds an M.S. in mathematics from Western Washington University and a Ph.D. in higher education from the University of Michigan Center for the Study of Higher and Postsecondary Education. Shannon Waits ([email protected]) is the Director of Academic Assessment and Placement at Highline College, a new position created in 2013 to examine and guide the College's placement and assessment practices to ensure alignment with evidence-based practice, student-attainment goals, and college-wide values around educational equity. She holds an M.Ed. in Adult Education and a B.A. in Human Services. She has taught mathematics in ABE departments, King County Jail, and the YWCA's Young Parent Program. Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
I.
Relevant research on institutional placement practices and challenges
After a decade-long developmental mathematics reform movement that resulted in technologymediated acceleration models that shorten the mathematics sequences or pathways models that tailor course outcomes to students’ educational or career goals, concerns persist about the expense of remediation compared to the benefit to students in terms of outcomes (Bailey, Jeong, & Cho, 2008; Cullinane & Treisman, 2010). Consequently, the national conversation has broadened to include institutional policy and practice around assessment and placement (A&P) of students into their initial mathematics course (Couturier & Cullinane, 2015; Hughes & Scott-Clayton, 2011; Melguizo, Kosiewicz, Prather, & Bos, 2014; Rodriguez, Bowden, Belfield, & Scott-Clayton, 2014). Placing into developmental mathematics reduces a student’s probability of completing college-level math and thus hinders national efforts to boost student degree attainment (Adelman, 1999, 2004; Attewell, Lavin, Domina, & Levey, 2006; Bahr, 2010; Bettinger & Long, 2005; Cullinane & Treisman, 2010). For example, Melguizo et al.’s (2014) study of nearly 80,000 transcripts of students in the Los Angeles Community College District revealed that 45% of the students who placed five levels below college-level math never enrolled in a math class compared to 18% of students who placed one level below. Thus, ensuring that students are accurately placed into their initial mathematics course is an institutional imperative. As a result, research has shifted to examining the validity and reliability of A&P instruments and to overall institutional A&P policies and practices, including student awareness of the impact of initial placement (Fay, Bickerstaff, & Hodara, 2013; Rodriguez, et al., 2014; Hughes and ScottClayton, 2011). The components of Highline College’s math placement redesign drew from research recommending that math placement should be an educative process that takes a team approach and that multiple measures should be used for student placement (e.g., Couturier & Cullinane, 2015; Fay et al., 2013; Hughes & Scott-Clayton, 2011; Melguizo et al., 2014). Research suggests students tend to take placement tests without understanding their purpose or high-stakes nature (Fay et al., 2013; Hughes & Scott-Clayton, 2011). Because of this, a key component of Highline College’s math placement redesign is to educate students about the purpose and implications of math placement and to bolster students’ tendency to study prior to taking a math placement test. Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
II.
Components of comprehensive math placement redesign at Highline College1
Provide an educational and participatory math placement process that is focused on students’ educational pathway Our goal is that students are actively involved in their placement experience and have sufficient information to make informed decisions about their approach to the process. When students are involved and communicated with, it allows us to contextualize the assessment based on their program of interest, their math pathway. This ensures students’ goals are at the center of the placement process (Couturier & Cullinane, 2015). The Assessment and Placement Advisor is a new position created to improve communication with students about placement options, results, and next steps. A second new project position is an embedded math faculty position, to deliver brush-up workshops and math-specific advising for students before and after placement in the Placement and Testing Center (PTC). Use multiple measures, rather than single test score, to place students into math classes The recommendation to use multiple measures to evaluate students’ knowledge—including talking to students about their prior coursework or time commitment—emerged from concerns about the predictive validity of the primary tests currently used for placement (Hughes & Scott-Clayton, 2011). At Highline, once each student’s educational pathway is determined, multiple measures are used to assess their math skills, including: high school transcripts, GED scores, courses from other colleges, and other institutions’ placement test scores. Overall, we hope to reduce the number of students requiring a test. Use a math placement test that aligns with your math curriculum After a decade of redesigning Highline’s developmental math sequence, the math department opted to replace COMPASS with an instrument (MyMathTest: MMT) that was better aligned with the scope and sequence of mathematics topics in the curriculum. The college adopted MMT and created customized mathematics placement tests starting in 2013. After one year, we examined placement data and modified cut scores and test items. We have found it essential to have mathematics faculty involved with determining, maintaining, and evaluating assessment measures. Provide students with ways to “brush-up” prior to or after taking the placement test Students that place into developmental math upon initial assessment are routed to one of two brush-up workshop interventions: face-to-face or online. Both forms of brush-up leverage the individualized study plan created by MMT based on test results. The face-to-face brush-up workshops are offered 5 times per week. The online brush-up is an open-enrollment, Canvas mediated course currently being piloted. Fay et al. (2013) identified barriers to students studying, including lack of awareness of existing study materials, knowledge of how to study effectively, math confidence, and a belief that studying was a form of cheating. Further, in trying to allay students’ anxiety about testing, staff gave the false impression that there was no need to study. The redesign provides us with a unique opportunity to explore methods for bolstering students’ interest and willingness to engage in brush-up. Educate campus stakeholders about the redesign Campus advisers and faculty receive training and information about placement redesign so that students receive a unified message from staff and faculty they encounter during admissions and enrollment. The training includes face-to-face meetings as well as promotional material. It is noteworthy that Melguizo et al. (2014) found that students, educational advisers, and faculty tended to be unfamiliar with how multiple measures worked in practice, resulting in continued heavy reliance on the results of a single, standardized placement test.
1
This project is funded in part by a grant from College Spark Washington http://www.collegespark.org/
Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
III.
Project evaluation
Project Outcomes Increase the percentage of students who earn college-level math credit within two years of enrollment at Highline from 41% to 44%. Increase the percentage of eligible students who attain college level math credit within the current year from 30% to 33%.
Increase the number of students who transition from developmental math to college-level math within one year from 47% to 50% Decrease the percentage of enrolling students who place into developmental math courses from 83% to 75%.
Measurement Method College enrollment data tracking a cohort of students with Fall/Summer start Student Achievement Initiative data from the Washington State Board for Community and Technical Colleges http://www.sbctc.ctc.edu/college/e_studentachi evement.aspx Student Achievement Initiative data from the Washington State Board for Community and Technical Colleges Student Placement Data, for each fall cohort of enrolling students with placement data over the life of the grant.
In addition, the project evaluation will answer the following key questions: 1. 2. 3.
What impact does an educational and participatory placement process have on student behavior? Which elements of the placement process are working effectively? Is this process equitable?
Additional Measures Placement Testing Center (PTC) database. Any student visiting the PTC will answer three questions at intake. This tool is used to document the “assessment path” that is determined for the student. Students requiring MMT will be tracked in terms of whether they took the test or decided to study first. Data from educational planning staff. Surveys (pre- and post) and focus groups will be used to measure increases in educational planning staff awareness of the components of the redesign and shifts in student advising related to placement. Baseline surveys were conducted in summer of2013. Pre-post surveys of students. Students in math classes will be surveyed about their experience with assessment and placement. Baseline surveys were conducted in fall of 2013. Student focus groups. Throughout the project we will conduct focus groups to hear from students about their experiences with the placement process. MMT Takers. For the subgroup of students required to take MMT for placement, we will measure engagement and track activities through MMT Analytics and tools developed related to the Canvas Brush-up.
Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
References Adelman, C. (1999). Answers in the toolbox: Academic intensity, attendance patterns, and bachelor’s degree attainment. Washington, D.C.: Office of Educational Research and Improvement, U.S. Department of Education. Retrieved from http://www.ed/gov/pubs/Toolbox/toolbox.html Adelman, C. (2004). Principal indicators of student academic histories in post-secondary education, 1972-2000. Washington, DC: Institute of Education Sciences, U.S. Department of Education. Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. The Journal of Higher Education, 77(5), 886-924. Bailey, T., Jeong, D. W., & Cho, S.W. (2008). Referral, enrollment, and completion in developmental education sequence in community colleges (CCRC Research Brief No. 45). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/student-progression-through-developmentalsequences-brief.pdf Bahr, P. R. (2010). Revisiting the efficacy of postsecondary remediation: The moderating effects of depth/breath of deficiency. The Review of Higher Education, 233(2), 177-205. Bettinger, E. P., & Long, B. T. (2005). Addressing the needs of under-prepared students in higher education: Does college remediation work? (NBER Working Paper No. 11325). Retrieved from National Bureau of Economic Research Website: www.nber.org/papers/w11325 Couturier, L., & Cullinane, J. (2015) A Call to Action to Improve Math Placement Policies and Processes. Retrieved from http://www.jff.org/publications/call-action-improve-math-placement-policies-and-processes Cullinane, J., & Treisman, P. U. (2010). Improving developmental mathematics education in community colleges: A prospectus and early progress report on the Statway Initiative. Paper presented at the National Center for Postsecondary Research Conference, New York, N.Y. Retrieved from National Center for Postsecondary Research Wedsite: http://www.postsecondaryresearch.org/conference/downloads.html Fay, M.P., Bickerstaff, S., & Hodara, M. (2013). Why students do not prepare for math placement exams: Student perspectives (CCRC Research Brief No. 57). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/publications/whystudents-do-not-prepare.html Hughes, K. L. & Scott-Clayton, J. (2011). Assessing developmental assessment in community colleges. Community College Review, 39(4), 327-351, DOI 10.1177/0091552111426898 Melguizo, T., Kosiewicz, H., Prather, G., & Bos, J. (2014). How are community college students assessed and placed in developmental math? Grounding our understanding in reality. Journal of Higher Education, 85(5), 691722. Rodriguez, O., Bowden, B., Belfield, C., & Scott-Clayton, J. (September, 2014). Remedial placement testing in community colleges: What resources are required and what does it cost? (CCRC Working Paper No. 73). New York, NY: Columbia University, Teachers College, Community College Research Center. Retrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/remedial-placement-testing-resources.pdf
Helen Burn ([email protected]) is an instructor of mathematics at Highline College (1994 to present), director of the Curriculum Research Group (2009 to present), and adjunct faculty in the College of Education at Seattle University (2009 to present). She has served as math department coordinator (1998-2000) and chair of the Pure and Applied Sciences division (2006-2009) at Highline. She holds an M.S. in mathematics from Western Washington University and a Ph.D. in higher education from the University of Michigan Center for the Study of Higher and Postsecondary Education. Shannon Waits ([email protected]) is the Director of Academic Assessment and Placement at Highline College, a new position created in 2013 to examine and guide the College's placement and assessment practices to ensure alignment with evidence-based practice, student-attainment goals, and college-wide values around educational equity. She holds an M.Ed. in Adult Education and a B.A. in Human Services. She has taught mathematics in ABE departments, King County Jail, and the YWCA's Young Parent Program. Suggested citation: Burn, H. E., & Waits, S. (2015, November). Improving the mathematics placement process. Paper presented at the meeting of the American Mathematical Association for Two-Year Colleges, New Orleans, LA.
