Attitude Thermometers
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Attitude Thermometers 1
(Individual or group level activity)
Directions: 1. Draw a thermometer on a piece of paper. You are using the thermometer symbol to demonstrate HIGH and LOW levels. You can use any symbol that has local, cultural meaning. Examples: water in a cup, fire, red/yellow/green like a stop light, a tree, etc. Use a separate thermometer for each question.
2. Mark the sides of the symbol with numbers, signs or words to indicate what the levels of possible responses mean. For example: high/medium/low, very/not very, many/few, etc.
3. Have participants place their answers on the thermometer (or other symbol). This can be done individually, each person has an individual piece of paper, or collectively, everyone places their answer on the same large paper. This can be done anonymously, with everyone marking an X, or using names so that you can track individual responses. People can use markers or stickers to mark their responses. If using one large paper for all the answers, be cautious about using this activity for sensitive topics as the answers are publicly visible!
Materials Needed: Paper: either large flip chart paper, or individual copies of the thermometer. 1 paper for each question. Markers or stickers for participants to write their answers.
Time needed: 5‐10 minutes depending on the number of questions.
As an evaluation tool: You can use this method at one or more points in time. At the beginning of a program or during a program it can be used for formative assessment to get feedback on how a program is going. Using the EXACT same thermometer at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can demonstrate outcomes (increased knowledge, confidence, change in feelings, etc). In order to compare across time, you will need to keep the physical papers and/or enter the data into a tracking form.
If you link the data with individual names (which can be done with either individual level or group level activity), you can make statements about individual level % change (ex: 80% of participants increased their confidence level by the end of the program.) If you do not have a way to identify individual answers, you can state the % of how participants answered the questions before and after the program (ex: At the end of the program, 80% of participants felt confident about their ability to…, as opposed to only 30% who felt confident at the beginning of the program.)
1
Attitude thermometers were developed by Lyn Paleo, http://lynpaleoevaluation.com
1
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
How comfortable are you talking with your partner about safe sex?
VERY
MEDIUM
NOT AT ALL
2
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Values Ranking Directions: 1. Develop a values ranking sheet by writing all the things you are ranking on one piece of paper with a space next to each to write the ranking number. Participants will individually RANK them in order from MOST important to LEAST important (put a 1 next to the most important, a 2 next to next most important, etc.) For lower literacy levels, you can use pictures to show the items to be ranked. Instead of writing numbers to rank them, participants can place stickers or make dots next to what is most important, least important, etc. 2. Give each participant a copy of the values ranking paper. 3. Explain that they are going to make a list of which things are MOST important to them. Explain that on their papers are 10 things that are important. 4. Ask them to write a number 1 next to the thing that is MOST important to them. Ask them to write a 2 next to the thing that is the next most important. Ask them to number the items from 1 to 10, with 1 being the MOST important and 10 being the LEAST important. Explain that they can only use each number one time. 5. When all participants are finished, have a discussion as a large group, or in small groups, about what they wrote. Why did they rank things in the order they did? Talk about cultural messages, or any key points you want to make after the participants have all had a chance to share. 6. Collect all of the papers and SAVE them. Materials Needed: Copies of the values ranking worksheet for each participant Time needed: 10‐30 minutes depending on amount of discussion desired. As an evaluation tool: This can also be done as an individual or group activity. As a group activity, either use one large piece of paper and have the group discuss and decide how to rank the items, or write each of the items to be ranked (or draw a picture representing the item) on a separate piece of paper. Ask the participants to put the papers in order from MOST important to LEAST important. Have the group work together to put the papers in order. You can use this method at one or more points in time. Using the EXACT same values ranking sheet at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can demonstrate outcomes (change in values). If you link the data with individual names, you can make statements about individual level change in values (ex: 80% of participants increased their ranking of how important safe sex is so them by the end of the program.) If you do not have a way to identify individual answers, you can state the % of how participants answered the questions before and after the program (ex: At the end of the program, 80% of participants felt safe sex was in their top 3 values…, as opposed to only 30% who felt this at the beginning of the program.) 3
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Values Ranking: What is MOST important to YOU in terms of evaluation? Write the numbers 1‐10 in the spaces below to rank the following things in order of how important they are to you. 1 means it is the MOST important thing to you and 10 means it is the LEAST important thing to you. You can only use each number 1 time.
Having rigorous data you can publish Letting participants use their own words
Quick to administer
Easy to analyze the data
Easy to administer/Don’t need to train staff
Participants can do it themselves
Gives qualitative results
Gives quantitative results Fun 4
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Skills Demonstration Checklists 2
(Individual level activity)
Directions: 1. Have participants divide into pairs. 2. Give each pair 2 copies of the skills demonstration checklist. The checklists should be two different colors; each pair gets one of each color. 3. Chose which color will be the first observer. (This is the color that is strongest evaluation data!) Their partner will demonstrate the skills; the observer will watch and fill out the checklist. 4. Have the person demonstrating turn their checklist over so that they cannot see it while they demonstrate. 5. Make sure to explain that observers shouldn’t discuss or give hints until after their partner has finished. 6. After the first person has gone, partners should switch roles so that both people have a chance to demonstrate. 7. Talk about how the activity felt. Where there things that were hard to do? Easy to do? 8. Collect all checklists. You may want to include in the evaluation data only the ones filled out by the first observers (The second person to demonstrate has the advantage of seeing their partners gone first. Depending on what you are trying to measure, this may result in biased results. Including only the data from the first demonstration prevents this bias.)
Materials Needed: Any props needed to demonstrate the skill(s) Colored checklists (should be in two colors to demarcate first and second demonstrators)
Time needed: 10+ minutes depending on the complexity of the skill(s) being demonstrated.
As an evaluation tool: Skills demonstrations are an outcome measure. They can be used at one or more points in time. Most commonly they are used at the end of a program to demonstrate mastery of specific skills. Used at both the beginning and end of a program, they could also demonstrate an increase in skills. The most essential question in using skills demonstration checklists as an evaluation tool is deciding whether to include all of the checklists as data, or just the checklists from the first demonstrator. This depends on how your objectives are written and the complexity of the skills being demonstrated (and thus how subject to the bias of going second they are).
You can link the checklists with individual names, or do it anonymously. Either way you can generally make statements about the % of participants who can successfully demonstrate specific skills by the end of a program. If using checklists at multiple points in time, you can make statements about % change over time (ex: “50% of participants increased their skill” (if tracking by name) or “at the end of the program, 80% of participants successfully demonstrated skill as opposed to only 30% who could do it at the beginning of the program” (if done anonymously)).
2
Skills demonstration checklists were developed by Lyn Paleo, http://lynpaleoevaluation.com
5
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Fishes and Boulders3
(Group level activity) Directions: 1. Draw a river on the large paper, with a clear indication of the goal you are trying to reach at the end of the river. 2. Pass out cut outs of fishes and boulders to all participants. 3. Ask participants to write on the fishes and boulders things that are supports for reaching the goal (fishes) and barriers to reaching the goal (boulders). You can also use sizes and colors of fishes and boulders to indicate the type or significance of supports and barriers. (For example: Institutional barriers, individual strengths, etc) 4. Have participants put the fishes and boulders on the stream. Placement of fishes and boulders can also indicate temporal chronology of supports and barriers (For example: the historical timeline of supports and barriers, the temporal place in a process, etc). 5. Facilitate a discussion in which participants share the supports and barriers they identified. Depending on the goal of the conversation, this can also be an opportunity to strategize about how to overcome identified barriers and/or this activity can segue into a planning/strategizing activity.
Materials Needed: Large flip chart or butcher paper Cut out shapes of fishes and boulders (at least 1‐2 of each type for each person in the group) Tape
Time needed: 20‐60 minutes depending on the amount of discussion desired and the type of group.
As an evaluation tool: You can use this method at one or more points in time. At a single point in time, Fishes and Boulders is fundamentally a community mapping exercise. It can be used at community, organizational, programmatic, or individual levels (Examples: Goal=healthy community, organization with stable leadership, a program that reaches 100 clients a year, graduation from high school). Use at a single point in time is fundamentally an activity of formative assessment or needs assessment, it gives us information about the community or issue we are trying to address. Using the EXACT same activity at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can also be used to demonstrate outcomes (increased capacity, ability to identify supports, or progress towards the identified goal). To compare across time, you will need to keep the physical papers and/or enter the responses into a tracking form.
Fishes and Boulders is a qualitative method. The type of evaluation data generated includes lists of barriers and supports, strategies to reach goals and potential progress toward goals (if the activity is done at multiple points in time). Although statements may be made about increases or decreases in supports/barriers over time (if the activity is done at multiple points in time), the data should not be translated into quantitative findings. (One should not say, “Participants identified 50% more supports by the end of the program.”)
3
Developed by Coupal, F. (1997). Participatory Monitoring and Evaluation Workbook. Ottawa, Mosaic.net International.
6
Attitude Thermometers 1
(Individual or group level activity)
Directions: 1. Draw a thermometer on a piece of paper. You are using the thermometer symbol to demonstrate HIGH and LOW levels. You can use any symbol that has local, cultural meaning. Examples: water in a cup, fire, red/yellow/green like a stop light, a tree, etc. Use a separate thermometer for each question.
2. Mark the sides of the symbol with numbers, signs or words to indicate what the levels of possible responses mean. For example: high/medium/low, very/not very, many/few, etc.
3. Have participants place their answers on the thermometer (or other symbol). This can be done individually, each person has an individual piece of paper, or collectively, everyone places their answer on the same large paper. This can be done anonymously, with everyone marking an X, or using names so that you can track individual responses. People can use markers or stickers to mark their responses. If using one large paper for all the answers, be cautious about using this activity for sensitive topics as the answers are publicly visible!
Materials Needed: Paper: either large flip chart paper, or individual copies of the thermometer. 1 paper for each question. Markers or stickers for participants to write their answers.
Time needed: 5‐10 minutes depending on the number of questions.
As an evaluation tool: You can use this method at one or more points in time. At the beginning of a program or during a program it can be used for formative assessment to get feedback on how a program is going. Using the EXACT same thermometer at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can demonstrate outcomes (increased knowledge, confidence, change in feelings, etc). In order to compare across time, you will need to keep the physical papers and/or enter the data into a tracking form.
If you link the data with individual names (which can be done with either individual level or group level activity), you can make statements about individual level % change (ex: 80% of participants increased their confidence level by the end of the program.) If you do not have a way to identify individual answers, you can state the % of how participants answered the questions before and after the program (ex: At the end of the program, 80% of participants felt confident about their ability to…, as opposed to only 30% who felt confident at the beginning of the program.)
1
Attitude thermometers were developed by Lyn Paleo, http://lynpaleoevaluation.com
1
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
How comfortable are you talking with your partner about safe sex?
VERY
MEDIUM
NOT AT ALL
2
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Values Ranking Directions: 1. Develop a values ranking sheet by writing all the things you are ranking on one piece of paper with a space next to each to write the ranking number. Participants will individually RANK them in order from MOST important to LEAST important (put a 1 next to the most important, a 2 next to next most important, etc.) For lower literacy levels, you can use pictures to show the items to be ranked. Instead of writing numbers to rank them, participants can place stickers or make dots next to what is most important, least important, etc. 2. Give each participant a copy of the values ranking paper. 3. Explain that they are going to make a list of which things are MOST important to them. Explain that on their papers are 10 things that are important. 4. Ask them to write a number 1 next to the thing that is MOST important to them. Ask them to write a 2 next to the thing that is the next most important. Ask them to number the items from 1 to 10, with 1 being the MOST important and 10 being the LEAST important. Explain that they can only use each number one time. 5. When all participants are finished, have a discussion as a large group, or in small groups, about what they wrote. Why did they rank things in the order they did? Talk about cultural messages, or any key points you want to make after the participants have all had a chance to share. 6. Collect all of the papers and SAVE them. Materials Needed: Copies of the values ranking worksheet for each participant Time needed: 10‐30 minutes depending on amount of discussion desired. As an evaluation tool: This can also be done as an individual or group activity. As a group activity, either use one large piece of paper and have the group discuss and decide how to rank the items, or write each of the items to be ranked (or draw a picture representing the item) on a separate piece of paper. Ask the participants to put the papers in order from MOST important to LEAST important. Have the group work together to put the papers in order. You can use this method at one or more points in time. Using the EXACT same values ranking sheet at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can demonstrate outcomes (change in values). If you link the data with individual names, you can make statements about individual level change in values (ex: 80% of participants increased their ranking of how important safe sex is so them by the end of the program.) If you do not have a way to identify individual answers, you can state the % of how participants answered the questions before and after the program (ex: At the end of the program, 80% of participants felt safe sex was in their top 3 values…, as opposed to only 30% who felt this at the beginning of the program.) 3
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Values Ranking: What is MOST important to YOU in terms of evaluation? Write the numbers 1‐10 in the spaces below to rank the following things in order of how important they are to you. 1 means it is the MOST important thing to you and 10 means it is the LEAST important thing to you. You can only use each number 1 time.
Having rigorous data you can publish Letting participants use their own words
Quick to administer
Easy to analyze the data
Easy to administer/Don’t need to train staff
Participants can do it themselves
Gives qualitative results
Gives quantitative results Fun 4
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Skills Demonstration Checklists 2
(Individual level activity)
Directions: 1. Have participants divide into pairs. 2. Give each pair 2 copies of the skills demonstration checklist. The checklists should be two different colors; each pair gets one of each color. 3. Chose which color will be the first observer. (This is the color that is strongest evaluation data!) Their partner will demonstrate the skills; the observer will watch and fill out the checklist. 4. Have the person demonstrating turn their checklist over so that they cannot see it while they demonstrate. 5. Make sure to explain that observers shouldn’t discuss or give hints until after their partner has finished. 6. After the first person has gone, partners should switch roles so that both people have a chance to demonstrate. 7. Talk about how the activity felt. Where there things that were hard to do? Easy to do? 8. Collect all checklists. You may want to include in the evaluation data only the ones filled out by the first observers (The second person to demonstrate has the advantage of seeing their partners gone first. Depending on what you are trying to measure, this may result in biased results. Including only the data from the first demonstration prevents this bias.)
Materials Needed: Any props needed to demonstrate the skill(s) Colored checklists (should be in two colors to demarcate first and second demonstrators)
Time needed: 10+ minutes depending on the complexity of the skill(s) being demonstrated.
As an evaluation tool: Skills demonstrations are an outcome measure. They can be used at one or more points in time. Most commonly they are used at the end of a program to demonstrate mastery of specific skills. Used at both the beginning and end of a program, they could also demonstrate an increase in skills. The most essential question in using skills demonstration checklists as an evaluation tool is deciding whether to include all of the checklists as data, or just the checklists from the first demonstrator. This depends on how your objectives are written and the complexity of the skills being demonstrated (and thus how subject to the bias of going second they are).
You can link the checklists with individual names, or do it anonymously. Either way you can generally make statements about the % of participants who can successfully demonstrate specific skills by the end of a program. If using checklists at multiple points in time, you can make statements about % change over time (ex: “50% of participants increased their skill” (if tracking by name) or “at the end of the program, 80% of participants successfully demonstrated skill as opposed to only 30% who could do it at the beginning of the program” (if done anonymously)).
2
Skills demonstration checklists were developed by Lyn Paleo, http://lynpaleoevaluation.com
5
Moving Beyond Surveys: Creative and Engaging Evaluation Activities for Youth and Others who Hate Tests, USCA 9/2010 Re‐use with permission only. Sonya Dublin, [email protected]
Fishes and Boulders3
(Group level activity) Directions: 1. Draw a river on the large paper, with a clear indication of the goal you are trying to reach at the end of the river. 2. Pass out cut outs of fishes and boulders to all participants. 3. Ask participants to write on the fishes and boulders things that are supports for reaching the goal (fishes) and barriers to reaching the goal (boulders). You can also use sizes and colors of fishes and boulders to indicate the type or significance of supports and barriers. (For example: Institutional barriers, individual strengths, etc) 4. Have participants put the fishes and boulders on the stream. Placement of fishes and boulders can also indicate temporal chronology of supports and barriers (For example: the historical timeline of supports and barriers, the temporal place in a process, etc). 5. Facilitate a discussion in which participants share the supports and barriers they identified. Depending on the goal of the conversation, this can also be an opportunity to strategize about how to overcome identified barriers and/or this activity can segue into a planning/strategizing activity.
Materials Needed: Large flip chart or butcher paper Cut out shapes of fishes and boulders (at least 1‐2 of each type for each person in the group) Tape
Time needed: 20‐60 minutes depending on the amount of discussion desired and the type of group.
As an evaluation tool: You can use this method at one or more points in time. At a single point in time, Fishes and Boulders is fundamentally a community mapping exercise. It can be used at community, organizational, programmatic, or individual levels (Examples: Goal=healthy community, organization with stable leadership, a program that reaches 100 clients a year, graduation from high school). Use at a single point in time is fundamentally an activity of formative assessment or needs assessment, it gives us information about the community or issue we are trying to address. Using the EXACT same activity at multiple points in time (generally once in the very beginning of a program (baseline), and once at the “end” point of a program), this method can also be used to demonstrate outcomes (increased capacity, ability to identify supports, or progress towards the identified goal). To compare across time, you will need to keep the physical papers and/or enter the responses into a tracking form.
Fishes and Boulders is a qualitative method. The type of evaluation data generated includes lists of barriers and supports, strategies to reach goals and potential progress toward goals (if the activity is done at multiple points in time). Although statements may be made about increases or decreases in supports/barriers over time (if the activity is done at multiple points in time), the data should not be translated into quantitative findings. (One should not say, “Participants identified 50% more supports by the end of the program.”)
3
Developed by Coupal, F. (1997). Participatory Monitoring and Evaluation Workbook. Ottawa, Mosaic.net International.
6
