Practice Midterm Answer Key AWS
Practice Midterm Answer Key Part One: Research Situations Study A. A group of reading experts affiliated with the Office of Instructional Development at a large university wanted to investigate whether replacing the academic skills course that all students were required to take with a speed-reading course or a vocabulary-building course might improve their performance in their other classes. In the fall term, the reading experts randomly selected 420 entering freshmen (from the population of entering freshmen) to participate in the study. Somewhat unusually, all 420 agreed to participate and were randomly assigned to one of three groups. One group received the usual academic skills course. The second group received a special speedreading course, and the third group received a vocabulary-building course. All three courses met the same number of times throughout the term for the same number of hours. At the end of the term, the grade point averages for each group were calculated and compared to determine whether the new courses might be more beneficial than the academic skills course. The reading experts also investigated whether the native speakers of English and non-native speakers of English performed differently depending on which type of academic skills course they had. 1. Appropriately name the independent variable(s). Type of academic skills training 2.
Identify the type of scale used to measure or define the independent variable(s) and the levels of the independent variable(s). The independent variable is nominal and has three levels: speed reading, vocabulary building, “usual” academic skills 3. Appropriately name the dependent variable(s). Students’ learning, as indicated or measured by their first-term grade point average 4. What type of scale is used to measure the dependent variable(s)? Interval scale 5. Identify and appropriately name any explicitly stated moderator variables. L1 (or native/non-native English speaker) 6. Identify and appropriately name any explicitly stated control variables. I don’t see any that are explicitly mentioned as being control variables, though perhaps “term” might be a control variable (only students who begin in the fall term are included in the study). 7.
Identify the major category of research design that the study best fits and explain your choice. There is a proposed innovation or change to what is typically done, so there is an experimental treatment. The participants are identified through random selection
© Taylor & Francis 2014
from a population; and then they’re randomly assigned to one level of the independent variable. The study design is true experimental. 8.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Here are two possible threats. 1) Students may take different courses, graded in different ways: reduce the threat by including only participants from the population taking certain general education courses that freshmen are required to take in their first term. 2) Irregular attendance or lack of attendance: monitoring and reporting attendance doesn’t remove the threat, but reporting attendance allows readers to understand the extent to which attendance might have been a factor in the study. Study B. The chairperson of the Department of Languages and Culture at a large university recognized the need to conduct research on the impact that different manners of representing Arabic might have on learners’ development of accurate pronunciation in Arabic and on their attitudes toward the target language and target language culture. He applied for and received a grant to investigate this issue. In the first stage of his research, three intermediate Arabic curriculums were designed, each taught using a content-based approach, with the content being Arabic culture. For one class, all materials were authentic in the sense that they were written for native speakers of Arabic and presented in the Arabic alphabet. For the second class, the materials were also authentic, but they were presented in Anglicized script. For the third class, the materials were authentic but were presented using the International Phonetic Alphabet (IPA) rather than Arabic. The foreign language program typically offered only the first type of materials. Intermediate-level students of Arabic were allowed to choose which class to take, but each class had an enrollment limit of 15; there were 15 students in each class and a wait list for each. Before instruction started, all of the students were administered a computerdelivered Oral Proficiency Test in Arabic. The examinees’ responses were scored for pronunciation accuracy, and the scores were checked to verify that there were no preexisting differences among the groups. The students also completed a questionnaire that provided an “attitude toward the target culture” score for each student. The researchers verified that the three groups were similar at the outset in attitudes too. At the end of the term, the students took a parallel form of the computer-delivered Oral Proficiency Test in Arabic (equivalent form reliability = .92) and completed the questionnaire again as well. The change in each participant’s pronunciation accuracy from pretest to posttest was calculated, the mean “change” scores were determined, and the means for the three instructional groups were compared statistically to see if there were differences among them. Next the mean attitude change scores were determined and the three means were compared to determine if there were differences among the groups on this variable. 9.
Appropriately name the independent variable(s).
© Taylor & Francis 2014
Manner in which Arabic is represented in written course materials (or type of learning material) 10.
Identify the type of scale used to measure the independent variable(s) and the levels of the variable. Nominal. The independent variable has three levels: Arabic script, Anglicized script, phonetic alphabet. 11.
Appropriately name the dependent variable(s) and the type of scale used to measure the variable(s). There are two dependent variables: 1) Pronunciation accuracy 2) Attitude toward target culture Both appear to be measured (or defined) using an interval scale. (The “scores” for both dependent variables are treated as if they have mathematical properties—the means are calculated—so I hope they are interval-scale or interval-like!) 12.
Identify and appropriately name any explicitly stated moderator variables and the type of scale used to measure the variable(s). I don’t believe there is an explicit moderator variable. 13. Identify and appropriately name any explicitly stated control variables. Class level may be the control variable (only intermediate-level students are included in the study). 14.
Identify the major category of research design that this study best fits and explain your choice. There is a treatment, and the researcher used a pretest to check that the groups were comparable at the outset of the study. 15.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Here are two: 1) Previous study of Arabic. To reduce the threat, reconsider the level of students included in the study and use beginners! 2) The presence of heritage speakers in the classes. Consider introducing a control variable (including only non–heritage speakers in the study). 16.
Identify one possible threat to the external validity of the study (other than problematic internal validity) and describe what could have been done to reduce it. Here are two suggestions: 1) Size of class. Include class size as a moderator variable and have various or typical class sizes represented and studied. 2) Class level. The study is limited to only intermediate-level students; include other levels. © Taylor & Francis 2014
Study C. Interpreter trainers/researchers at a small private graduate school were curious about the possible relationship between the scores of Interpretation Studies students on the Modern Language Association Aptitude Test, a norm-referenced test, and the students’ performance on an interpreting skills test taken by the students after completing all of the coursework for an MA in Interpreting, which yields normally distributed data when given to its intended audience. The researchers decided to focus their efforts on the students in the Korean program. Only students who completed all of the MA-Interpreting courses in this program were included in the study. Students with three working languages (Korean, English, and another language) were excluded. The students took the aptitude test in the first week of their studies; after they completed all of the coursework for the MA-I, they took the interpreting skills test. The data were analyzed to investigate whether there was a statistically significant relationship between aptitude and interpreting skill. 17. Appropriately name the independent variable(s) and the levels. Language aptitude. There are as many levels as there are points on the test that is used to measure language aptitude. 18.
Appropriately name the dependent variable(s) and the type of scale used to measure the variable(s). Interpreting skill. It is an interval scale variable. 19. Identify and appropriately name any explicitly stated moderator variables. None 20. Identify and appropriately name any explicitly stated control variables. Language seems to be a control variable (only Korean-English students are included in the study). Number of working languages seems to be a control variable too; participants have only two working languages—those with three are excluded from the study. 21.
Identify the major category of research design that this study best fits and explain your choice. Ex post facto (there is no experimental treatment) 22.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Some ideas: 1) Some participants may already be doing professional interpreting. Solution: create a control variable called +/‒ working and include people in the study who do not yet have professional experience as interpreters. 2) Time to complete the MA degree. Solution: create a control variable called time to complete degree and include only participants who completed the degree within 2 academic years.
© Taylor & Francis 2014
3) Environment in which the tests are taken. Solution: train and observe the test administrators to ensure that the tests are given in a similar manner. 4) Lack of interrater reliability. Solution: train test scorers and check the degree of rater reliability. 23.
Identify one possible threat to the external validity of the study (other than problematic internal validity) and describe what could have been done to reduce it. Since all of the participants are in the Korean-English combination, the finding cannot be applied to other language combinations. Solution: include other language combinations in the study! Part Two: Calculations Study D. The scores of the 14 students enrolled in an intermediate language class are: 11 4 6 9 6 10 8 8 8 10 8 9 9 11 See R commands below. 24.
The mean is 8.36.
25.
The mode is 8.
26.
The median is 8.5.
27.
The range is 7 points.
28.
The standard deviation is 1.98.
29.
Draw a frequency distribution of the scores.
30.
How would you characterize the shape of the distribution? © Taylor & Francis 2014
Based on my review of the histogram, I think the distribution approaches a normal distribution, though it could be very slightly negatively skewed. The Shapiro–Wilk test indicates that the data are likely to be normally distributed (W = 0.9294, p-value = 0.2991), but note that the Shapiro–Wilk test is not very rigorous when the dataset is very small. Enter data
> studyd = c(11,6,6,8,8,8,9,4,9,10,8,10,9,11)
Calculate mean, median; get minimum and maximum for range To calculate only the mean, you can use this command Calculate range using minimum and maximum scores Calculate standard deviation
> summary (studyd)
Determine the mode (it’s 8, and there are 4 scores of 8 in the dataset) Make a histogram
> subset (table (studyd), table (studyd)==max (table (studyd)))
Min. 1st Qu. Median Mean 3rd Qu. Max. 4.000 8.000 8.500 8.357 9.750 11.000
> mean (studyd) [1] 8.357143
> 11-4 [1] 7
> sd(studyd) [1] 1.984833
8 4
> hist(studyd, breaks = 7, col = "sky blue")
© Taylor & Francis 2014
Check the normality of the distribution
> shapiro.test (studyd) Shapiro–Wilk normality test data: studyd W = 0.9294, p-value = 0.2991
Study E. A teacher was asked by the school’s review board to provide evidence of test performance for a student who had filed a grievance, claiming that he’d unfairly received a failing grade in the course. The class was large, and since the teacher’s tests were normreferenced and her students’ scores fell in a normal distribution, she decided to standardize the student’s scores for the review board. There were three main tests, one on a 35-point scale, one on a 40-point scale, and one on a 100-point scale. Using the following information, compute the z-scores for the student. Test one: mean = 27.5 sd = 4.8 Test two: mean = 31 sd = 2.7 Test three: mean = 83 sd = 7 Score 25 30 81
z-score ‒.52 ‒.37 ‒.29 ‒1.18 ‒.39
31. 32. 33. 34. 35.
test one test two test three total z = mean z =
36.
Students with a mean z-score of less than 0 fail this course. Based on the student’s mean z-score, should he have failed the course? (Circle your choice)
YES
© Taylor & Francis 2014
Identify the type of scale used to measure or define the independent variable(s) and the levels of the independent variable(s). The independent variable is nominal and has three levels: speed reading, vocabulary building, “usual” academic skills 3. Appropriately name the dependent variable(s). Students’ learning, as indicated or measured by their first-term grade point average 4. What type of scale is used to measure the dependent variable(s)? Interval scale 5. Identify and appropriately name any explicitly stated moderator variables. L1 (or native/non-native English speaker) 6. Identify and appropriately name any explicitly stated control variables. I don’t see any that are explicitly mentioned as being control variables, though perhaps “term” might be a control variable (only students who begin in the fall term are included in the study). 7.
Identify the major category of research design that the study best fits and explain your choice. There is a proposed innovation or change to what is typically done, so there is an experimental treatment. The participants are identified through random selection
© Taylor & Francis 2014
from a population; and then they’re randomly assigned to one level of the independent variable. The study design is true experimental. 8.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Here are two possible threats. 1) Students may take different courses, graded in different ways: reduce the threat by including only participants from the population taking certain general education courses that freshmen are required to take in their first term. 2) Irregular attendance or lack of attendance: monitoring and reporting attendance doesn’t remove the threat, but reporting attendance allows readers to understand the extent to which attendance might have been a factor in the study. Study B. The chairperson of the Department of Languages and Culture at a large university recognized the need to conduct research on the impact that different manners of representing Arabic might have on learners’ development of accurate pronunciation in Arabic and on their attitudes toward the target language and target language culture. He applied for and received a grant to investigate this issue. In the first stage of his research, three intermediate Arabic curriculums were designed, each taught using a content-based approach, with the content being Arabic culture. For one class, all materials were authentic in the sense that they were written for native speakers of Arabic and presented in the Arabic alphabet. For the second class, the materials were also authentic, but they were presented in Anglicized script. For the third class, the materials were authentic but were presented using the International Phonetic Alphabet (IPA) rather than Arabic. The foreign language program typically offered only the first type of materials. Intermediate-level students of Arabic were allowed to choose which class to take, but each class had an enrollment limit of 15; there were 15 students in each class and a wait list for each. Before instruction started, all of the students were administered a computerdelivered Oral Proficiency Test in Arabic. The examinees’ responses were scored for pronunciation accuracy, and the scores were checked to verify that there were no preexisting differences among the groups. The students also completed a questionnaire that provided an “attitude toward the target culture” score for each student. The researchers verified that the three groups were similar at the outset in attitudes too. At the end of the term, the students took a parallel form of the computer-delivered Oral Proficiency Test in Arabic (equivalent form reliability = .92) and completed the questionnaire again as well. The change in each participant’s pronunciation accuracy from pretest to posttest was calculated, the mean “change” scores were determined, and the means for the three instructional groups were compared statistically to see if there were differences among them. Next the mean attitude change scores were determined and the three means were compared to determine if there were differences among the groups on this variable. 9.
Appropriately name the independent variable(s).
© Taylor & Francis 2014
Manner in which Arabic is represented in written course materials (or type of learning material) 10.
Identify the type of scale used to measure the independent variable(s) and the levels of the variable. Nominal. The independent variable has three levels: Arabic script, Anglicized script, phonetic alphabet. 11.
Appropriately name the dependent variable(s) and the type of scale used to measure the variable(s). There are two dependent variables: 1) Pronunciation accuracy 2) Attitude toward target culture Both appear to be measured (or defined) using an interval scale. (The “scores” for both dependent variables are treated as if they have mathematical properties—the means are calculated—so I hope they are interval-scale or interval-like!) 12.
Identify and appropriately name any explicitly stated moderator variables and the type of scale used to measure the variable(s). I don’t believe there is an explicit moderator variable. 13. Identify and appropriately name any explicitly stated control variables. Class level may be the control variable (only intermediate-level students are included in the study). 14.
Identify the major category of research design that this study best fits and explain your choice. There is a treatment, and the researcher used a pretest to check that the groups were comparable at the outset of the study. 15.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Here are two: 1) Previous study of Arabic. To reduce the threat, reconsider the level of students included in the study and use beginners! 2) The presence of heritage speakers in the classes. Consider introducing a control variable (including only non–heritage speakers in the study). 16.
Identify one possible threat to the external validity of the study (other than problematic internal validity) and describe what could have been done to reduce it. Here are two suggestions: 1) Size of class. Include class size as a moderator variable and have various or typical class sizes represented and studied. 2) Class level. The study is limited to only intermediate-level students; include other levels. © Taylor & Francis 2014
Study C. Interpreter trainers/researchers at a small private graduate school were curious about the possible relationship between the scores of Interpretation Studies students on the Modern Language Association Aptitude Test, a norm-referenced test, and the students’ performance on an interpreting skills test taken by the students after completing all of the coursework for an MA in Interpreting, which yields normally distributed data when given to its intended audience. The researchers decided to focus their efforts on the students in the Korean program. Only students who completed all of the MA-Interpreting courses in this program were included in the study. Students with three working languages (Korean, English, and another language) were excluded. The students took the aptitude test in the first week of their studies; after they completed all of the coursework for the MA-I, they took the interpreting skills test. The data were analyzed to investigate whether there was a statistically significant relationship between aptitude and interpreting skill. 17. Appropriately name the independent variable(s) and the levels. Language aptitude. There are as many levels as there are points on the test that is used to measure language aptitude. 18.
Appropriately name the dependent variable(s) and the type of scale used to measure the variable(s). Interpreting skill. It is an interval scale variable. 19. Identify and appropriately name any explicitly stated moderator variables. None 20. Identify and appropriately name any explicitly stated control variables. Language seems to be a control variable (only Korean-English students are included in the study). Number of working languages seems to be a control variable too; participants have only two working languages—those with three are excluded from the study. 21.
Identify the major category of research design that this study best fits and explain your choice. Ex post facto (there is no experimental treatment) 22.
Identify one possible threat to the internal validity of the study and describe what could have been done to reduce it. Some ideas: 1) Some participants may already be doing professional interpreting. Solution: create a control variable called +/‒ working and include people in the study who do not yet have professional experience as interpreters. 2) Time to complete the MA degree. Solution: create a control variable called time to complete degree and include only participants who completed the degree within 2 academic years.
© Taylor & Francis 2014
3) Environment in which the tests are taken. Solution: train and observe the test administrators to ensure that the tests are given in a similar manner. 4) Lack of interrater reliability. Solution: train test scorers and check the degree of rater reliability. 23.
Identify one possible threat to the external validity of the study (other than problematic internal validity) and describe what could have been done to reduce it. Since all of the participants are in the Korean-English combination, the finding cannot be applied to other language combinations. Solution: include other language combinations in the study! Part Two: Calculations Study D. The scores of the 14 students enrolled in an intermediate language class are: 11 4 6 9 6 10 8 8 8 10 8 9 9 11 See R commands below. 24.
The mean is 8.36.
25.
The mode is 8.
26.
The median is 8.5.
27.
The range is 7 points.
28.
The standard deviation is 1.98.
29.
Draw a frequency distribution of the scores.
30.
How would you characterize the shape of the distribution? © Taylor & Francis 2014
Based on my review of the histogram, I think the distribution approaches a normal distribution, though it could be very slightly negatively skewed. The Shapiro–Wilk test indicates that the data are likely to be normally distributed (W = 0.9294, p-value = 0.2991), but note that the Shapiro–Wilk test is not very rigorous when the dataset is very small. Enter data
> studyd = c(11,6,6,8,8,8,9,4,9,10,8,10,9,11)
Calculate mean, median; get minimum and maximum for range To calculate only the mean, you can use this command Calculate range using minimum and maximum scores Calculate standard deviation
> summary (studyd)
Determine the mode (it’s 8, and there are 4 scores of 8 in the dataset) Make a histogram
> subset (table (studyd), table (studyd)==max (table (studyd)))
Min. 1st Qu. Median Mean 3rd Qu. Max. 4.000 8.000 8.500 8.357 9.750 11.000
> mean (studyd) [1] 8.357143
> 11-4 [1] 7
> sd(studyd) [1] 1.984833
8 4
> hist(studyd, breaks = 7, col = "sky blue")
© Taylor & Francis 2014
Check the normality of the distribution
> shapiro.test (studyd) Shapiro–Wilk normality test data: studyd W = 0.9294, p-value = 0.2991
Study E. A teacher was asked by the school’s review board to provide evidence of test performance for a student who had filed a grievance, claiming that he’d unfairly received a failing grade in the course. The class was large, and since the teacher’s tests were normreferenced and her students’ scores fell in a normal distribution, she decided to standardize the student’s scores for the review board. There were three main tests, one on a 35-point scale, one on a 40-point scale, and one on a 100-point scale. Using the following information, compute the z-scores for the student. Test one: mean = 27.5 sd = 4.8 Test two: mean = 31 sd = 2.7 Test three: mean = 83 sd = 7 Score 25 30 81
z-score ‒.52 ‒.37 ‒.29 ‒1.18 ‒.39
31. 32. 33. 34. 35.
test one test two test three total z = mean z =
36.
Students with a mean z-score of less than 0 fail this course. Based on the student’s mean z-score, should he have failed the course? (Circle your choice)
YES
© Taylor & Francis 2014
