PSYC 2030 Test 1 Notes Chapter 5: Introduction to Experimental ...
PSYC 2030 Test 1 Notes Chapter 5: Introduction to Experimental Research Process versus Content: The knowledge we have about the different content areas of psychology have the same or similar underlying process. Derived from research, the process can be applied to any content area. Critical thinking is another type derived from process that is important in interpretation.
Critical Thinking is a logical and rational process of avoiding one's preconceptions by gathering evidence contemplating and evaluating alternatives and coming to a conclusion. Evaluation of the data and hypothesis at hand without holding subjective preconceptions makes a critical thinker develop overtime.
Experimental Research: the experimenter varies one or more variables while holding others constant. There are usually two or more variables that are particularly concerned in every research, but variables can be anything that can take on a different value or that can change. The typical variables include the Independent (IV) and Dependent (DV). o
The IV is the cause that is manipulated to see if it has an effect on behavior (DV).
o
The DV is the behavior that is measured to see if the IV had an effect.
Extraneous Variable: variables that are not controlled and are of no interest to the experiment. They are not a problem as long as they are held constant. Compared to Confounding Variable, the variables are unwanted because they cause problems to the gathered results in the study.
Usually, the first thing sought to resolve the confounding variable is to re-examine the relationship between the IV and "extraneous" variable resulting in the values of DV.
Nevertheless, there are advantages to experimental research in general. The researcher can infer a cause and effect relationship and influence them to the expectations of the
researchers. Unfortunately, there is an also ethical question regarding studies done in an "unnatural environment" compared to a natural environment where the researcher has absolutely NO control over it. It is important to have a Control Group (the group that gets the "zero" level of the IV) in a study that compares the relationship between the effect of the IV on the DV and the effect of the baseline (Control Group) on the DV. It is used as the basis for comparison in studies.
Experimental Group: any group other than the control group.
If G0 is the control group, the G1, G2, G3, etc. is the experimental group. IV can be a manipulated variable: a subject variable (already existing characteristics of subjects).
Whenever you have a subject variable, you have a quasi-experiment (almost true).
The problem of having a subject variable is that the experimenter may not be able to locate extraneous variables (potential to be confounded) in a particular IV because the extraneous factors are generally too large or too difficult to attribute the cause. o
All that can be concluded is that the groups performed differently on the DV. It cannot be said that the IV was the cause.
Field Research: Broad term referring to experimental research outside the laboratory.
Situational Variable: An IV featuring different aspects of the surrounding environment.
Instructional Variable: An IV used to perform a particular task in different ways.
The lack of differences due to different groups scoring both high (ceiling effect) and both low (floor effect) in the research study because everyone gets extreme scores.
There are concerns regarding statistical conclusion validity (drawing an appropriate conclusion based on gathered data), construct validity (adequacy of operational definitions for IV and DV being properly defined), and external validity (generalized beyond the experiment) in research.
Kohlberg was criticized for his research based on grounds of external validity.
Ecological validity is the research with relevance for everyday cognitive activities of people trying to adapt to their environment. Internal validity is the degree to which an experiment is methodologically sound and confound-free.
Pretest involves testing the subject before the experiment while posttest involves testing the subject after the experiment. Subject selection effects may occur if groups are not equivalent because random assignment or matching was used to pick participants.
The regression to the mean is a phenomenon commonly described as a “bell curve.”
Testing is considered a threat to internal validity when the mere fact of taking a pretest has an effect on posttest scores. There could be a practice effect of repeated testing.
Instrumentation is a problem when there are changes in the measurement instrument from pretest to posttest.
Attrition refers to subjects failing to complete the study. It is a type of subject selection effect occurring through certain demographical traits of participants who dropped out of the study (uneven representation left behind). Subjects cannot be randomly assigned into groups. Chapter 6: Control Problems in Experimental Research Control Problems in Experimental Research are often seen as a plague that never ends in research findings. Normally in an experiment, it is ideal to have different groups being equal to each other in every way except for the IV. Between-Subject Design is a when subjects receives either level A or B (not both) of the IV. One advantage of using this model is that chances of initiating a “practice effect” are decreased. When a subject is within the between-subject design, there is always a risk of initiating "demand characteristics" (true purpose of the study becomes nearly aware to the subject and risking
experimenters into "blowing their cover"). Another disadvantage is that individual differences among groups can be a problem with acquiring constant results in the “equivalent groups.”
Random assignment is when each group has an equal chance of being placed in a group. The actual purpose of random assignment is to take individual differences and spread them out evenly among the groups so that you don't have a disproportionate number of individual difference variables in one group compared to another.
Block Randomization is a procedure ensuring that each condition of the study has a subject randomly assigned to it before any condition is repeated a second time.
In matching, participants are paired together on some trait then one member of each pair is randomly assigned to the group.
Within-Subject Design is when the subjects get both levels of the IV. The advantage is that it requires fewer subjects, and there is no need for individual differences. If true individual differences exist among conditions, you will be more likely to detect them using a within-subject design. The major problem of using this design is that once a participant has completed the first part of the study, the experience or altered circumstances could influence performance in later parts of the study (sequence/order effect).
Psychophysics is the study of sensory thresholds. Subjects are asked to judge whether or not they can detect some stimulus or whether two stimuli are equal or different. Each situation requires a number of trials and comparisons to be made within the same individual. A typical study just uses a few participants and measures them repeatedly.
Progressive Effect refers to the change in performance of the subject from trial to trial.
Carryover Effect is the effect the experiment has on the subject after the first condition. A between-subject design is used to reduce the carryover effect. The same kind of effect is
usually produced by following both sequences (regardless of the order effect). Certain exceptions exist however, that asymmetric transfer can result in a confounding variable. The typical way to control sequence effects in a within-subject design is to use more than one sequence, a strategy known as counterbalancing. One solution to the sequence problem when subjects are tested once per condition is to use complete counterbalancing (X!). This means that every possible sequence will be used at least once. Whenever a subgroup of the total number of sequences is used, the result is called partial counterbalancing (“incomplete”). This is used as an alternative to complete counterbalancing in case the X! is too large and requires randomization.
Balanced Latin Square design assures that every condition occurs equally often in every position, every condition precedes and follows every other condition exactly once.
Block randomization can also be used to counterbalance the sequence effects.
Counterbalancing strategies include comparing the subjects who participated in low imaging first, then high imaging after, and the subjects who participated in high imaging first, then low imaging after. Subjects being tested more than once in a given condition (first in sequence, next in reverse) are known as reverse counterbalancing.
Developmental Research looks at changes in behavior, cognition, and attitude that may occur as children develop. Cross-sectional study examines a between-subjects approach (examining different subjects on comparison) and longitudinal study examines a within-subjects approach (single group over a period of time). A cohort effect can reduce internal validity of crosssectional studies because differences between groups could result from the effects of life experience (extraneous variables).
A cohort sequential design is a strategy used to balance cohort and attrition problems. It combines aspects of cross-sectional and longitudinal designs & it allows the evaluation of
whether or not there is a cohort effect. Subjects are tested on a periodical basis to determine the effectiveness of practice effect. Diagonal examination of the cohort variables examines the same age in different years' environments (this is a cohort effect, the variables share the same variable, but they were studied in different environments). Aside falsification and parsimony, experimenter bias attempts to expose the experimenter’s objective influencing the outcome of the study in favor of the experimenter. Research protocols are in place for experimenters to report their findings in detail (including failed experiments).
Double blind study is a control procedure designed to reduce experimenter bias.
Neither the subject nor the experimenter knows which condition of the study is being tested. However, participant bias can occur in any study when behavior of participants is influenced by their beliefs about how they are supposed to behave in a study. Hawthorne effect is a form of participant bias that behavior is influenced by the mere knowledge that the participant is in an experiment and therefore of some importance to the experimenter.
Even a good subject role is a form of participant bias in which subjects try to guess the experimenter’s hypothesis and then behave in such a way as to confirm it.
Evaluation apprehension is a form of anxiety experienced by participants that leads them to behave so as to be evaluated positively by the experimenter. Manipulation check is used to debrief subjects and check if they were aware of the deceptive nature of the experiment they partook in. It also checks to see if systematic manipulation had an intended effect on the participants.
Chapter 7: Experimental Design I: Single-Factor Designs
Independent Groups Design is a between-subjects design that uses a manipulated IV and has at least two groups of participants. Subjects are randomly assigned to the groups.
Matched Groups Design is a between-subjects design that uses a manipulated IV and has at least two groups of participants. Subjects are matched on some variable assumed to affect the outcome before being randomly assigned to the groups.
Nonequivalent Groups Design is a between-subjects design with at least two groups of participants that uses a subject variable or that creates groups that are nonequivalent.
Repeated Measures Design is another name for a within-subject design. Participants are tested in each of the conditions in the experiment.
One assumption of the design is called the homogeneity of variance. It is one of the conditions that should be in effect in order to perform similar parametric inferential tests.
T-test for independent groups uses a between-subject design because they compare two samples of data in either matched groups design or a nonequivalent groups design. The advantage of using single-factor multilevel design is that it allows you to discover nonlinear effects if they exist. T-tests for related samples is an inferential statistical analysis used when comparing two samples of data in either a matched groups design or a repeated measures design.
A number of intermediate values exist in a continuum for a continuous variable. A discrete variable on the other hand represents distinction without intermediate points.
ANOVA = ANalysis Of VAriance is a single-factor, multilevel design (one-way analysis of variance). Using multiple t-tests risk Type I errors. If one way ANOVA is statistically significant, perform post-hoc tests. If it is not significant: STOP.
The probability of getting a Type I error = 1 - (1 - alpha)c, where c = number of comparisons. Alpha is usually by default = 0.5
Placebo control group is one type of control group designs that subjects are led to believe that they are receiving some treatment, although they are clearly not. o
The purpose of a placebo control group is to control for expectation effect that otherwise might be acting as a confounding variable.
Waiting list control group is another control group design that subjects are not yet receiving treatment, but will eventually receive one. It is used to ensure that those in experimental and control groups are similar to one another as an alternative to placebo (because placebo was considered to be unethical method to promote subject equality).
Yoked Control Group is the control group in which the treatment given a member of the control group is matched with the treatment given a member of the experimental group. Each subject in the control group is yoked (or matched) to a subject in the experimental group so that the group as a whole is similar. Ideally, each results should be similar.
Error bars indicate the amount of variability that occurred in each condition. It reflects standard deviations and confidence intervals around the mean.
Critical Thinking is a logical and rational process of avoiding one's preconceptions by gathering evidence contemplating and evaluating alternatives and coming to a conclusion. Evaluation of the data and hypothesis at hand without holding subjective preconceptions makes a critical thinker develop overtime.
Experimental Research: the experimenter varies one or more variables while holding others constant. There are usually two or more variables that are particularly concerned in every research, but variables can be anything that can take on a different value or that can change. The typical variables include the Independent (IV) and Dependent (DV). o
The IV is the cause that is manipulated to see if it has an effect on behavior (DV).
o
The DV is the behavior that is measured to see if the IV had an effect.
Extraneous Variable: variables that are not controlled and are of no interest to the experiment. They are not a problem as long as they are held constant. Compared to Confounding Variable, the variables are unwanted because they cause problems to the gathered results in the study.
Usually, the first thing sought to resolve the confounding variable is to re-examine the relationship between the IV and "extraneous" variable resulting in the values of DV.
Nevertheless, there are advantages to experimental research in general. The researcher can infer a cause and effect relationship and influence them to the expectations of the
researchers. Unfortunately, there is an also ethical question regarding studies done in an "unnatural environment" compared to a natural environment where the researcher has absolutely NO control over it. It is important to have a Control Group (the group that gets the "zero" level of the IV) in a study that compares the relationship between the effect of the IV on the DV and the effect of the baseline (Control Group) on the DV. It is used as the basis for comparison in studies.
Experimental Group: any group other than the control group.
If G0 is the control group, the G1, G2, G3, etc. is the experimental group. IV can be a manipulated variable: a subject variable (already existing characteristics of subjects).
Whenever you have a subject variable, you have a quasi-experiment (almost true).
The problem of having a subject variable is that the experimenter may not be able to locate extraneous variables (potential to be confounded) in a particular IV because the extraneous factors are generally too large or too difficult to attribute the cause. o
All that can be concluded is that the groups performed differently on the DV. It cannot be said that the IV was the cause.
Field Research: Broad term referring to experimental research outside the laboratory.
Situational Variable: An IV featuring different aspects of the surrounding environment.
Instructional Variable: An IV used to perform a particular task in different ways.
The lack of differences due to different groups scoring both high (ceiling effect) and both low (floor effect) in the research study because everyone gets extreme scores.
There are concerns regarding statistical conclusion validity (drawing an appropriate conclusion based on gathered data), construct validity (adequacy of operational definitions for IV and DV being properly defined), and external validity (generalized beyond the experiment) in research.
Kohlberg was criticized for his research based on grounds of external validity.
Ecological validity is the research with relevance for everyday cognitive activities of people trying to adapt to their environment. Internal validity is the degree to which an experiment is methodologically sound and confound-free.
Pretest involves testing the subject before the experiment while posttest involves testing the subject after the experiment. Subject selection effects may occur if groups are not equivalent because random assignment or matching was used to pick participants.
The regression to the mean is a phenomenon commonly described as a “bell curve.”
Testing is considered a threat to internal validity when the mere fact of taking a pretest has an effect on posttest scores. There could be a practice effect of repeated testing.
Instrumentation is a problem when there are changes in the measurement instrument from pretest to posttest.
Attrition refers to subjects failing to complete the study. It is a type of subject selection effect occurring through certain demographical traits of participants who dropped out of the study (uneven representation left behind). Subjects cannot be randomly assigned into groups. Chapter 6: Control Problems in Experimental Research Control Problems in Experimental Research are often seen as a plague that never ends in research findings. Normally in an experiment, it is ideal to have different groups being equal to each other in every way except for the IV. Between-Subject Design is a when subjects receives either level A or B (not both) of the IV. One advantage of using this model is that chances of initiating a “practice effect” are decreased. When a subject is within the between-subject design, there is always a risk of initiating "demand characteristics" (true purpose of the study becomes nearly aware to the subject and risking
experimenters into "blowing their cover"). Another disadvantage is that individual differences among groups can be a problem with acquiring constant results in the “equivalent groups.”
Random assignment is when each group has an equal chance of being placed in a group. The actual purpose of random assignment is to take individual differences and spread them out evenly among the groups so that you don't have a disproportionate number of individual difference variables in one group compared to another.
Block Randomization is a procedure ensuring that each condition of the study has a subject randomly assigned to it before any condition is repeated a second time.
In matching, participants are paired together on some trait then one member of each pair is randomly assigned to the group.
Within-Subject Design is when the subjects get both levels of the IV. The advantage is that it requires fewer subjects, and there is no need for individual differences. If true individual differences exist among conditions, you will be more likely to detect them using a within-subject design. The major problem of using this design is that once a participant has completed the first part of the study, the experience or altered circumstances could influence performance in later parts of the study (sequence/order effect).
Psychophysics is the study of sensory thresholds. Subjects are asked to judge whether or not they can detect some stimulus or whether two stimuli are equal or different. Each situation requires a number of trials and comparisons to be made within the same individual. A typical study just uses a few participants and measures them repeatedly.
Progressive Effect refers to the change in performance of the subject from trial to trial.
Carryover Effect is the effect the experiment has on the subject after the first condition. A between-subject design is used to reduce the carryover effect. The same kind of effect is
usually produced by following both sequences (regardless of the order effect). Certain exceptions exist however, that asymmetric transfer can result in a confounding variable. The typical way to control sequence effects in a within-subject design is to use more than one sequence, a strategy known as counterbalancing. One solution to the sequence problem when subjects are tested once per condition is to use complete counterbalancing (X!). This means that every possible sequence will be used at least once. Whenever a subgroup of the total number of sequences is used, the result is called partial counterbalancing (“incomplete”). This is used as an alternative to complete counterbalancing in case the X! is too large and requires randomization.
Balanced Latin Square design assures that every condition occurs equally often in every position, every condition precedes and follows every other condition exactly once.
Block randomization can also be used to counterbalance the sequence effects.
Counterbalancing strategies include comparing the subjects who participated in low imaging first, then high imaging after, and the subjects who participated in high imaging first, then low imaging after. Subjects being tested more than once in a given condition (first in sequence, next in reverse) are known as reverse counterbalancing.
Developmental Research looks at changes in behavior, cognition, and attitude that may occur as children develop. Cross-sectional study examines a between-subjects approach (examining different subjects on comparison) and longitudinal study examines a within-subjects approach (single group over a period of time). A cohort effect can reduce internal validity of crosssectional studies because differences between groups could result from the effects of life experience (extraneous variables).
A cohort sequential design is a strategy used to balance cohort and attrition problems. It combines aspects of cross-sectional and longitudinal designs & it allows the evaluation of
whether or not there is a cohort effect. Subjects are tested on a periodical basis to determine the effectiveness of practice effect. Diagonal examination of the cohort variables examines the same age in different years' environments (this is a cohort effect, the variables share the same variable, but they were studied in different environments). Aside falsification and parsimony, experimenter bias attempts to expose the experimenter’s objective influencing the outcome of the study in favor of the experimenter. Research protocols are in place for experimenters to report their findings in detail (including failed experiments).
Double blind study is a control procedure designed to reduce experimenter bias.
Neither the subject nor the experimenter knows which condition of the study is being tested. However, participant bias can occur in any study when behavior of participants is influenced by their beliefs about how they are supposed to behave in a study. Hawthorne effect is a form of participant bias that behavior is influenced by the mere knowledge that the participant is in an experiment and therefore of some importance to the experimenter.
Even a good subject role is a form of participant bias in which subjects try to guess the experimenter’s hypothesis and then behave in such a way as to confirm it.
Evaluation apprehension is a form of anxiety experienced by participants that leads them to behave so as to be evaluated positively by the experimenter. Manipulation check is used to debrief subjects and check if they were aware of the deceptive nature of the experiment they partook in. It also checks to see if systematic manipulation had an intended effect on the participants.
Chapter 7: Experimental Design I: Single-Factor Designs
Independent Groups Design is a between-subjects design that uses a manipulated IV and has at least two groups of participants. Subjects are randomly assigned to the groups.
Matched Groups Design is a between-subjects design that uses a manipulated IV and has at least two groups of participants. Subjects are matched on some variable assumed to affect the outcome before being randomly assigned to the groups.
Nonequivalent Groups Design is a between-subjects design with at least two groups of participants that uses a subject variable or that creates groups that are nonequivalent.
Repeated Measures Design is another name for a within-subject design. Participants are tested in each of the conditions in the experiment.
One assumption of the design is called the homogeneity of variance. It is one of the conditions that should be in effect in order to perform similar parametric inferential tests.
T-test for independent groups uses a between-subject design because they compare two samples of data in either matched groups design or a nonequivalent groups design. The advantage of using single-factor multilevel design is that it allows you to discover nonlinear effects if they exist. T-tests for related samples is an inferential statistical analysis used when comparing two samples of data in either a matched groups design or a repeated measures design.
A number of intermediate values exist in a continuum for a continuous variable. A discrete variable on the other hand represents distinction without intermediate points.
ANOVA = ANalysis Of VAriance is a single-factor, multilevel design (one-way analysis of variance). Using multiple t-tests risk Type I errors. If one way ANOVA is statistically significant, perform post-hoc tests. If it is not significant: STOP.
The probability of getting a Type I error = 1 - (1 - alpha)c, where c = number of comparisons. Alpha is usually by default = 0.5
Placebo control group is one type of control group designs that subjects are led to believe that they are receiving some treatment, although they are clearly not. o
The purpose of a placebo control group is to control for expectation effect that otherwise might be acting as a confounding variable.
Waiting list control group is another control group design that subjects are not yet receiving treatment, but will eventually receive one. It is used to ensure that those in experimental and control groups are similar to one another as an alternative to placebo (because placebo was considered to be unethical method to promote subject equality).
Yoked Control Group is the control group in which the treatment given a member of the control group is matched with the treatment given a member of the experimental group. Each subject in the control group is yoked (or matched) to a subject in the experimental group so that the group as a whole is similar. Ideally, each results should be similar.
Error bars indicate the amount of variability that occurred in each condition. It reflects standard deviations and confidence intervals around the mean.
