Week 1
Answer the following questions:
1. Jackson (2012) even-numbered Chapter Exercises (p. 244).
2. What is the purpose of conducting an experiment? How does an experimental design accomplish its purpose?
3. What are the advantages and disadvantages of an experimental design in an educational study?
4. What is more important in an experimental study, designing the study in order to make strong internal validity claims or strong external validity claims? Why?
5. In an experiment, what is a control? What is the purpose of a control group? Of single or multiple comparison groups?
6. What are confounds? Give an example of a design that has three confounds. Describe three ways to alter the design to address these confounds and explain the advantages and disadvantages of each.
7. What does “cause” mean and why is it an important concept in research? How are correlation and causation related?
8. You are a researcher interested in addressing the question: does smiling cause mood to rise (i.e., become more positive)? Sketch between-participants, within-participants, and matched-participants designs that address this question and discuss the advantages and disadvantages of each to yielding data that help you answer the question. Describe and discuss each design in 4-5 sentences.
Week 2
This is a two part assignment that will be submitted within one document.
Part I
Part I checks your understanding of key concepts from Jackson and Trochim & Donnelly.
Answer the following questions:
1. Jackson even-numbered Chapter exercises (pp. 220-221; 273-275)
2. What are degrees of freedom? How are the calculated?
3. What do inferential statistics allow you to infer?
4. What is the General Linear Model (GLM)? Why does it matter?
5. Compare and contrast parametric and nonparametric statistics. Why and in what types of cases would you use one over the other?
6. Why is it important to pay attention to the assumptions of the statistical test? What are your options if your dependent variable scores are not normally distributed?
Part II
Part II introduces you to a debate in the field of education between those who support Null Hypothesis Significance Testing (NHST) and those who argue that NHST is poorly suited to most of the questions educators are interested in. Jackson (2012) and Trochim and Donnelly (2006) pretty much follow this model. Northcentral follows it. But, as the authors of the readings for Part II argue, using statistical analyses based on this model may yield very misleading results. You may or may not propose a study that uses alternative models of data analysis and presentation of findings (e.g., confidence intervals and effect sizes) or supplements NHST with another model. In any case, by learning about alternatives to NHST, you will better understand it and the culture of the field of education.
Answer the following questions:
1. What does p = .05 mean? What are some misconceptions about the meaning of p =.05? Why are they wrong? Should all research adhere to the p = .05 standard for significance? Why or why not?
2. Compare and contrast the concepts of effect size and statistical significance.
3. What is the difference between a statistically significant result and a clinically or “real world” significant result? Give examples of both.
4. What is NHST? Describe the assumptions of the model.
5. Describe and explain three criticisms of NHST.
6. Describe and explain two alternatives to NHST. What do their proponents consider to be their advantages?
7. Which type of analysis would best answer the research question you stated in Activity 1? Justify your answer.
Week 3
Answer the Following Questions
1. Jackson, even-numbered Chapter Exercises, pp. 308-310.
2. What is an F-ratio? Define all the technical terms in your answer.
3. What is error variance and how is it calculated?
4. Why would anyone ever want more than two (2) levels of an independent variable?
5. If you were doing a study to see if a treatment causes a significant effect, what would it mean if within groups, variance was higher than between groups variance? If between groups variance was higher than within groups variance? Explain your answer
6. What is the purpose of a post-hoc test with analysis of variance?
7. What is probabilistic equivalence? Why is it important?
Week 4
Answer the Following Questions:
1. Jackson, even-numbered Chapter Exercises, pp. 335-337.
2. Explain the difference between multiple independent variables and multiple levels of independent variables. Which is better?
3. What is blocking and how does it reduce “noise”? What is a disadvantage of blocking?
4. What is a factor? How can the use of factors benefit a design?
5. Explain main effects and interaction effects.
6. How does a covariate reduce noise?
7. Describe and explain three trade-offs present in experiments.
Week 5
Quasi-Experimental Designs
Part I – Answer the following questions:
1. Jackson (2012), even-numbered chapter exercises, p 360.
2. Describe the advantages and disadvantages of quasi-experiments? What is the fundamental weakness of a quasi-experimental design? Why is it a weakness? Does its weakness always matter?
3. If you randomly assign participants to groups, can you assume the groups are equivalent at the beginning of the study? At the end? Why or why not? If you cannot assume equivalence at either end, what can you do? Please explain.
4. Explain and give examples of how the particular outcomes of a study can suggest if a particular threat is likely to have been present.
5. Describe each of the following types of designs, explain its logic, and why the design does or does not address the selection threats discussed in Chapter 7 of Trochim and Donnelly (2006):
a. Non-equivalent control group pretest only
b. Non-equivalent control group pretest/posttest
c. Cross-sectional
d. Regression-Discontinuity
6. Why are quasi-experimental designs used more often than experimental designs?
7. One conclusion you might reach (hint) after completing the readings for this assignment is that there are no bad designs, only bad design choices (and implementations). State a research question for which a single-group post-test only design can yield relatively unambiguous findings.
Part II – Answer the following questions:
1. What research question(s) does the study address?
2. What is Goldberg’s rationale for the study? Was the study designed to contribute to theory? Do the results of the study contribute to theory? For both questions: If so, how? If not, why not?
3. What constructs does the study address? How are they operationalized?
4. What are the independent and dependent variables in the study?
5. Name the type of design the researchers used.
6. What internal and external validity threats did the researchers address in their design? How did they address them? Are there threats they did not address? If so how does the failure to address the threats affect the researchers’ interpretations of their findings? Are Goldberg’s conclusions convincing? Why or why not?
Week 7
Samples, Power Analysis, and Design Sensitivity
Warm-up Activity
Download G*Power and play around with it. See how changes in assumptions and parameters affect sample size estimates.
Part 1
1. Compare and contrast internal and external validity. Describe and give examples of research questions for which external validity is a primary concern. Describe and give examples of research questions in which internal validity is a primary concern. Discuss strategies researchers use in order to make strong claims about the applicability of their findings to a target population.
2. Compare and contrast random selection and random assignment. Be sure to include a discussion of when you would want to do one or the other and the possible consequences of failing to do random selection or random assignment in particular situations.
3. Explain the relationship between sample size and the likelihood of a statistically significant difference between measured values of two groups. In other words, explain why, all else being equal, as sample size increases the likelihood of finding a statistically significant relationship increases.
4. Compare and contrast probability and non-probability sampling. What are the advantages and disadvantages of each?
Part 2
If you do a quantitative study for your dissertation, you must estimate the sample size you will need in order to have a reasonable chance of finding a relationship among the variables stated in your research hypotheses (should one exist), given your statistical analysis(es) and assumptions/calculations of factors 2-4 above. You must do this, even if you plan to use a convenience sample (see below). There are a number of sample size calculators available. Northcentral uses G*Power, which is required in this Activity. You will use G*Power’s “a priori power analysis” function to calculate a sample size. If it yields an unrealistically large size sample, you will rethink your design and assumptions and, perhaps, use G*Power’s “compromise power analysis” to estimate a workable sample size that makes sense. If you plan on using a convenience sample, you would use both analyses as part of your argument that your convenience sample is large enough.
Submit the Following
1. Calculate the sample size needed given these factors:
· one-tailed t-test with two independent groups of equal size
· small effect size (see Piasta, S.B., & Justice, L.M., 2010)
· alpha =.05
· beta = .2
· Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample half the size. Indicate the resulting alpha and beta. Present an argument that your study is worth doing with the smaller sample.
2. Calculate the sample size needed given these factors:
· ANOVA (fixed effects, omnibus, one-way)
· small effect size
· alpha =.05
· beta = .2
· 3 groups
· Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample approximately half the size. Give your rationale for your selected beta/alpha ratio. Indicate the resulting alpha and beta. Give an argument that your study is worth doing with the smaller sample.
3. In a few sentences, describe two designs that can address your research question. The designs must involve two different statistical analyses. For each design, specify and justify each of the four factors and calculate the estimated sample size you’ll need. Give reasons for any parameters you need to specify for G*Power.
Week 8
Write your mock Concept Paper using the Concept Paper template found in the Dissertation Center. Follow the template guidelines for each section.
1. Write an Introduction describing your topic.
2. Write the Statement of the Problem section.
3. Describe the Purpose of the Study. Include the results of your power analysis.
4. State your Research Question and your null and alternative hypotheses. Be sure that your question aligns with your purpose.
5. Write a Brief Review of the Literature.
6. Complete the Research Methods section (including the Operational Definition of Variables, Constructs, and Measurement sub sections). Follow the instructions in the CP template. Be sure to:
a. Identify the strengths and weaknesses of your envisioned design and methods.
b. Identify threats to validity and how your design will address them.
c. Justify why your chosen design and methods are more appropriate for your research question than alternatives you have considered.
d. Define the constructs you will measure and what you will do in order to determine how to operationalize them.
e. Describe the sample you propose to study and its characteristics; this should include, but is not limited, to: 1) age; 2) gender; 3) ethnicity; 4) additional cultural factors; and 5) education level. Justify your choice of sample.
f. Describe your method of sampling.
g. Describe the type of data you need to collect and how you will collect it.
h. Briefly describe any ethical issues you foresee with your study. Make a preliminary assessment of the level of risk associated with participation in your study that might need to be raised with the Institutional Review Board.
i. Describe and justify how you will analyze your data and the descriptive statistics will you present.
j. Explain how you conducted your power analysis.
k. Describe how you will handle your data, check for accuracy etc.
l. What problems do you foresee in implementing the design? How might you prevent them?
