DMGT 830Homework 1Summer 20161) In plain language, for a practitioner audience, discuss the basic logic underlying the ideal socialscientific experiment. Very specifically, why would a researcher employ the “experimentalgroup – control group” design? In what ways do they believe this design to be “ideal?” Forexample, what kind of logic helps them draw casual conclusions from this type of experiment?In your answer, talk through that logic so a practitioner can grasp it.2)C. Jones and B. Mutzabaugh (USA Today, May 15, 2013) reported that Alaska Airlines andJetBlue had, at that time, the highest flier satisfaction ratings of any airline in North America,according to an Airline Satisfaction Study from J.D. Power & Associates. The survey included11,800 fliers who traveled on a major North American carrier between April 2012 and March2013. In their conclusions, the authors suggested a link between fliers’ ability to use mobiledevices to check-in online and increased satisfaction ratings. Specifically, those who can checkin online tend to be more satisfied.a. What kind of statistical relationship are these authors proposing between technology andairline passengers’ satisfaction? Which is the “independent” variable here, and which is the“dependent” variable? How do you know?b. Is the proposed relationship a “positive” or “negative” one? How do you know?c. Speculate: Is it conceivable that a causal relationship exists between online check-in and airpassengers’ satisfaction? How so? Explain and illustrate the possibility. Provide a clear line ofquantitative logic that can be followed by practitioners reading your report.d. If Jones and Mutzabaugh (2013) claimed unequivocally that mobile device check-in causesflier satisfaction, would their conclusion be justified? Would it flow from the quantitativeevidence provided by the study described above? Why or why not? Let your answerdemonstrate your command of the quantitative terms “correlation” and “causation.”3) See the data below. It indicates, for each department in a major insurance firm, the averageperformance rating for the employees in that department. One column shows the averageperformance rating assigned by managers. The other column shows the average performancerating employees gave themselves. The company produced a report based on these data.You’ve been asked in your capacity as a DM to read and comment on the company report,specifically addressing the items below. Note: You may use the statistical software of yourchoice to re-analyze these data if you feel it necessary or helpful. If you do, be sure to explainyour output and hightlight the portions of output which support your argument.DepartmentManager EmployeeClaims Intake14.316.8Claims processing15.017.8Customer Service(external)27.826.2Customer Service(internal)27.934.2R&D48.847.6Human Resources16.818.3Legal23.728.5Information Technology 32.833.1Custodial56.037.3a. The original report produced a scatterplot (copy/pasted below). The report author concludesthat employee-generated and manager-generated performance scores are positively related. Isthis conclusion correct? Why or why not?Average Manager Ratinge60504030Series120Linear (Series1)1000204060Average Employee Self-Ratingb.If the company’s conclusion in item a above were correct what would be the practical meaningof such a finding? The report author left these implications unstated. Help the companyunderstand the potential value and meaning of this conclusion – is it “good” or “bad” for theinsurance company? How so? Explain in ways that make sense to the company leadership.c. The original report claims the correlation coefficient (r) between these variables is r = .86 andthe p-value for this correlation is p > .10. The report concludes that this indicates a “strongrelationship” between manager ratings and employee self-ratings. Based solely on a comparisonof the correlation coefficient and the p-value reported here, is their claim justified? Why or whynot?d. Given what you’ve learned in your DM training, you decide to double-check the findings byrunning a "t-test for the significance of a correlation coefficient," easily calculated by hand. Theformula for this test is available online and is given below. (Note, it is not the same as an“independent groups t-test for comparing means” or even a “paired groups t-test.” Nor is it thesame as a “t-test for proportions.” There are many kinds of t-tests! You have learned tocorrectly choose and correctly interpret the various tests.) Your goal is to determine thefollowing: Is the correlation in the company report “big enough” to be considered “real” orstatistically significant? That is, would we be likely to find the same correlation if we hadmeasured the entire population of insurance company departments, rather than only those of asingle company? How do you know?Degrees of freedom (a statistical concept necessary for finding the critical value in a statistical table) isalways N -2 for this t-test. So in this case, df = 74) A manager in a highly diverse organization theorizes that employee preferences for the type ofmanager they’d like to work with might be different for different age groups. The manager recently didsome reading in which she learned that age is a proxy for “generational culture” which is known toaffect our expectations of work. The manager collected some survey data concerning the type ofmanager each age group prefers (each respondent can choose only one type of manager as his/her“favorite” to work under). Ultimately, the manager organized the data into the following table andasked you to examine the data:Managerial Style AManagerial Style BManagerial Style CManagerial Style DTotala.Age = 30 and under107737552307Age = 35 – 451191029779397Age = 50+133127109107476What will be the manager’s “independent variable” in this research project? How do youknow?b. The manager says she plans to analyze these data via correlation analysis, but asks your opinion.How do you advise her? Why? Let your explanation clearly demonstrate your command ofcourse materials on “choosing the correct test.”c. The outcome of the statistical test employed in item b above reveals the following output:X2 = 6.73 (6df, 12.59), p = .35Does the output justify the manager’s conclusion that age is “not related” to employees’preferences for a particular type of managerial style? Why or why not?5. On a business trip you read an empirical article you’re considering for your dissertation. The authorpresents some output from a Confirmatory Factor Analysis software program. The results are presentedas a correlation matrix (the correlation of every operationalization with every other operationalization).As a convention, only correlations below the diagonal are reported, since those above the diagonal willbe a mirror image of those below.Under the correlation matrix are two rows that represent the “factor loadings” of each measure. Thatis, the program reports the strength with which each operationalization is associated with theunderlying true score presumed to be driving it.The author argued persuasively, based on prior research, that he was measuring two distinct variables,one with three operationalizations and the other with 4 operationalizations. Specifically, items 1 -3were survey questions designed to measure the independent variable, managers’ “self-efficacy.” Items4-7 were behavioral observations designed to measure the dependent variable, managers’ “mentoringbehaviors.”1234567Factor 1Factor 211.00.94.88.40.37.76.26.77.322345671.00.91.26.27.79.30.79.341.00.22.19.81.17.84.271.00.96.27.89.21.791.00.18.91.32.861.00.30.79.341.00.33.84a. Carefully examine the correlations themselves. Based on this information alone, was theresearcher justified in calling his measures “valid?” Why or why not? Let your answer displayyour command of course materials on measurement.b.Carefully examine the factor loadings. Based on this information alone, was the researcherjustified in calling his measures “valid?” Why or why not? Let your answer demonstrate thelogic of quantitative measurement in general, and confirmatory factory analysis in particular.c. Based on your DM training in quantitative methodology and measurement, can you offer soundadvice for improving this researcher’s measures? What would that advice be and on what logicis it based?6. The Organizational Communication Conflict Instrument (Putnam & Wilson, 1982) is designed tomeasure employees’ communication strategies for handling conflict with supervisors. This measure isknown to have high reliability (usually above .80, according to Rubin, Palmgreen, and Sypher, 1994).Let’s run an informal test to dis/confirm its reliability. The original OCCI was 65 questions long! Themost recent version is only 30 items. Still, for this exercise we have taken liberties and shortened thismeasure to serve our purposes.Your job will be to locate some employed friends/relatives/acquaintances willing to answer the 6questions below. Be sure to let them know that this is NOT a formal experiment, but just a simple classassignment to satisfy your personal curiosity. The response options are:1 = never4 = sometimes5 = often2 = very seldom6 = very often3 = seldom7 = always1. I blend my ideas with others to create new alternatives for resolving a conflict2. I suggest solutions which combine a variety of viewpoints3. I integrate arguments into a new solution from issues raised in a dispute4. I offer creative solutions to disagreements5. I avoid my supervisor when I suspect s/he wants to discuss a point on which we disagree.6. I suggest working together to create solutions to disagreementsYour tasks:a.Enter the responses from each of your friends/relatives/acquaintances into the software ofyour choice. Then determine the reliability (Chronbach’s Alpha) of this 6-item measure. Are yousufficiently satisfied with the reliability that you would agree with Rubin et al (1994) that itsreliability is “high?” Why or why not?b. Now let’s say that your organization was considering the OCCI for its next “employeeengagement” survey. Based on your experience with item a, above, do you think yourorganization’s leadership is wise to use the OCCI? Why or why not?Be certain your answer makes sense to practitioners. Be certain it touches on the implicationsof your answer in item a, above: What makes “alpha” important? What does it indicate? Howdoes it do this? Why should practitioners care about “alpha?”7) Examine the 6 questions in our shortened version of the OCCI in item 6 above. Think critically aboutthe content of these questions. Logically, how well do they “hang together” the way a valid measureshould? Examine the data you collected. Explore it. Let it suggest to you a way to improve the OCCI.Might there be hope for using this measure in the company’s next employee engagement survey (thatis, if it were revised)? Offer an answer that speculates. But remember! Statistics is not about math! Itis about human logic, dis/confirmed with math. You must never allow statistics to “speak forthemselves.” Instead, you must take a stand and convince readers that your stance is reasonable, giventhe evidence.8) Discuss: What does your investigation of the OCCI in items 6 and 7 above reveal to you about the linkbetween “validity” and “reliability?” How does sample size affect this link? Be certain your answerdemonstrates your command of important course materials on sampling, reliable measures, and validmeasures.ReferencesPutnam, L. L., & Wilson, C. E. (1982). Communicative strategies in organizational conflicts: Reliabilityand validity of a measurement scale. Communication Yearbook 6, 629-652.Rubin, R. B., Palmgreen, P., & Sypher, E. (1994). Communication research measures: A sourcebook.New York: Guilford Press.

Published by

Essays

View all posts