Sample Research Critique Essay

Chris K.
Research Critique 1

       Jamber, E. A., & Zhang, J .J. (1997).  Investigating leadership, gender, and coaching level using the Revised Leadership for Sport Scale. Journal of Sport Behavior, 20, 313-322.
 

       The purpose of the study was to determine possible differences in leadership behaviors,

using the Revised Leadership for Sport Scale (RLSS), between male and female coaches 

and among different coaching levels.  The researchers submitted two hypotheses.  The first 

hypothesis was that male and female coaches would respond differently to the RLSS in 

overall leadership behaviors.  The second hypothesis was that differences on the RLSS 

would occur among coaching levels: junior high, high school, and college.

     The sample was nonrandom, including 162 coaches that were chosen on a volunteer 

basis.  Within the sample, 118 (0.73) of the coaches were male, while 44 (0.27) were 

female.  With regard to coaching level, 25 (0.15) were junior high coaches, 99 (0.61) high 

school, and 38 (0.24) at the college level.   While this is a good sample size, the problem lies 

with the distribution of the sample.  The sample number for junior high coaches, in particular, 

is rather low.  A larger sample with regard to all categories would have aided in the data 

analysis, particularly when looking for possible interactions between gender and coaching 

level. 



     The instrument utilized was the Revised Leadership for Sport Scale (RLSS) developed 

by Zhang, Jensen, and Mann in 1996.  This scale is used to measure six leadership 

behaviors:  training and instruction, democratic, autocratic, social support, positive feedback, 

and situational consideration.  The scale uses 60 statements, which were preceded by “In 

coaching, I:” A Likert scale was then given for each statement: 1 = never; 2 = seldom; 3 = 

occasionally; 4 = often; and 5 = always.  This produced an ordinal level data set.  Scales 

were administered in a number of environmental settings: classrooms, gymnasiums, practice 

fields, and offices.  The internal consistency for each section was calculated: 0.84 for training 

and instruction; 0.66 for democratic; 0.70 for autocratic; 0.52 for social support; 0.78 for 

positive feedback; and 0.69 for situational consideration.  There was no information, 

however, regarding the validity of the RLSS. 

     A MANOVA was used to analyze the data for differences between male and female 

coaches with regard to leadership behaviors.  This is not consistent with the type of data 

collected.  The RLSS used a Likert scale (ordinal), yet a MANOVA would be most 

applicable for normally distributed, quantitative data.  The analysis showed there were no 

significant differences between male and female coaches in overall leadership behaviors. 

When the six leadership styles were examined separately, there was a significant difference 

in social support between males and females.  In general, females scored much higher than 

did the male coaches.



       A MANOVA was also used to examine the data for differences between the three 

levels of coaching (junior high, high school, and college) with regard to leadership behavior 

in general.  There were significant differences between the three levels.  When breaking 

down the six behaviors and examining them individually, an ANOVA was used to analyze 

the data.  Again, because the data for the RLSS is ordinal, an ANOVA is not the best 

analysis tool.   The three coaching levels scored differently on three of the six behaviors: 

democratic behaviors, training and instruction, and social support. High school coaches 

scored much higher than college level coaches in democratic behavior.  Junior high coaches 

were significantly lower in training and instruction than either high school or college coaches. 

Junior high coaches also demonstrated a lesser degree of social support than either the high 

school or college coaches.

     A MANOVA was again used to analyze the data for any interaction between gender and 

coaching level with regard to overall leadership behavior.  Once again, a better analysis 

method could have been chosen based on the nature of the data collected. The results 

indicated no significant interactions. 

      The ecological generaliziability for the study is fairly high.  The surveys were mailed out, 

and returned on a volunteer basis.  However,  due to the nonrandom nature of the sample, 

the results would not generalizable beyond the 162 participants in the study. There was no 

effect size is listed for the study. 

      In order to reduce threats to internal validity, the participants were asked to respond 

honestly and confidentiality was stressed so that the “coaches might feel more at ease in 

responding.”   No other efforts were indicated. 



      The researchers mention that the scales were given in a variety of settings.  This could 

present a threat to the internal validity in that participants might not have been entirely 

focused on completing the scale, but instead on coordinating practice, completing 

paperwork, etc.  There are a number of other factors that could effect the internal validity of 

the study, yet were not addressed by the researchers. Coaching experience would greatly 

effect the responses of the participants, yet this was not considered in the study.  The gender 

of the athletes may be a contributing factor to the coaches’ responses.  It is not unreasonable 

to suppose that coaches of female athletes, particularly at the junior high and high school 

levels, will demonstrate more social support than those of male athletes. The nature of the 

sport could also be critical.  Certain coaching styles are more applicable for individual sports 

(wrestling, track, and tennis) than for team sports (football, soccer, and basketball).  The 

socioeconomics and population of the school itself could play a factor.  Certain schools have 

better athletes and programs in a particular sport, while others may not be able to field a 

winning team.  In addition, at the high school level, coaches are occasionally asked/forced to 

work with a program they have no knowledge of or desire to coach due to staffing 

shortages.  This could dramatically influence a coach’s response to the scale questions. The 

history of the program as well as the individual coach’s personal coaching history could 

greatly influence responses.  If the program has had several losing seasons in a row, perhaps 

the attitude of the coach could be different than that of a coach who has recently won a state 

title. 



     An additional set of questions regarding the personal history of the coach in question 

could have helped reduce many of these threats.  With additional information, the 

researchers may have been able to use a modified matching system when analyzing the 

results.  By increasing the number of independent variables to include things such as 

coaching experience and gender of the athletes, the researchers could have reduced some of 

the potential threats to internal validity.  In addition, bringing coaches together to a common 

setting could have reduced location threat.  Coaches meet seasonally for clinics.  Perhaps 

obtaining permission to administer the survey during these meetings would have been 

possible. It would have also been possible to actually go to individual schools and meet with 

the coaches as a group to administer surveys.  This method would have given a good 

cross-section of gender and coaching experience for a variety of sports. 

     While the study has merit, the methods need to be re-evaluated.  The power of the study 

needs to be increased by obtaining a larger sample size.  The numerous potential threats to 

internal validity need to be addressed and minimized where possible.  It would also be 

helpful to be given data regarding the validity of the RLSS.  Without these, it is impossible to 

evaluate the potential meaningfulness of this study. 
 

 

Critique of a Research Article

The goal of this activity is to give you an opportunity to apply whatever you learned in this course in evaluating a research paper. Warning!!!!You might have done some article summaries or even critical evaluation of some resources. However, this activity is unique because you evaluate a research article from a methodology perspective.

For this assignment you briefly summarize and extensively evaluate the attached educational research article (If you cannot download the article please go to BeachBoard/Content/Articles to download the article). 

 

This assignment should be done individually. In the summary section, you should write a brief (up to 500 words) summary of the article in your own words. Don’t use copy and paste try to rephrase. This will be a good practice for your final project’s literature review. In the critique section, you evaluate the article using the following grading criteria.

Grading criteria for research critique

In your summary, you should identify main elements of the research including

1.        Research problem

2.        Research goal

3.        Hypothesis

4.       Research Questions

5.       Research Method (briefly explain)

6.       Sample (participants)

7.       Variables

8.       Tools (instruments, tests, surveys)

9.       Main findings (brief summary of the results)

10.   Conclusion

The critique part should be 2-3 pages (1000-2000 words) and include to the following sections. Your critique should be longer than your summary and you pay special attention to the design and procedure. Your grade on this assignment is based on your answer the following questions.

There is a long list of questions. You don’t have to address all questions. However, you should address highlighted questions. Some questions are relevant to this article some are not. I listed so many questions simply because I’d like you to learn what to look for in evaluating a research article.

The format of your paper should NOT be like a Q & A list. Instead, you should integrate your answers into an essay format similar to the given examples.

Introduction


Problem

1.     Is there a statement of the problem?

2.     Is the problem “researchable”? That is, can it be investigated through the collection and analysis of data?

3.     Is background information on the problem presented?

4.     Is the educational significance of the problem discussed?

5.     Does the problem statement indicate the variables of interest and the specific relationship between those variables which are investigated? When necessary, are variables directly or operationally defined?


Review of Related Literature

1.     Is the review comprehensive?

2.     Are all cited references relevant to the problem under investigation?

3.     Are most of the sources primary, i.e., are there only a few or no secondary sources?

4.     Have the references been critically analyzed and the results of various studies compared and contrasted, i.e., is the review more than a series of abstracts or annotations?

5.     Does the review conclude with a brief summary of the literature and its implications for the problem investigated?

6.     Do the implications discussed form an empirical or theoretical rationale for the hypotheses which follow?

 

Hypotheses

1.     Are specific questions to be answered listed or specific hypotheses to be tested stated?

2.     Does each hypothesis state an expected relationship or difference?

3.     If necessary, are variables directly or operationally defined?

4.     Is each hypothesis testable?

Method
         Subjects

1.     Are the size and major characteristics of the population studied described?

2.     If a sample was selected, is the method of selecting the sample clearly described?

3.     Is the method of sample selection described one that is likely to result in a representative, unbiased sample?

4.     Did the researcher avoid the use of volunteers?

5.     Are the size and major characteristics of the sample described?

6.     Does the sample size meet the suggested guideline for minimum sample size appropriate for the method of research represented?     


Instruments

1.     Is the rationale given for the selection of the instruments (or measurements) used?

2.     Is each instrument described in terms of purpose and content?

3.     Are the instruments appropriate for measuring the intended variables?

4.     Is evidence presented that indicates that each instrument is appropriate for the sample under study?

5.     Is instrument validity discussed and coefficients given if appropriate?

6.     Is reliability discussed in terms of type and size of reliability coefficients?

7.     If appropriate, are subtest reliabilities given?

8.     If an instrument was developed specifically for the study, are the procedures involved in its development and validation described?

9.     If an instrument was developed specifically for the study, are administration, scoring or tabulating, and interpretation procedures fully described?


Design and Procedure

1.     Is the design appropriate for answering the questions or testing the hypotheses of thestudy?

2.     Are the procedures described in sufficient detail to permit them to be replicated by another researcher?

3.     If a pilot study was conducted, are its execution and results described as well as its impact on the subsequent study?

4.     Are the control procedures described?

5.     Did the researcher discuss or account for any potentially confounding variables that he or she was unable to control for?

 


Results

1.     Are appropriate descriptive or inferential statistics presented?

2.     Was the probability level, α, at which the results of the tests of significance were evaluated,

specified in advance of the data analyses?

3.     If parametric tests were used, is there evidence that the researcher avoided violating the

required assumptions for parametric tests?

4.     Are the tests of significance described appropriate, given the hypotheses and design of the

study?

5.     Was every hypothesis tested?

6.     Are the tests of significance interpreted using the appropriate degrees of freedom?

7.     Are the results clearly presented?

8.     Are the tables and figures (if any) well organized and easy to understand?

9.     Are the data in each table and figure described in the text?


Discussion (Conclusions and Recommendation)

1.     Is each result discussed in terms of the original hypothesis to which it relates?

2.     Is each result discussed in terms of its agreement or disagreement with previous results

obtained by other researchers in other studies?

3.     Are generalizations consistent with the results?

4.     Are the possible effects of uncontrolled variables on the results discussed?

5.     Are theoretical and practical implications of the findings discussed?

6.     Are recommendations for future action made?

7.     Are the suggestions for future action based on practical significance or on statistical

significance only, i.e., has the author avoided confusing practical and statistical

significance?

8.     Are recommendations for future research made?

 

Make sure that you cover the following questions in your critique even if you have already covered them in your crtique.

1.   Is the research important? Why?

2.   In your own words what methods and procedures were used? Evaluate the methods and procedures.

3.   Evaluate the sampling method and the sample used in this study.

4.   Describe the reliability and validity of all the instruments used.

5.   What type of research is this?  Explain.

6.   How was the data analyzed?

7.   What is (are) the major finding(s)? are these findings important?

8.What are your suggestions to improve this research?

 

Help

Here is a hint on how to evaluate an article.

Use this resource for writing and APA style.

Examples (please note some examples are longer than what is expected for this article)

·         Good example

·         Poor example

More examples

·         Original article

·         Article critique

 

0 Thoughts to “Sample Research Critique Essay

Leave a comment

L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *