The art of dissecting journal articles

GUEST POST: TEMPLE UNIVERSITY, USA

Dana Miller-Cotto (@danacotto) is a second year doctoral student in the Educational Psychology department at Temple University in Philadelphia, Pennsylvania. Her research interests aim to investigate the role of ethnic identity on achievement and academic motivation.

In her free time, she enjoys traveling, hiking, and attempting new pastry recipes. To learn more about her research interests, vist her blog Dana-Cotto.Wordpress

This semester has yielded numerous important milestones in my journey as a scholar. The biggest milestone for me has been developing a better handle on analyzing journal articles. One of the courses I’m taking requires us to get in to groups and to discuss an experiment or study from our assigned readings. Admittedly, I was a little apprehensive when I learned we would be partaking in this weekly activity. I have often been intimidated by the critical thinking that is needed when reading scholarly work. Thankfully, I’ve learned that it is a skill that is acquired after reading numerous articles over time. I’ve recently noticed a pattern of questions I now ask myself when tackling these articles. As a guideline for future use (for myself or for teaching undergrads) I have decided to post them here.

I break these down by section.

Literature Review

• Have the authors convinced you that their research needed to be conducted? What argument have they made to convince the reader why this study/experiment is important?
• If so, who would benefit from their research? How can it be applied to real world situations?
• Have they covered a wealth of previous literature?
• Is there a flaw in their logic? Have they made any unwarranted assumptions about previous research?

Methods

• Evaluate the sample size. Is it large enough to be generalized? (Think power analysis).
• How generalizable are the results based on the sample characteristics? For instance, is there sample too specific to be applied to other groups (i.e. sample includes upper middle class White students) ?
• Is there anything about the way the authors selected participants that may have made this sample bias?
• Are there any issues with the measures (e.g. could their measures being measuring something other than what they intended)?
• What are the variables? Independent variable? Dependent?
• Does the design align with their research goals? If not, how might they have improved on the design?

Results

• Were the correct analyses performed for the authors¿¿ research goals?
• Using your knowledge of statistics, do the results imply what the authors claim they imply? Were the results as significant as the authors claim? Are there any other exaggerated conclusions?

Discussion

• Overall, did the researchers draw appropriate conclusions?

General Questions

• What kind of journal does the article appear in? Is it a top tier journal?
• If the article is in a mediocre journal, what flaws in the experiment/study may have caused the article to appear in this journal?
• If there are clear flaws in the experiment/journal, and the article appears in a top tier journal, why might the article have been published despite these flaws? Funding opportunities? The experiment is from an area outside of the journal’s main area?
• Is there anything that is unclear?

I am so grateful for my experience developing my critical thinking in this class that I intend to do the same activity when I begin teaching.

Brief, relevant anecdote: a few days ago a friend of mine and I were discussing different techniques to get undergraduate students to think critically. We came to the conclusion that it is at first very difficult for them to think this way because they may not have been given many opportunities to do so in the past. High school, depending on where they attended, tends to require a lot of memorization and rote learning. Critical thinking is not something that is asked of them until later and when a professor first asks them to engaged in critical thinking, they’re often confused.

Are there any questions or tips you can add to this list? Feel free to add them in the comments.

 

This post was originally published in 2012 on the PhD Life Blog.

Photo Credit: Pedro Simoes/ Creative Commons