Critically reviewing articles (as well as books and speaker presentations) has been one of the most significant methods of assessment throughout my MA course. A practical research skill that is of benefit to anyone in academic circles, I thought I would briefly summarise all the advice I have been offered, heard at Study Skills workshops, or read about, in case anyone gets asked specifically about this in a session.
What is a critical review?
In short, it is reading between the lines of what is presented on the page. Every article (or book etc) will have been written with a purpose, an agenda, while the author may have bias, the facts may have been misconstrued, or the methods may not be as sound as the researchers might have liked them to be. At University, it is important to stop blindly following what we are taught by our Tutors and the seminal books we read, and start questioning. It may be that the article is absolutely fantastic, or the theory is as awe-inspiring as Einstein’s Gravitational Waves! On the other hand, it might not be quite so significant, original, or rigorous (the big three for decent research).
What should we look for?
After reading the article as a whole, to get the general gist of it, the kinds of questions we need to be asking when reviewing it critically are as follows:
• What is the topic?
• Who is the intended audience, which can alter the way the article is written, and therefore reviewed?
• How much background information/context/literature review is provided? Have the most recent publications been mentioned? Is there a good evaluation by the author(s) of the literature they themselves have reviewed?
• Does the rest of the article follow smoothly and logically from this background?
• Is it written with clarity, an organised and clear structure, good referencing, and a definite line of argument?
• What is the research question or hypothesis, and why this one? If there isn’t one, why not? Does the article still have a clear purpose?
• Has the author(s) made it clear what limits have been placed on the study? Eg ‘I will look at only monolingual Welsh-speaking children, rather than bilinguals.’
• What are the research methods used, including any relevant statistical methods or theoretical frameworks?
• Who were the participants, the materials, what was the equipment, how were variables measured etc? Why were these specific methods chosen?
• How were data analysed?
• Would someone else reading this study be able to replicate the exact experiment or theoretical study to try to come up with the same results as the author(s)?
• What results have been found, in terms of data or new theories for instance?
• How do the results relate to the background information?
• Was the discussion (interpretation) of the results logical and reasonable, without overstating any conclusions that can be drawn for instance?
• Are the statistics significant? If they are, are they also meaningful? Statistics can determine if there is any effect between variables, but it cannot provide an argument as to the real-world value of that effect. Eg, if there is a difference between 2 people, or a few seconds etc in any given experiment, does that really mean anything?
• Is there any discussion of the study’s own limitations, or suggestions for improvement?
• Was the research question or hypothesis answered sufficiently?
• Was anything left unfinished?
• What are the implications, or ongoing questions, for further research?
• What is the researcher’s background? Is this their field? Might their research have been biased by who funded the study?
• Maybe, depending on what our review is for, we could compare this article with others on the same topic too?
How and why do we use this critiquing skill?
Obviously, if we have as assignment where we need to review an article explicitly, we can just follow these kinds of guideline questions, using our own judgement to notice anything else that crops up. But more broadly speaking, it has been stated by many of my Tutors that it’s good practice to get into the habit of really interrogating the literature we read for any assignment, and not to take the results, interpretations, or clout of the researcher at face value. Anyone can make a mistake, falsify data, cherry-pick their evidence, or just miss some valuable feature of their (or others’) research, not just we ‘lowly’ students! This is also an important skill to adopt for any higher level research that we might undertake, or even out in the big wide world: to discern whether advertising is just hype or not, or whether our workplace budget has realistically accounted for all factors, for instance.