# Welcome to SMIRK Unit
4 - Evaluating research
articles

- General pointers
- The literature review
- Research design
- Ethical issues
- Data analysis
- Findings
- Research glossary
- Research types
- Qualitative research
- Quantitative research
- Back to main index

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# General pointers

Once you have done your literature search and found some research articles, you need to review them. This checklist will mainly apply to articles, you may find that other items such as focus group reports will not fit into the categories given. In this case, you can move on to the next question in the list. If you are not sure how to deal with a certain format, please talk to your tutor. This list has been updated with material provided by the EBE Learning Development Centre.
Here is a general list of key questions:

#### Who were the researchers?

- State who they are
- Comment on them if they seem unusual candidates to do the research
- Are there any potential conflicts of interest?
- Who supported the research? Some research on the nutritional value of a type of food may be supported by an industry organisation rather than an academic institution. This may affect the findings of the study.

#### What is the research about?

- Does the title reflect the content?
- Does the abstract provide a full summary of the article content?
- State in your own words the question, aim, or objectives and the hypothesis (quantitative only) of the study
- Do the authors make all of the above clear?

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# The literature review

- Give a summary of the content of the literature covered by the author
- Have the authors given a comprehensive review of previous research looking at the topic from all perspectives?
- Have they shown why this literature cannot answer their problems?
- Is the literature up to date? Older literature related to policy is acceptable.
- Have a range of sources been used by the authors?

#### When was the research published?

- The age of the research may affect the validity of the findings. This may be especially true of areas such as health and life sciences or engineering and the built environment.

#### Where was the research carried out?

- The location may affect the outcomes of the research. Health research carried our in the USA may refer to different funding or community care models. Research outcomes on sustainable building issues may vary according to the country where the housing is to be built.

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Research design

- Have the authors explained why they chose a particular design (for example, survey, ethnography, phenomenology, experimental)? Not sure about the definitions of these words - our research glossary will explain.
- How have they justified their choice?
- Do you feel that their choice was appropriate?
- Give a description of the research approach and design backed up by a reference. We have a more in-depth explanation of research design in Unit 3: Finding and gathering information.

#### Population and sample

- Give a summary of the sample information provided by the author. If the research is to be applicable and relevant to other populations the study sample (the group selected to be observed) must be representative of the group from which the sample is drawn (study population). This should be typical of the wider population to whom the research might apply (target population).
- For both qualitative and quantitative approaches, does the author:
  - state the study population?
  - identify how the sample group was recruited?
  - give the sample size?
  - is the sample size appropriate for the research approach?
  - identify the response rate. In most studies, some of the study population don't respond; some refuse and other don’t know the answer. A poor response rate can bias the results of a study if not acknowledged. The study should compare responders and non-responders to acknowledge any impact of this potential bias.
  - is the control group (if applicable) adequate? Was the definition of controls adequate? Are the control group, in all other respects, the same as the subjects?
- specific critiquing issues for qualitative and quantitative approaches are available.

###### For a handy checklist, look in The research process in nursing, edited by Kate Gerrish. You will find it at call number 610.73072 RES on the second floor of the library.

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Ethical issues

- Summarise the ethical issues addressed by the researcher
- Have the authors addressed all the ethical issues such as:
  - informed consent, anonymity, confidentiality, beneficence, and maleficence
  - data protection
  - ethics committee approval?
  - How does this affect the study?
- Are there any ethical issues not addressed?

#### Data collection

- Brief description of the methods used to collect data
- Did the authors develop data collection tools specifically for this study, if so, how were they created and were they tested in a pilot study?
- Qualitative data: usually interviews, observation, diaries, historical data or videos.
- Quantitative data: usually questionnaires, scales, treatments, or physiological measures.

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Data analysis

- How did the authors analyse their data?
- Is it appropriate to the research approach
- Validity? For example, if a study investigates braking distance in cars aged over ten years and above, how might the age of the cars affect the results?
- Reliability. Can any of the test results obtained in the article be reproduced?
- Blindness? This refers to the amount of potential bias in the study.
- Completeness:
  - Compliance. This can be a problem in studies in health. If there is compliance in a study, this may have the effect of reducing any apparent improvement in a new system.
  - Missing data? Check that data has not been missed out in the final results section. If no explanation is given for this, then there is the potential that the result is biased and the validity of the findings compromised.

#### Qualitative approach - Usually content analysis or thematic analysis.

- Do the authors show in detail the process that occurred to get to the final categories (auditibility)?
- Do the authors show the categories of themes by using participant dialogue (quotes) or field notes to represent the themes (credibility)
- Do more than one researcher analyse the data (peer review)?
- How does all this affect the findings?

#### Quantitative approach - Usually either descriptive or inferential statistics

- What statistical tests did the researcher use?
- Did the researcher explain why they used descriptive and/or inferential statistics?
- Did they explain which tests they used and the relevance to the data?
- Are the tables and graphs easy to understand and well explained?
- This all affects validity of the findings
- What is the overall effect of this on the results?

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Findings

- What are the findings?
- State briefly in your own words

#### Discussion issues

- Summarises key discussion points.
- Have the authors discussed their findings in relation to other research and their own original aims?
- Have the authors mentioned limitations of their study and what are they?
- Have you identified any additional limitations?
- Have the authors given recommendations for general practice, this is acceptable if you think the findings are transferable or generalisable?
- Summarise any recommendations.
- Have the authors mentioned the need for future research and what form should it take? Summarise.

#### Overall view of research

- Briefly state what you thought of this as a piece of research and why.
- How could this research be used in practice.

This is the end of our checklist of questions to ask. It should help give you a framework to structure your assignment.

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Research glossary

These definitions are provided by the Learning Development Centres at Glasgow Caledonian University. We also have a list of types of research.

- Abstract A brief written summary of the purpose, aims, results and conclusions of your research; separate from the dissertation.
- Bibliography An ordered list of all works consulted or referred to in the assignment.
- Convenience sample A sample chosen because it is easily accessed.
- Data All the information and facts collected.
- Dependent variable A variable which takes on values when an independent variable is deliberately altered.
- Empirical Based on observation, experiment and recording.
- Epistemology The philosophy of knowledge and ways of knowing.
- Ethics Protocols adopted to ensure research is confidential and sensitive
- Ethnography Study of a group’s culture, customs and behaviour by observation and recording.
- Experiment Obtaining data by quantitative methods with scientific precision and control.
- Generalisability This tests whether your results can be applied or be useful to others.
- Hypothesis A tentative or supposed proposition based on observed happenings or theories about a testable relationship or quality.
- Independent variable The variable controlled by the researcher to investigate its effects on others.
- Instrument Any research ‘tool’ such as a questionnaire used to obtain data.
- Interpretative Subjective explaining and interpreting data.
- Interval scale A scale where test and measurement results have supposedly equal intervals.
- Nominal scale Scales designed to label or categorise groups and their qualities.
- Ordinal scale Ranking people according to how they possess a given quality.
- Positivist Considering only observable, objective facts.
- Purposive sample A non-random sample of those believed to be representative.
- Questionnaire A structured set of written questions to obtain information from a sample.
- Random sample Every member of the population has an equal chance of being selected.
- Ratio scale A scale that has intervals and an absolute zero.
- Reliability Would you get the same results if you repeated the procedure?
- Research question A proposed focus of inquiry; what you want to find out.
- Sample A group selected from a studied population which supposedly possesses the same characteristics as the whole.
- Stratified sample A proportional sample selected to represent an identifiable group
- Survey A method of obtaining data about variables from a population or a sample.
- Systematic sample A sample taken at equal, regular intervals, such as every tenth name on a register.
- Validity How far you are actually measuring the variable that you say you are measuring.

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Research types

These definitions are provided by the Learning Development Centres at Glasgow Caledonian University. We also have a research glossary to explain what the words mean.

- Action research Where the researcher is involved in a situation, makes changes and observes the effects.
- Case-control study An observational study in which the characteristics of a particular group are compared with those of selected people without the characteristic under study. For example, people with a particular disease (the case study) are compared with a group without the disease (the control group).
- Case report A description of one interesting or unusual case
- Case series A description of several cases in which no attempt is made to answer specific hypotheses, or to compare the results with another group of cases.
- Case study Research into a particular business or defined institution, describing and investigating the issues and relationships.
- Cohort study An observational study of a group of people with a specified characteristic who are followed up over a period of time in order to detect any changes to the research question over time. Comparison may be made with a control group.
- Controlled trial An experimental study in which an intervention is applied to one group of people and the outcome compared with that in a similar group (controls) not receiving the intervention.
- Correctional research Exploring the possible relationships between variables.
- Cross sectional study is a survey of a particular phenomena, for example, the frequency of disease or risk factors in a defined population at one particular time.
- Descriptive research Describing what exists in your sample, without analysing variables.
- Historical research Structured study of past events to help understand current problems.
- Qualitative Concerned with description, qualities and observation – used to investigate views and opinions.
- Quantitative Concerned with measurement and numbers – used to demonstrate incidence and prevalence
- Questionnaire A structured set of written questions to obtain information from a sample.
- Semi-structured A schedule for interviews which has a partially planned framework.
- Structured A completely planned framework of interview questions.
- Survey A method of obtaining data about variables from a population or a sample.

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Qualitative research questions

#### Research questions

- Do the authors show the fittingness of the participants to represent the population by:
  - showing a wide representation of the population who have knowledge of the phenomenon
- Randomisation would not be used in this approach
- Types of sampling detailed in the research design page
- How do all the above issues affect the transferability of the findings?
- Small numbers are acceptable

#### Data collection

The authors need to explain in detail how they collected the data:

- where it occurred and the influences this may have on the credibility of the data.
- the influence of the researchers on the data gathered
- how it was collected, the influences on the participants and the data. For example, taping, interpretation, and recall.
- member checking of raw data, that is, was it taken back to the participants for verification.
- How does all this affect the research?

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.

# Quantitative research questions

#### Research questions

- Do the authors totally ramdomise the sample to make it more representative of the population? How do they do this?
- This reduces bias and therefore makes the findings more generalisable.
- If a sample of convenience is used, then to reduce bias it needs to be large enough to be representative.
- Sometimes a large convenient sample may be randomised into smaller groups.
- Types of sampling are detailed in the research design page.
- Did they use statistical calculations (power calculations) to determine how many people were needed in the sample to make the findings generalisable.
- Large numbers required

#### Data collection

The authors need to explain in detail how they collected the data:

- what tools have been used to collect the data?
- have the authors shown them to be valid (measure what they are supposed to measure) and reliable (consistent over time)?
- if it is a person or machine obtaining the data has this been shown to have inter-rater reliability between each assessment?
- are interventions shown to be consistent for each subject?
- How does all this affect the research?

####

######
SMIRK - How to assess a research article by GCU School of Health and Life Sciences modified by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution 4.0 International License.