Journal Article Reporting Standards for Qualitative Primary, Qualitative Meta-Analytic, and Mixed Methods Research in Psychology

Journal Article Reporting Standards for Qualitative Primary, Qualitative

Meta-Analytic, and Mixed Methods Research in Psychology: The APA

Publications and Communications Board Task Force Report

Heidi M. Levitt

University of Massachusetts Boston

Michael Bamberg

Clark University

John W. Creswell

University of Michigan Medical School

David M. Frost

University College London

Ruthellen Josselson

Fielding Graduate University

Carola Suárez-Orozco

University of California, Los Angeles

The American Psychological Association Publications and Communications Board Working

Group on Journal Article Reporting Standards for Qualitative Research (JARS–Qual Working

Group) was charged with examining the state of journal article reporting standards as they applied

to qualitative research and with generating recommendations for standards that would be appro-

priate for a wide range of methods within the discipline of psychology. These standards describe

what should be included in a research report to enable and facilitate the review process. This

publication marks a historical moment—the first inclusion of qualitative research in APA Style,

which is the basis of both the Publication Manual of the American Psychological Association

(APA, 2010) and APA Style CENTRAL, an online program to support APA Style. In addition to

the general JARS–Qual guidelines, the Working Group has developed standards for both quali-

tative meta-analysis and mixed methods research. The reporting standards were developed for

psychological qualitative research but may hold utility for a broad range of social sciences. They

honor a range of qualitative traditions, methods, and reporting styles. The Working Group was

composed of a group of researchers with backgrounds in varying methods, research topics, and

approaches to inquiry. In this article, they present these standards and their rationale, and they

detail the ways that the standards differ from the quantitative research reporting standards. They

describe how the standards can be used by authors in the process of writing qualitative research

for submission as well as by reviewers and editors in the process of reviewing research.

Keywords: qualitative research methods, qualitative meta-analysis, reporting standards,

mixed methods, APA Style

Historically, APA Style, which is the basis for both the

Publication Manual of the American Psychological Associ-

ation (hereinafter referred to as the Publication Manual;

APA, 2010) and APA Style CENTRAL, has defined the

standards and style of research reporting for psychology as

well as many other social science journals. APA Style,

however, has not included reporting standards for qualita-

tive research. As a result, authors preparing reports of

Heidi M. Levitt, Department of Psychology, University of Massachusetts

Boston; Michael Bamberg, Department of Psychology, Clark University; John

W. Creswell, Department of Family Medicine, University of Michigan Med-

ical School; David M. Frost, Department of Social Science, University College

London; Ruthellen Josselson, School of Psychology, Fielding Graduate Uni-

versity; Carola Suárez-Orozco, Graduate School of Education, University of

California, Los Angeles.

The authors of this article are members of the APA Publications

and Communications Board Working Group on Qualitative Research

Reporting Standards (Working Group). The Working Group thanks

the APA Publications and Communications Board, the Society for

Qualitative Inquiry in Psychology’s International Committee, and the

Council of Editors for comments and suggestions on a draft of this

article. This report was prepared with assistance from Emily Leonard

Ayubi and Anne Woodworth.

Correspondence concerning this article should be addressed to Heidi M. Levitt,

Department of Psychology, University of Massachusetts Boston, 100 Morrissey

Boulevard, Boston, MA 02466. E-mail: [email protected]

American Psychologist

0003-066X/18/$12.00 http://dx.doi.org/10.1037/amp0000151

qualitative and mixed methods research have faced chal-

lenges when deciding how to prepare manuscripts for sub-

mission. The American Psychological Association (APA)

standards often did not make sense for their inquiry tradi-

tions, methods, or research goals. Similarly, journal editors

and reviewers were often confused about how reports

should be evaluated. Should they insist that qualitative

research articles model the reporting style and include com-

ponents that were helpful for evaluating quantitative re-

search? Given that qualitative research involves a plurality

of inquiry traditions, methods, and goals, it was uncertain

how to best adapt the existing standards. Instead, standards

of reporting were needed that can be applicable to, and

coherent with, diverse qualitative research methods.

The Working Group on Journal Article Reporting Stan-

dards for Qualitative Research (JARS–Qual Working

Group) was formed to develop recommendations to the

APA Publications and Communications Board. Their

charge was to form recommendations to become the basis

for journals and publications using APA Style. They strove

to form reporting standards that could advance qualitative

research in a way that is sensitive to traditions in the field,

while recognizing the complexity of addressing constituen-

cies who have quite varied language and assumptions. To be

clear, the standards developed are focused on the act of

reporting—that is, they articulate what information should

be expected in a manuscript to enable its adequate evalua-

tion. They are an explicit set of criteria for authors to reflect

upon in preparing manuscripts and for reviewers to consider

while evaluating the rigor of a manuscript. They were not

developed to act as a primer on qualitative research tradi-

tions, to teach how to design qualitative research, to de-

scribe the evaluation of rigor, or to articulate the justifica-

tions for using certain procedures. Instead, the Working

Group reviewed the literature on qualitative research report-

ing standards and considered a broad range of qualitative

methods and traditions in the process of shaping these

standards. This article articulates the process of developing

their recommendations and presents the reporting standards

that were generated for general qualitative research as well

as for qualitative meta-analyses and mixed methods re-

search.

Reviewing Qualitative Research

Research employing qualitative methods has made sig-

nificant contributions to psychology since its early devel-

opment; however, at the turn of the 19th century, psychol-

ogists began to define their field by its focus on

experimental and correlational research methods (Danziger,

1990). Instead of supporting multiple approaches to inquiry

and philosophical assumptions about the research endeavor,

qualitative research was thought to threaten the credibility

of psychology as a science and was marginalized (Harré,

2004). This turn was poignantly recounted in Danziger’s

(1979) description of the systematic erasure of Wundt’s

cultural psychology tradition (based within introspective

approaches to research) in favor of his psychophysiology

laboratory (based within experimental approaches). Al-

though qualitative methods remained in use after a postposi-

tivist approach came into vogue, they were not systematized

and tended not to be reported as part of the formal inquiry

process within psychology (Wertz, 2014). Over the past

half-century, however, there has been a gradual revival of

qualitative methods and a great number of qualitative meth-

ods now have been detailed and advanced in the field. Many

of the methods that have been embraced in psychology have

had multidisciplinary roots in philosophy, social sciences,

or practice disciplines, such as nursing (e.g., Giorgi, 2009;

Glaser & Strauss, 1967). Although qualitative methods have

become accepted in the field, as indicated by their increased

publication in journals, increased representation in graduate

coursework and dissertations (Ponterotto, 2005a, 2005c),

and the retitling of APA Division 5 to Quantitative and

Qualitative Methods, many psychologists are still unfamil-

iar with these approaches to investigation and continue to

marginalize them.

What Are Qualitative Methods?

The term qualitative research is used to describe a set of

approaches that analyze data in the form of natural language

(i.e., words) and expressions of experiences (e.g., social

interactions and artistic presentations). Researchers tend to

centralize the examination of meanings within an iterative

process of evolving findings—typically viewing this pro-

cess as driven by induction (cf. Wertz, 2010)—and viewing

subjective descriptions of experiences as legitimate data for

analyses. An iterative process of inferences means that

researchers tend to analyze data by identifying patterns tied

to instances of a phenomenon and then developing a sense

of the whole phenomenon as informed by those patterns.

Seeing the pattern can shift the way the whole is understood,

just as seeing a pattern in the context of the whole phenom-

enon can shift the way it is understood. In this way, a

number of writers have theorized that this hermeneutic

circle contains fundamental inferential processes within

qualitative inquiry (see Levitt, Motulsky, Wertz, Morrow, &

Ponterotto, 2017; Osbeck, 2014; Rennie, 2012; Wertz et al.,

2011). This cycle is self-correcting; as new data are ana-

lyzed, their analysis corrects and refines the existing find-

ings.

Qualitative data sets typically are drawn from fewer

sources (e.g., participants) than quantitative studies, but

include rich, detailed, and heavily contextualized descrip-

tions from each source. Following from these characteris-

tics, qualitative research tends to engage data sets in inten-

sive analyses, to value open-ended discovery rather than

QUALITATIVE RESEARCH REPORTING STANDARDS

verification of hypotheses, to emphasize specific histories

or settings in which experiences occur rather than expect

findings to endure across all contexts, and to recursively

combine inquiry with methods that require researchers’

reflexivity (i.e., self-examination) about their influence

upon research process. As such, qualitative reports need to

be evaluated in terms of their own logic of inquiry. The data

or findings from these analyses may or may not be trans-

formed into future numerical quantification in quantitative

or mixed methods analyses.

There is a broad range of qualitative methods, however,

and they stem from a diversity of philosophical assump-

tions, intellectual disciplines, procedures, and goals (e.g.,

K. J. Gergen, 2014; K. J. Gergen, Josselson, & Freeman,

2015). Also, they use varied forms of language in detailing

their processes and findings, which complicates the devel-

opment of uniform reporting standards. To provide a few

examples, methods more widely used in psychology that

fall under this rubric include narrative (e.g., Bamberg, 2012;

Josselson, 2011), grounded theory (e.g., Charmaz, 2014;

Glaser & Strauss, 1967), phenomenological (e.g., Giorgi,

2009; Smith, 2004), critical (e.g., Fine, 2013; Steinberg &

Cannella, 2012), discursive (e.g., Pea, 1993; Potter & Weth-

erell, 1987), performative (e.g., M. M. Gergen & K. J.

Gergen, 2012), ethnographic (e.g., Suzuki, Ahluwalia, Mat-

tis, & Quizon, 2005; Wolcott, 2010), consensual qualitative

research (e.g., Hill, 2012), case study (e.g., Fishman &

Messer, 2013; Yin, 2013), psychobiography (e.g., Schultz,

2005), and thematic analysis (e.g., Braun & Clarke, 2006;

Finfgeld-Connett, 2014) approaches. Many of these ap-

proaches can take multiple forms by virtue of shifts in

philosophical assumptions or the evolution of their proce-

dures. Reviewing or conducting qualitative research does

not only entail a familiarity with broad distinctions between

qualitative and quantitative methods, then, but requires a

familiarity with the method used, the form selected of that

method, and the process of adapting methods and proce-

dures to the goals, approach to inquiry, and characteristics

of a given study.

What Research Goals Do Qualitative

Methods Advance?

Qualitative methods are increasingly prevalent and cen-

tral in research training (Ponterotto, 2005a, 2005c). Quali-

tative designs are used for research goals, including, but not

limited to, developing theory, hypotheses, and attuned un-

derstandings (e.g., Hill, 2012; Stiles, 1993), examining the

development of a social construct (e.g., Neimeyer, Hogan,

& Laurie, 2008), addressing societal injustices (e.g., Fine,

2013), and illuminating social discursive practices—that is,

the way interpersonal and public communications are en-

acted (e.g., Parker, 2015). In particular, these methods have

been found useful to shed light upon sets of findings or

literatures that are contradictory, problematic, or ill-fitting

for a subpopulation (e.g., Chang & Yoon, 2011); to give a

voice to historically disenfranchised populations whose ex-

periences may not be well-represented in the research liter-

ature (e.g., American Psychological Association Presiden-

tial Task Force on Immigration, 2012; Frost & Ouellette,

2011); and to develop initial understandings in a less ex-

plored area (e.g., Creswell, 2013). Qualitative methods may

stand alone, serve as the basis for metasyntheses, or be

combined with quantitative methods in mixed methods de-

signs. This article will consider all three contexts in turn.

The Need for Qualitative Reporting Standards

Without the guidance of reporting standards, qualitative

researchers, reviewers, and editors have faced numerous

complications (e.g., Levitt et al., 2017). Authors have suf-

fered from conflicting manuscript expectations in the style

or content of reporting. For instance, they may be asked to

adhere to standards and rhetorical styles that are inappro-

priate for their methods. Authors may also be asked to

educate reviewers about basic qualitative methods’ assump-

tions or to defend qualitative methods as a field in articles

focused otherwise. Also, editors and reviewers face chal-

lenges when they lack training in qualitative methods,

which may make them uncertain about what information

should be reported and how qualitative approaches may be

distinctive. Reporting guidelines can support authors in

writing manuscripts, encourage reviewers to better evaluate

qualitative methods, and assist editors in identifying when

reviewers’ responses are appropriate for a given article.

Rhetorical Distinctions of Qualitative Research

In developing our recommendations, we worked to iden-

tify reporting standards that could facilitate the review of

research and that would be applicable across a range of

qualitative traditions. We recognized, however, that there

are characteristic features in the general form reporting of

qualitative research that may be unfamiliar to some readers

(Gilgun, 2005; Sandelowski & Leeman, 2012; Walsh,

2015). The following sections describe key features of this

rhetorical style and responses to facilitate adequate reviews

in light of these features.

Representation of Process Rather Than

Standardized Section Demarcation

Qualitative approaches to inquiry may utilize distinct

styles of reporting that may still be unfamiliar to many

psychologists and social scientists (Sandelowski & Leeman,

2012). These can include a narrative style of reporting, in

which the research endeavor is presented as a story. These

reports may be organized thematically or chronologically.

They may be presented in a reflexive first-person style,

LEVITT ET AL.

detailing the ways in which researchers arrived at questions,

methods, findings, and considerations for the field. We

encourage reviewers and editors to learn to recognize

whether reporting standards have been met regardless of the

rhetorical style of the research presentation. In particular,

qualitative researchers often combine Results and Discus-

sion sections, as they may see both as intertwined and

therefore not possible to separate a given finding from its

interpreted meaning within the broader frame of the analy-

sis. Also, they may use headings that reflect the values in

their tradition (such as “Findings” instead of “Results”) and

omit ones that do not. As long as the necessary information

is present in a given manuscript, we do not suggest man-

dating that manuscripts be segmented into the same sections

and subsections that organize the presentation of the stan-

dards in the present article.

An Ethic of Transparency

Qualitative researchers often are concerned with how

their expectations and assumptions might influence the re-

search process. As a result, qualitative traditions tend to be

based within approaches to inquiry that value transparency

in the reporting of data-collection and data-analytic strate-

gies as well as ethical procedures. Researchers typically

enact this value by communicating both their perspectives

and their influence upon the research process. As such,

many traditions prefer not to use objectivist rhetoric and

instead tend to prefer to use reporting styles that make overt

the researchers’ influences on data collection and analysis

(Morrow, 2005; Rennie, 1995). Following from this con-

cern, for example, is a preference for the use of first-person

and personal narratives to convey the positions and experi-

ences of researchers. Because of the wide range of qualita-

tive approaches, it is not possible to describe how reporting

might be tailored to every approach, but we consider how

approach to inquiry might influence the reporting of data

collection, analysis, and ethics.

Data collection often involves processes of self-reflection

and making explicit how investigators’ values guided or

limited the formation of analytic questions. Similarly, the

demonstration of analyses tends to transparently convey the

ways that interpretations were shaped or observations were

formed. Across approaches to inquiry, qualitative research-

ers embrace a reporting standard of transparency, as it

enhances methodological integrity (Levitt et al., 2017; Ren-

nie, 1995). When researchers openly describe the ways their

perspectives guided their research (e.g., in critical methods),

this transparency provides the reader with information that

permits an understanding of their goals and increases the

trustworthiness of the researchers’ reports. When transpar-

ency involves describing how researchers approached the

task of setting aside their own expectations (e.g., in empir-

ical phenomenology; Giorgi, 2009), it also enhances the

trust in the report, as it demonstrates the efforts by which the

researcher sought to remain open to the phenomenon. In

addition, by recognizing their own standpoint and position-

ality in relation to the topic of the research and the popu-

lation under study (e.g., Harding, 1992), researchers en-

hance the credibility of their claims by simultaneously

pointing out their contextual embeddedness (or lack thereof)

and its role in the interpretative process (e.g., Hernández,

Nguyen, Casanova, Suárez-Orozco, & Saetermoe, 2013).

Because the data-collection and data-analytic strategies

may be shaped recursively, the process of inquiry shifts

across the course of a qualitative study. Incoming data

might alter the questions that are asked and preliminary

findings might encourage new recruitment procedures. The

shifting of procedures in use and, sometimes, extensive

interpersonal contact with participants can mean that re-

search ethics within a study require continual reconsidera-

tion (see Haverkamp, 2005; Josselson, 2007). For instance,

if participants find it unduly taxing to answer questions

related to a traumatic experience, those questions may need

to be dropped or altered, and other supports might need to

be recruited for the study to continue—even within the

process of a single interview. Qualitative researchers strive

to be explicit on the ways their procedures and perspectives

might influence their study and how they might shift across

the study. For these reasons, the value of transparency is at

the root of the reporting standards across qualitative meth-

ods.

Contextualization

Because their work tends to focus on human experiences,

actions, and social processes, which fluctuate, qualitative

researchers do not aim to seek natural laws that extend

across time, place, and culture, but to develop findings that

are bound to their contexts. Qualitative researchers report

their research to reflect the situatedness of their research in

a number of ways. First, as described in the previous sec-

tion, the context of the investigators themselves is an issue.

Researchers’ relationship to the study topic, with their par-

ticipants, and to related ideological commitments may all

have bearing upon the inquiry process. Second, qualitative

researchers describe the context within which a phenomenon

or study topic is being construed as well. For instance,

studying sexual orientation in the 2000s in the New England

would be quite different from studying it in Russia in

the1980s. Third, they also describe the contexts of their data

sources. Interviews with immigrants from Mexico and im-

migrants from England might relay very different experi-

ences and concerns.

In addition to describing the phenomena, data sources,

and investigators in terms of their location, era, and time

periods, qualitative researchers seek to situate these factors

in relation to relevant social dynamics. A description of

QUALITATIVE RESEARCH REPORTING STANDARDS

their position within a social order or key relationships can

aid readers in understanding and transferring a study’s find-

ings. For instance, to the extent that experiences of margin-

alization and privilege influence the issue under investiga-

tion, the explication of these relationships is necessary. For

instance, African American students in predominantly

White institutions of learning may have experiences with a

phenomenon that are distinct from those in historically

Black ones because of the different minority stressors in

those contexts. This contextual description, along with the

need for exemplification of the analytic process, and trans-

parent reporting all contribute to the length of a qualitative

article.

Length of Manuscripts

Strong qualitative and mixed methods manuscripts both

tend to be longer than quantitative articles and require more

manuscript pages. Because readers are less familiar with

qualitative methods, and methods are often idiosyncratically

adapted to fit a problem at hand, the Method sections may

need to detail procedures and rationales at each point in the

analysis. In addition, qualitative method descriptions entail

a discussion of the researchers’ own backgrounds and be-

liefs when approaching and engaging in a study. Results

sections also tend to be lengthy because the methodological

integrity of qualitative methods is enhanced within a de-

monstrative rhetoric in which authors show how they

moved within the analysis from their raw data to develop

their findings.

When journals expect authors of qualitative research to

present their work within restrictive page limits, authors

must often leave out parts of their manuscript that justify the

use of their methods and/or present results less convinc-

ingly. Because reviewers may hold differing opinions, jour-

nal expectations may be challenging to predict and authors

may be unsure which aspects to emphasize. It can be helpful

for editors and reviewers to keep in mind that qualitative

articles typically have concise literature reviews and discus-

sions, and have often excluded central information to meet

page restrictions. If further information within an article can

be clarifying, editors and reviewers can engage authors

within the review process to assist them in identifying

which aspects of a manuscript should be prioritized.

Some journals indicate in their instructions to authors that

they will allocate extra pages to support the adequate de-

scription of qualitative methods rather than expect qualita-

tive reporting to conform to quantitative standards. If an

extension is not possible in printed versions of an article,

journals may want to permit qualitative manuscripts to

submit longer Method or Results sections for review, with

the understanding that editors can direct some supplemental

material to be posted on a website postreview. This practice

can help support the appropriate review and reading of

qualitative research when page lengths cannot be extended.

In general, however, we agree with the recommendation of

the Society for Qualitative Research in Psychology task

force (Levitt et al., 2017) that providing an extension of at

least 10 pages for qualitative research (as is the practice of

the Journal of Counseling Psychology), and more for mixed

methods research, would be ideal, and that this decision

should be informed by a journal’s existing page limits and

its desire to support reporting that permits an adequate

appraisal of articles by its readers and reviewers. The fol-

lowing two sections describe responses for authors, review-

ers, and editors, given the specific rhetorical features of

qualitative methods reporting.

Letter to Editor

Before a research review begins, researchers submit their

work to a journal editor, who assigns reviewers to a project.

Information that is advisable to share in these letters in-

cludes a description of the method used, the type of phe-

nomenon explored, and the participants or form of data

studied. This description can aid editors in selecting review-

ers who are competent to review a particular manuscript and

can suggest to informed editors that the article might use a

reporting style in line with a specific tradition of inquiry. In

these letters, authors who have collected data from human

participants should provide assurance that relevant ethical

processes of data collection and consent were used (e.g.,

institutional review board approval).

If relevant, there should be a description of how the

current analysis is related to already-published work from

the same data set. It is common for qualitative researchers to

divide results into several articles with distinct foci because

of the richness of the data and the challenges in meaning-

fully representing that work within a journal-length manu-

script. Thus, researchers will want to assure the editor of the

distinct focus of a submission and describe how it emerged

from a subset of data that has not been published yet or that

has been published with an alternative goal (e.g., a content-

focused article vs. a method-focused article).

Selecting Reviewers and Communicating About

Reviewers’ Competencies

Although much of this article speaks to the concerns of

authors preparing manuscripts, this section addresses

how editors and reviewers can ensure an adequate review

of qualitative research. Because of the need to understand

how to evaluate qualitative research across a range of

research traditions and methods, we recommend that

journals have at least one associate, consulting, or action

editor who has expertise in multiple qualitative ap-

proaches to inquiry. Although these general standards

can assist in the review process, they do not replace the

LEVITT ET AL.

need to learn about how to use or evaluate qualitative

methods. Editors can use the information in a manuscript

and its accompanying letter to the editor to seek review-

ers who are appropriate for both the content and the

methods of the manuscript. Although it may not be

possible to obtain reviewers who have expertise in both

the design and the content area, editors should be aware

of the type of expertise reviewers bring to evaluate the

manuscript or should ask reviewers to clarify this. In this

way, editors might appropriately prioritize content-

related concerns of some reviewers and method-related

concerns of others. This process is similar to the process

of assigning quantitative manuscripts for review, but

differences exist.

Presumably, editors would expect that most reviewers of

quantitative research with terminal degrees would have had

some graduate coursework in, and experience using, quan-

titative methods. These experiences provide reviewers with

an understanding of both the theory underlying analyses and

ideal approaches and how research methods often require

adaption in practice. Although a similar level of expertise is

needed to review qualitative research, most psychology

programs still do not require training in the use of qualita-

tive methods, although the number is growing (Ponterotto,

2005a). As a result, it can be challenging for editors to

assess reviewers’ competence by their degree. Systems that

invite reviewers to indicate their methodological areas of

expertise can be helpful in this regard. Examinations of

potential reviewers’ past publications can be useful as well.

In any case, reviewers should assess their own degree

and scope of competence. To provide a competent, com-

plete review, a reviewer would have a depth of under-

standing of (a) the topic being studied, (b) the specific

method in use (keeping in mind that multiple versions

exist of many qualitative methods and these may be

based in varying traditions of inquiry; see Levitt, 2015),

and (c) the processes of appropriately adapting qualita-

tive methods to specific projects. If a reviewer does not

have experience using the specific method at hand or in

adapting qualitative methods for use in research projects,

it can be helpful for the reviewer to check with the editor

on the appropriateness of the assignment. The editor still

may request that a reviewer provide commentary on the

literature review from a position as a content expert. At

minimum, one of the reviewers should have expertise and

experience as a qualitative researcher—preferably in a

method similar to the one in use. In any case, reviewers

should clarify the basis of their expertise in their reviews

so that editors can consider how to weigh their remarks in

relation to other reviewers’ comments. Regardless of

reviewers’ areas of expertise, they should be mindful of

the distinctive reporting standards in the JARS-Qual. As

well, the APA has produced a video that provides guid-

ance on reviewing qualitative manuscripts, free of

charge, that can be a helpful resource for reviewers

(Levitt, 2016), based upon recommendations for design

and review within the task force report of the Society of

Qualitative Inquiry in Psychology (Levitt et al., 2017).

Editors may wish to support reviewers by routinely point-

ing to these resources in review request letters.

Process of Developing the JARS–Qual

The JARS–Qual Working Group met in Washington, DC,

at the APA for an intensive 2-day meeting to develop the

core of the JARS–Qual. Prior to this meeting, the members

reviewed readings on qualitative methods reporting (e.g.,

Madill & Gough, 2008; Neale & West, 2015; O’Brien,

Harris, Beckman, Reed, & Cook, 2014; Tong, Flemming,

McInnes, Oliver, & Craig, 2012; Tong, Sainsbury, & Craig,

2007; Walsh, 2015; Wisdom, Cavaleri, Onwuegbuzie, &

Green, 2012; Wong, Greenhalgh, Westhorp, Buckingham,

& Pawson, 2013), a task force report to the Society for

Qualitative Inquiry in Psychology, a section of APA Divi-

sion 5, on the recommendations regarding publishing and

reviewing of qualitative research (Levitt et al., 2017), and

the initial quantitative APA journal article reporting stan-

dards (APA Publications and Communications Board

Working Group on Journal Article Reporting Standards,

2008). The work of these leaders in qualitative methods

provided valuable suggestions for us to consider in the

formation of our standards. When they met, the group

reviewed a summary chart of these readings developed by

the JARS–Qual Working Group chair (Heidi M. Levitt).

In this process, the Working Group decided that separate

modules were needed for qualitative meta-analyses (some-

times called metasyntheses) as well as for mixed methods

research. The members discussed the items on the chart and

decided together on the items to be included as the basis of

the JARS–Qual. The chair (Heidi M. Levitt) developed an

initial draft based on the conclusions of this meeting, and

the members edited and added into this version. They then

divided into two subgroups to develop modules on qualita-

tive meta-analysis article reporting standards (QMARS; Mi-

chael Bamberg, Ruthellen Josselson, and Heidi M. Levitt) and

on mixed methods article reporting standards (MMARS; John

W. Creswell, David M. Frost, and Carola Suárez-Orozco).

These modules were based on the general JARS–Qual stan-

dards and their efforts to maintain relevance to a wide range

of qualitative methods, but specified when there were dif-

ferences in the reporting standards that were particular to

these two approaches to research. The subgroups presented

their findings to the larger group for feedback. The group

continued to engage in cycles of seeking feedback and

creating revisions until the Working Group members were

satisfied with the recommendations. Then they were pre-

sented to the APA Council of Editors, the International

Committee of the Society for Qualitative Research in Psy-

QUALITATIVE RESEARCH REPORTING STANDARDS

chology, and the APA Publications and Communications

Board; feedback was requested and revisions were then

made. The APA Publications and Communications Board

endorsed the recommendations. In addition, the JARS–Qual

Working Group presented their recommendations for re-

porting standards at the annual convention of the APA in

2016 (Levitt, Bamberg, Frost, & Josselson, 2016) to seek

feedback and comments from the research community. Al-

though the text in this article may be reworked in commu-

nications of APA Style, such as the Publication Manual and

APA Style CENTRAL, the reporting standards should re-

main the same.

The JARS–Qual Working Group recognized that before

the standards could be presented, the terms that will be used

in their report needed to be defined. The following sections

relay this information, which will be relevant to both the

JARS–Qual and its modules.

Defining Terms

Although we welcome researchers to use the terms that

reflect their local research strategies and values, we needed

to settle on a vocabulary for use in the description of our

recommendations for reporting standards. As a result, we

define here terms that are used throughout our article. We

use the term approach to inquiry to refer to the philosoph-

ical assumptions that describe researchers’ understanding of

the research traditions or strategies. Researchers may wish

to make explicit these assumptions, especially when they

are useful in illuminating the research process. These as-

sumptions are described in varied literatures as the research-

ers’ epistemological beliefs, worldview, paradigm, strate-

gies, or research traditions (Creswell, 2013; Morrow, 2005;

Ponterotto, 2005b). For instance, they could indicate

whether their approaches to inquiry are descriptive, inter-

pretive, feminist, psychoanalytic, postpositivist, critical,

postmodern, or constructivist; theorists often carve these

philosophies along different lines (e.g., Guba & Lincoln,

2005; Madill & Gough, 2008; Mertens, 2010; Parker, 2004).

Although some research is firmly grounded in one or more

sets of these assumptions, research may also be question-

driven and conducted pragmatically (Morgan, 2007).

The term data-collection strategies refers to the many

ways qualitative researchers gather data. These can include

activities such as conducting archival research, focus

groups, interviews, ethnographic observation, fieldwork,

media searches, and reflexive note-taking. In contrast, the

term data-analytic strategies refers to the procedures used

to analyze the data (e.g., constant comparison, eidetic re-

duction, the generation of themes). These strategies may be

creatively combined in response to the specific goals of a

research project, as is typical of the bricoleur tradition in

qualitative research (e.g., Denzin & Lincoln, 2005; Kuck-

artz, 2014; McLeod, 2011), in which researchers generate

their own design by assembling procedures to best meet the

goals and characteristics of a research project.

When we refer to research design, we mean the combi-

nation of approaches to inquiry, data-collection strategies,

and data-analytic strategies selected for use in a given study.

Data-collection and data-analytic strategies may be in-

formed by established qualitative methods or designs (e.g.,

grounded theory: Glaser & Strauss, 1967; narrative:

Lieblich, Tuval-Mashiach, & Zilber, 1998; phenomenology:

Giorgi, 2009), but because many of these methods have

been utilized within varied approaches to inquiry (e.g.,

Charmaz, 2014; Glaser & Strauss, 1967), a complete de-

scription of a design should articulate each of these ele-

ments, even when an established method or design is in use.

Because qualitative researchers describe their analyses

and frameworks using diverse perspectives and terminol-

ogy, we encourage authors to translate our terms into those

of their own preferred approaches, taking care to define

terms for readers. We also encourage reviewers and editors

to view our terms as placeholders that may be usefully

varied by authors to reflect the values of their research

traditions. We recognize that our language inevitably carries

philosophical implications (e.g., do we discover, under-

stand, or coconstruct findings?). This said, we have worked

to generate substantive recommendations that are congruent

with, and would enhance, the reporting of qualitative meth-

ods when imported within a diverse range of approaches.

Methodological Integrity

Reporting standards indicate the content that is needed so

that the rigor of research can be evaluated. Qualitative

researchers have long sought language to describe rigor in

their approach. Trustworthiness is a concept that qualitative

researchers often use to reflect the idea that the evaluation

of the worth of a qualitative research presentation is based

in the judgments of its readers and its ability to be presented

to them in a convincing manner (Lincoln & Guba, 1985;

Morrow, 2005). This concept may include evaluations that

are not related to the research processes themselves (e.g.,

reputation of authors, congruence with readers’ own expec-

tations and beliefs, or cosmetic features of presentation).

Methodological integrity is a concept that has been ad-

vanced by a task force of the Society for Qualitative

Inquiry in Psychology (a section of APA Division 5), in

consultation with a broad range of leading qualitative

researchers, as the underlying methodological basis of

trustworthiness, independent of nonmethod qualities (see

Levitt et al., 2017 for details). It enriches considerations

of research design and is particularly relevant to a journal

review process in which these nonmethod aspects of

trustworthiness are not central bases of evaluation (e.g.,

cosmetic features) or are unavailable (e.g., authors’ iden-

tities, the resonance of the article for readers who differ

LEVITT ET AL.

from oneself). Instead, reviews should be focused on how

methodological processes are enacted throughout an ar-

ticle—including how well the literature review is con-

ducted to situate a study’s aims, approaches to inquiry are

selected to address those aims, methods and procedures

are used in an investigation to meet those aims, and the

articulation of implications are grounded in the methods

used and the findings produced.

Methodological integrity can be evaluated through its

two composite processes: fidelity to the subject matter

and utility in achieving research goals. Both fidelity and

utility have been conceptualized as having four central

features. Fidelity to the subject matter is the process by

which researchers select procedures that develop and

maintain allegiance to the phenomenon under study as it

is conceived within their approach to inquiry (e.g., the

phenomenon might be understood as a social construc-

tion). It is improved when researchers collect data from

sources that can shed light upon variations in the phe-

nomenon that are relevant to the research goals (data

adequacy); when they recognize and are transparent about the

influence of their own perspectives and appropriately limit that

influence within data collection (perspective management in

data collection); when they consider how these perspectives

influenced or guided their analytic process in order to enhance

their perceptiveness (perspective management in data analy-

sis); and when findings are rooted in data that support them

(groundedness).

The second composite process of methodological integ-

rity, utility in achieving research goals is the process by

which researchers select procedures that usefully answer

their research questions and address their aims (e.g., raising

critical consciousness, developing theory, deepening under-

standing, identifying social practices, forming conceptual

frameworks, and developing local knowledge). It is

strengthened when findings are considered in their con-

text—for instance, their location, time, and cultural situa-

tion (contextualization of data); when data are collected that

provide rich grounds for insightful analyses (catalyst for

insight); when analyses lead to findings that meaningfully

address the analytic goals (meaningful contributions); and

when differences within a set of findings are explained

(coherence among findings).

The evaluation of methodological integrity considers

whether the procedures used to enhance fidelity and utility

are coherent in relation to the researchers’ goals, approaches

to inquiry (e.g., philosophical assumptions), and study char-

acteristics (e.g., the particular subject matter, resources,

participants, researchers). In other words, fidelity and utility

need to be assessed in relation to the overall research design.

When procedures are used with coherence, they build a

foundation for increased confidence in the claims made.

When procedures are not used in synchrony with the study

design features, however, they will not support a foundation

of methodological integrity or might act to erode it.

Procedures that add to methodological integrity may re-

late to participant selection, recruitment, data-collection

strategies, data-analytic strategies, procedures used to check

findings (e.g., member-checking), as well as broader aspects

of the research, such as the formulation of research ques-

tions or the articulation of implications. A detailed descrip-

tion of fidelity and utility, and their constituent features, can

be found in Levitt et al. (2017). Principles can be found

therein to guide the evaluation of fidelity and utility meth-

odological integrity within both the process of research

design and manuscript review. In contrast, the standards in

the current article are concerned with the reporting of re-

search so that methodological integrity can be evaluated.

Journal Article Reporting Standards for

Qualitative Research

The reporting standards generated have been divided into

three tables that are reviewed in the following subsections.

The JARS-Qual table (see Table 1) was developed to be

the foundation of the recommended standards for

qualitative research. The standards for qualitative meta-

analyses were developed by adjusting the foundational

standards to the unique features of methods that review

primary qualitative research. The mixed methods report-

ing standards were developed while considering the stan-

dards for both qualitative and quantitative research and

identifying the unique reporting standards for designs

that integrate both of these approaches.

Table 1 has three columns. The first column contains

the topic to be reported on, which might be used to

structure an article’s section headings or might be de-

scribed in a narrative format. The second column con-

tains a description of the information to be reported. The

third column contains recommendations that are not stan-

dards but that might be useful for authors and reviewers

to consider.

Although we have developed a module on mixed methods

approaches, qualitative and quantitative analyses being re-

ported together, researchers may also combine two qualita-

tive analyses in the same study. For example, in the example

article by Frost (2011), both a content analysis and a nar-

rative analysis were conducted together to achieve the re-

searcher’s aims. In those types of articles, the reporting of

both analyses should follow the JARS-Qual guidelines.

Similar to the way that the mixed methods standards guide

authors to discuss the goals and integrate the insights of

qualitative and quantitative projects throughout their report-

ing (see Table 3), reporting two qualitative analyses in one

article should reflect upon the ways that the analyses work

together to meet the study objectives and how findings

enhance one another.

QUALITATIVE RESEARCH REPORTING STANDARDS

Table 1

Journal Article Reporting Standards for Qualitative Research (JARS-Qual): Information Recommended for Inclusion in Manuscripts

That Report Primary Qualitative Research

Paper section or element Description of information to be reported

Recommendations for authors to consider &

notes for reviewers

Title page

Title • Identify key issues/topic under consideration.

Author note • Acknowledge funding sources or contributors.

• Acknowledge conflicts of interest, if any.

Abstract • State the problem/question/objectives under investigation.

• Indicate the study design, including types of participants or data

sources, and analytic strategy, main results/findings, main

implications/significance.

• Identify five keywords.

• Authors: Consider including at least one

keyword that describes the method and one

that describes the types of participants or

phenomenon under investigation.

• Authors: Consider describing your approach to

inquiry when it will facilitate the review

process and intelligibility of your paper. If

your work is not grounded in a specific

approach to inquiry or your approach would

be too complicated to explain in the allotted

word count, however, it would not be

advisable to provide explication on this point

in the abstract.

Introduction

Description of research

problem or question

• Frame the problem or question and its context. • Reviewers: The introduction may include case

examples, personal narratives, vignettes, or

other illustrative material.

• Review, critique, and synthesize the applicable literature to identify

key issues/debates/theoretical frameworks in the relevant literature

to clarify barriers, knowledge gaps, or practical needs.

Study objectives/aims/

research goals

• State the purpose(s)/goal(s)/aim(s) of the study.

• State the target audience, if specific.

• Provide the rationale for fit of design used to investigate this

purpose/goal (e.g., theory building, explanatory, developing

understanding, social action, description, highlighting social

practices).

• Describe the approach to inquiry, if it illuminates the objectives and

research rationale (e.g., descriptive, interpretive, feminist,

psychoanalytic, postpositivist, constructivist, critical, postmodern or

constructivist, or pragmatic approaches).

• Authors: If relevant to objectives, explain the

relation of the current analysis to prior articles/

publications.

• Reviewers: Qualitative studies often legitimately

need to be divided into multiple manuscripts

because of journal article page limitations, but

each manuscript should have a separate focus.

• Reviewers: Qualitative studies tend not to

identify hypotheses, but research questions and

goals.

Method

Research design overview • Summarize the research design (data-collection strategies, data-

analytic strategies, and, if illuminating, approaches to inquiry (e.g.,

descriptive, interpretive, feminist, psychoanalytic, postpositivist,

critical, postmodern or constructivist, pragmatic approaches).

• Provide the rationale for the design selected.

• Reviewers: Method sections can be written in a

chronological or narrative format.

• Reviewers: Although they provide a method

description that other investigators should be

able to follow, it is not required that other

investigators arrive at the same conclusions,

but rather that their method should lead them

to conclusions with a similar degree of

methodological integrity.

• Reviewers: At times, elements may be relevant

to multiple sections and authors need to

organize what belongs in each subsection in

order to describe the method coherently and

reduce redundancy. For instance, the overview

and the objectives statement may be presented

in one section.

• Reviewers: Processes of qualitative research are

often iterative versus linear, may evolve through

the inquiry process, and may move between data

collection and analysis in multiple formats. As a

result, data collection and analysis sections might

be combined.

• Reviewers: For the reasons above and because

qualitative methods often are adapted and

combined creatively, requiring detailed description

and rationale, an average qualitative Method section

typically is longer than an average quantitative

Method section.

LEVITT ET AL.

Table 1 (continued)

Paper section or element Description of information to be reported

Recommendations for authors to consider &

notes for reviewers

Study participants or data

sources

Researcher

description

• Describe the researchers’ backgrounds in approaching the study,

emphasizing their prior understandings of the phenomena under

study (e.g., interviewers, analysts, or research team).

• Describe how prior understandings of the phenomena under study

were managed and/or influenced the research (e.g., enhancing,

limiting, or structuring data collection and analysis).

• Authors: Prior understandings relevant to the

analysis could include, but are not limited to,

descriptions of researchers’

demographic/cultural characteristics,

credentials, experience with phenomena,

training, values, decisions in selecting archives

or material to analyze.

• Reviewers: Researchers differ in the extensiveness

of reflexive self-description in reports. It may not

be possible for authors to estimate the depth of

description desired by reviewers without guidance.

Participants or other

data sources

• Provide the numbers of participants/documents/events analyzed.

• Describe the demographics/cultural information, perspectives of

participants or characteristics of data sources that might influence

the data collected.

• Describe existing data sources, if relevant (e.g., newspapers,

Internet, archive).

• Provide data repository information for openly shared data, if

applicable.

• Describe archival searches or process of locating data for analyses,

if applicable.

Researcher–

participant

relationship

• Describe the relationships and interactions between researchers and

participants relevant to the research process and any impact on the

research process (e.g., was there a relationship prior to research, are

there any ethical considerations relevant to prior relationships).

Participant recruitment

Recruitment process • Describe the recruitment process (e.g., face-to-face, telephone, mail,

e-mail, recruitment protocols).

• Describe any incentives or compensation, and provide assurance of

relevant ethical processes of data collection and consent process as

relevant (may include institutional review board approval, particular

adaptations for vulnerable populations, safety monitoring).

• Describe the process via which the number of participants was

determined in relation to the study design.

• Provide any changes in numbers through attrition and final number of

participants/sources (if relevant, refusal rates or reasons for dropout).

• Describe the rationale for decision to halt data collection (e.g.,

saturation).

• Convey the study purpose as portrayed to participants, if different from

the purpose stated.

• Reviewers: There is no agreed-upon minimum

number of participants for a qualitative study.

Rather, the author should provide a rationale

for the number of participants chosen.

• Authors/Reviewers: The order of the

recruitment process and the selection process

and their contents may be determined in

relation to the authors’ methodological

approach. Some authors will determine a

selection process and then develop a

recruitment method based upon those criteria.

Other authors will develop a recruitment

process and then select participants

responsively in relation to evolving findings.

Participant selection • Describe the participants/data sources selection process (e.g.,

purposive sampling methods such as maximum variation, diversity

sampling, or convenience sampling methods such as snowball

selection, theoretical sampling), inclusion/exclusion criteria.

• Provide the general context for study (when data were collected,

sites of data collection).

• If your participant selection is from an archived data set, describe

the recruitment and selection process from that data set as well as

any decisions in selecting sets of participants from that data set.

• Authors: A statement can clarify how the

number of participants fits with practices in

the design at hand, recognizing that

transferability of findings in qualitative

research to other contexts is based in

developing deep and contextualized

understandings that can be applied by

readers, rather than quantitative estimates of

error and generalizations to populations.

• Authors/Reviewers: The order of the

recruitment process and the selection process

and their contents may be determined in

relation to the authors’ methodological

approach. Some authors will determine a

selection process and then develop a

recruitment method based upon those

criteria. Other authors will develop a

recruitment process and then select

participants responsively in relation to

evolving findings.

(table continues)

QUALITATIVE RESEARCH REPORTING STANDARDS

Table 1 (continued)

Paper section or element Description of information to be reported

Recommendations for authors to consider &

notes for reviewers

Data collection

Data collection/

identification

procedures

• State the form of data collected (e.g., interviews, questionnaires,

media, observation).

• Describe the origins or evolution of the data-collection protocol.

• Describe any alterations of data-collection strategy in response to

the evolving findings or the study rationale.

• Describe the data selection or collection process (e.g., were others

present when data were collected, number of times data were

collected, duration of collection, context).

• Convey the extensiveness of engagement (e.g., depth of

engagement, time intensiveness of data collection).

• For interview and written studies, indicate the mean and range of

the time duration in the data-collection process (e.g., interviews

were held for 75 to 110 min, with an average interview time of 90

min).

• Describe the management or use of reflexivity in the data-collection

process, as it illuminates the study.

• Describe questions asked in data collection: Content of central

questions, form of questions (e.g., open vs. closed).

• Reviewers: Researchers may use terms for data

collection that are coherent within their

research approach and process, such as data

identification, collection, or selection.

Descriptions should be provided, however, in

accessible terms in relation to the readership.

• Reviewers: It may not be useful for researchers

to reproduce all of the questions they asked in

an interview, especially in the case of

unstructured or semistructured interviews as

questions are adapted to the content of each

interview.

Recording and data

transformation

• Identify data audio/visual recording methods, field notes,

transcription processes used.

Analysis

Data-analytic strategies • Describe the methods and procedures used and for what

purpose/goal.

• Explicate in detail the process of analysis, including some

discussion of the procedures (e.g., coding, thematic analysis, etc.)

with a principle of transparency.

• Describe coders or analysts and their training, if not already

described in the researcher description section (e.g., coder selection,

collaboration groups).

• Identify whether coding categories emerged from the analyses or

were developed a priori.

• Identify units of analysis (e.g., entire transcript, unit, text) and how

units were formed, if applicable.

• Describe the process of arriving at an analytic scheme, if applicable

(e.g., if one was developed before or during the analysis or was

emergent throughout).

• Provide illustrations and descriptions of their development, if

relevant.

• Indicate software, if used.

• Reviewers: Researchers may use terms for data

analysis that are coherent within their research

approach and process (e.g., interpretation,

unitization, eidetic analysis, coding).

Descriptions should be provided, however, in

accessible terms in relation to the readership.

• Authors: Provide rationales to illuminate analytic

choices in relation to the study goals.

Methodological

integrity

• Demonstrate that the claims made from the analysis are warranted and

have produced findings with methodological integrity. The procedures

that support methodological integrity (i.e., fidelity and utility) typically

are described across the relevant sections of a paper, but they could be

addressed in a separate section when elaboration or emphasis would be

helpful. Issues of methodological integrity include:

• Reviewers: Research does not need to use all

or any of the checks (as rigor is centrally

based in the iterative process of qualitative

analyses, which inherently includes checks

within the evolving, self-correcting iterative

analyses), but their use can augment a study’s

methodological integrity. Approaches to

inquiry have different traditions in terms of

using checks and which checks are most

valued.

• Assess the adequacy of the data in terms of its ability to capture forms

of diversity most relevant to the question, research goals, and inquiry

approach.

• Describe how the researchers’ perspectives were managed in

both the data collection and analysis (e.g., to limit their effect on

the data collection, to structure the analysis).

• Demonstrate that findings are grounded in the evidence (e.g.,

using quotes, excerpts, or descriptions of researchers’

engagement in data collection).

• Demonstrate that the contributions are insightful and meaningful

(e.g., in relation to the current literature and the study goal).

• Provide relevant contextual information for findings (e.g., setting

of study, information about participant, interview question asked

is presented before excerpt as needed).

• Present findings in a coherent manner that makes sense of

contradictions or disconfirming evidence in the data (e.g.,

reconcile discrepancies, describe why a conflict might exist in

the findings).

LEVITT ET AL.

Table 1 (continued)

Paper section or element Description of information to be reported

Recommendations for authors to consider &

notes for reviewers

• Demonstrate consistency with regard to the analytic processes (e.g.,

analysts may use demonstrations of analyses to support

consistency, describe their development of a stable perspective,

interrater reliability, consensus) or describe responses to

inconsistencies, as relevant (e.g., coders switching mid-analysis, an

interruption in the analytic process). If alterations in

methodological integrity were made for ethical reasons, explicate

those reasons and the adjustments made.

• Describe how support for claims was supplemented by any checks

added to the qualitative analysis. Examples of supplemental checks

that can strengthen the research may include:

• Transcripts/data collected returned to participants for feedback.

• Triangulation across multiple sources of information, findings, or

investigators.

• Checks on the interview thoroughness or interviewer demands.

• Consensus or auditing process.

• Member checks or participant feedback on findings.

• Data displays/matrices.

• In-depth thick description, case examples, illustrations.

• Structured methods of researcher reflexivity (e.g., sending

memos, field notes, diary, log books, journals, bracketing).

• Checks on the utility of findings in responding to the study

problem (e.g., an evaluation of whether a solution worked).

Findings/Results

subsections

• Describe research findings (e.g., themes, categories, narratives) and

the meaning and understandings that the researcher has derived

from the data analysis.

• Demonstrate analytic process of reaching findings (e.g., quotes,

excerpts of data).

• Present research findings in a way that is compatible with the study

design.

• Present synthesizing illustrations (e.g., diagrams, tables, models), if

useful in organizing and conveying findings. Photographs or links

to videos can be used.

• Reviewers: Findings section tends to be longer

than in quantitative papers because of the

demonstrative rhetoric needed to permit the

evaluation of the analytic procedure.

• Reviewers: Depending on the approach to

inquiry, findings and discussion may be

combined or a personalized discursive style

might be used to portray the researchers’

involvement in the analysis.

• Reviewers: Findings may or may not include

quantified information, depending upon the

study’s goals, approach to inquiry, and study

characteristics.

• Authors: Findings presented in an artistic

manner (e.g., a link to a dramatic presentation

of findings) should also include information in

the reporting standards to support the research

presentation.

• Reviewers: Use quotes or excerpts to augment

data (e.g., thick, evocative description, field

notes, text excerpts), but these should not

replace the description of the findings of the

analysis.

Discussion

Discussion subsections • Describe the central contributions and their significance in

advancing disciplinary understandings.

• Describe the types of contributions made by findings (e.g.,

challenging, elaborating on, and supporting prior research or theory

in the literature describing the relevance) and how findings can be

best utilized.

• Identify similarities and differences from prior theories and research

findings.

• Reflect on any alternative explanations of the findings.

• Identify the study’s strengths and limitations (e.g., consider how the

quality, source, or types of the data or the analytic processes might

support or weaken its methodological integrity).

• Describe the limits of the scope of transferability (e.g., what should

readers bear in mind when using findings across contexts).

• Revisit any ethical dilemmas or challenges that were encountered,

and provide related suggestions for future researchers.

• Consider the implications for future research, policy, or practice.

• Reviewers: Accounts could lead to multiple

solutions rather than a single one. Many

qualitative approaches hold that there may be

more than one valid and useful set of findings

from a given data set.

QUALITATIVE RESEARCH REPORTING STANDARDS

Table 2

Qualitative Meta-Analysis Article Reporting Standards (QMARS): Information Recommended for Inclusion in Manuscripts That

Report Qualitative Meta-Analyses

Paper section or element Description of information to be reported

Recommendations for authors to consider & notes

for reviewers

Title page

Title • Indicate the key issues/topic under consideration.

• Indicate that the work is a form of meta-analysis (e.g., qualitative

metasynthesis, meta-ethnography critical interpretive synthesis, review).

Author note • Acknowledge funding sources or contributors acknowledged.

• Acknowledge conflicts of interest.

Abstract • State the problem/question/objectives under investigation.

• Indicate the study design, the types of literature reviewed, analytic

strategy, main results/findings, and main implications/significance.

• Identify five keywords.

• Authors: Consider using one keyword that

describes the meta-analytic strategy and one that

describes the problem addressed.

• Authors: Consider describing your approach to

inquiry when it will facilitate the review process

and intelligibility of your paper. If your work is

not grounded in a specific approach to inquiry, or

your approach would be too complicated to

explain in the allotted word count, however, it

would not be advisable to provide explication on

this point in the abstract.

Introduction

Description of research

problem/question

• State the problem/question the meta-analysis addresses.

• Describe what literature is to be included and synthesized and the

relevant debates, theoretical frameworks, and issues contained therein.

• Describe the importance or relevance of the meta-analysis to clarify

barriers, knowledge gaps, or practical needs.

Study objectives/research

goals

• Describe the meta-analytic method (e.g., metasynthesis, meta-analysis,

meta-ethnography, thematic synthesis, narrative synthesis, or critical

interpretive analysis).

• Identify the purpose/goals of the study.

• Describe the approach to inquiry, if it illuminates the objectives and

meta-research rationale (e.g., descriptive, interpretive, feminist,

psychoanalytic, postpositivist, constructivist, critical, postmodern or

constructivist, or pragmatic approaches).

• Describe the contribution to be made.

Method

Research design

overview

• Summarize the research design (data-collection strategies, data/meta-

analytic strategies, and, if illuminating, approaches to inquiry (e.g.,

descriptive, interpretive, feminist, psychoanalytic, postpositivist,

constructivist, critical, postmodern or constructivist, or pragmatic

approaches).

• Provide the rationale for the design selected.

• Reviewers: This section may be combined into the

same section as the objectives statement.

Study data sources

Researcher description • Describe the researchers’ backgrounds in approaching the study,

emphasizing their prior understandings of the phenomena under study

(e.g., interviewers, analysts or research team).

• Describe how prior understandings of the phenomena were managed and/

or influenced the research (e.g., enhancing, limiting, or structuring data

collection and meta-analysis).

• Authors: Prior understandings relevant to the

meta-analysis could include, but are not limited

to, descriptions of researchers’ demographic–

cultural characteristics, credentials, experience

with phenomenon, training, values, decisions in

selecting archives or material to analyze.

• Reviewers: Researchers differ in the extensiveness

of reflexive self-description in reports. It may not

be possible for authors to estimate the depth of

description desired by reviewers without

guidance.

Study selection • Provide a detailed description of how studies to be reviewed were

selected, including search strategies and criteria for inclusion and

exclusion, and rationale.

• Reviewers: Qualitative meta-analyses may seek to

review the literature comprehensively or may use

iterative or purposive sampling strategies (e.g.,

maximum variation sampling, theoretical

sampling, saturation seeking). In any case, the

strategy should be described as well as the

rationale for its use.

• Describe search parameters (e.g., thematic, population, and/or method).

• Identify the electronic databases searched, web searches, or other search

processes (e.g., calls for papers).

• Indicate the final number of studies reviewed and how it was reached.

LEVITT ET AL.

Table 2 (continued)

Paper section or element Description of information to be reported

Recommendations for authors to consider & notes

for reviewers

Studies reviewed Present, when possible the following: • Reviewers: This information might be best

presented in a tabular format, but should also be

summarized in the text.

• Year of publication of studies

• Disciplinary affiliation of primary author

• Geographic location of study

• Language of study

• Method of data collection (e.g., interview, focus group, online)

• Method of analysis of study (e.g., thematic analysis, narrative analysis,

grounded theory)

• Purpose of primary studies and differences (if any) from the main

questions of the meta-analysis

• Number of participants

• Recruitment method of study (snowball, convenience, purposive, etc.)

Analysis

Data-analytic

strategies

• Describe the approach to extracting study findings. This description may

include the following:

• Reviewers: Findings of qualitative primary studies

may be presented in disparate ways and

researchers should be transparent in making clear

how they identified and extracted findings from

primary reports.

• Reviewers: Typically, qualitative researchers do

not assign numerical weights to findings in

qualitative meta-analyses as the analyses are not

statistical in nature.

• Description of coders or analysts and training, if not already described

(interrater reliability, if used)

• Description of which parts of studies were assessed or appraised (e.g.

abstract, Discussion, Conclusions, full article)

• Description of units for coding (words, concepts, interpretations)

• Description of software, if used

• Description of team or collaborative discussions relevant to determining

what constitutes findings of studies, how inconsistencies among analysts

were managed, and how consensus was determined

• Discussion of whether coding categories emerged from the analyses or

were developed a priori

• Describe the process of arriving at an analytic scheme, if applicable (e.g.,

if one was developed before or during the analysis or was emergent

throughout).

• Describe how issues of consistency were addressed with regard to the

analytic processes (e.g., analysts may use demonstrations of analyses to

support consistency, describe their development of a stable perspective,

interrater reliability, consensus) or how inconsistencies were addressed.

• Describe the appraisal process in cases in which some studies were

considered to be more consequential in the interpretive process or others

discounted.

• Describe how illustrations or other artistic products (if any) were

developed from the analytic process.

Methodological

integrity

• See the JARS–Qual Standards.

• Meta-analyses should describe the integrity of their secondary analyses as

well as comment on the integrity of the primary studies under review.

Findings/Results

subsections

• Describe the research findings and the meaning and understandings that

the researcher has derived from the analysis of the studies.

• Reviewers: Results section tends to be longer than

in quantitative meta-analyses because of the

demonstrative rhetoric needed to permit the

evaluation of the meta-analytic method.

• Reviewers: Findings may or may not include the

quantified presentation of relevant codes,

depending on the study goals, approach to

inquiry, and study characteristics.

• Provide quotations from the primary studies to illustrate and ground the

themes or codes identified, when relevant.

• Explore whether differences in themes across primary studies appear to

reflect differences in the phenomena under study or differences in the

rhetoric or conceptual stances of the researchers.

• Present findings in a manner that is coherent within the study design and

goals (e.g., common themes, common interpretations, situated

differences).

• Consider the contexts of the meta-analytic findings as well as

contradictions and ambiguities among the reviewed studies so that

findings are presented in a coherent manner or discrepancies are

addressed.

• Present synthesizing illustrations (e.g., diagrams, tables, models) if

helpful in organizing and conveying findings.

Situatedness • Reflect on the situatedness of the studies reviewed (e.g., the positions and

contexts of the primary researchers and their studies).

• Reviewers: Situatedness can be considered in the

Results or Discussion section.

• Simplify the complexity of displaying trends in studies by using tables as

is helpful.

(table continues)

QUALITATIVE RESEARCH REPORTING STANDARDS

Information for Inclusion in Qualitative

Meta-Analysis Reporting Standards

(QMARS)

Qualitative meta-analysis is a form of inquiry in which

qualitative research findings about a process or experience

are aggregated or integrated. The aims can be to synthesize

qualitative findings across primary studies, to generate new

theoretical or conceptual models, to identify gaps in re-

search, as well as to generate new questions (e.g., Paterson,

Thorne, Canam, & Jillings, 2001; Sandelowski & Barroso,

2007). There are a variety of methods that engage these

aims, including qualitative metasynthesis, metaethnogra-

phy, metamethod, and critical interpretive synthesis. The

term qualitative meta-analysis does not indicate a singular

procedure but refers to the aggregating function common to

these approaches. Qualitative meta-analyses are not to be

confused with quantitative reviews that generate a narrative

description of a quantitative literature base. We recommend

referring to those studies as narrative reviews to avoid

confusion with qualitative meta-analyses.

The methodological integrity of the results of meta-

analysis studies rests largely on the extent to which those

carrying out the analysis can detail and defend the choices

they made of studies to review and the process they under-

took to weigh and integrate the findings of the studies.

Authors of meta-analysis reports often are aggregating qual-

itative studies from multiple methodological or theoretical

approaches, and they must communicate the approaches of

the studies they review as well as the approach to secondary

data analysis that is in use. Qualitative meta-analysis in-

volves the interpretive aggregation of thematic findings

rather than reanalysis of primary data. Forms of qualitative

meta-analysis range on a continuum from assessing the

ways in which findings do or do not replicate each other to

arranging interpreted findings into narrative accounts or

other analytic structures that relate the studies’ findings to

one another. Meta-analyses enhance their fidelity to the

findings by considering the contradictions and ambiguities

within and across studies. Qualitative meta-analysis entails

the amplification of primary findings and can permit a

broader perspective on the types of findings that ensue from

analytic processes. In Table 2 are the reporting standards for

qualitative meta-analyses. The column headings organize

information in the same manner as the JARS–Qual table

(see Table 1).

Mixed Methods Article Reporting

Standards (MMARS)

The reporting standards recommendations for the mod-

ule on mixed methods research are presented in Table 3.

Mixed methods research is a methodology that combines

qualitative and quantitative approaches. It should not be

confused with mixed-models research, which is a quan-

titative procedure. It involves (a) collecting and analyz-

ing both qualitative and quantitative data in response to

overarching research aims, questions, and hypotheses; (b)

using rigorous methods for both qualitative and quanti-

tative research; (c) integrating or “mixing” the two forms

of data intentionally to generate new insights; (d) framing

the methodology with distinct forms of research designs

or procedures; and (e) using philosophical assumptions or

theoretical models to inform the designs (Creswell,

2015). It originated approximately 30 years ago, and its

procedures have been steadily developing across disci-

plines through multiple articles, an estimated 30 books,

and several dedicated journals (e.g., Creswell & Plano

Clark, 2011; Hesse-Biber, 2010; Onwuegbuzie, 2012;

Table 2 (continued)

Paper section or element Description of information to be reported

Recommendations for authors to consider & notes

for reviewers

Discussion

Discussion subsections • Provide a discussion of findings that interpretively goes beyond a

summary of the existing studies.

• Reviewers: Rather than having only one possible

set of findings, meta-analyses could lead to

multiple insights and understandings of the

literature, that each have methodological integrity.

• Include reflections on alternative explanations in relation to findings, as

relevant.

• Discuss the contributions that the meta-analysis presents to the literature

(e.g., challenging, elaborating on, and supporting prior research or theory

in the literature).

• Draw links to existing scholarship or disputes in the literature that the

meta-analysis is designed to address.

• Describe the significance of the study and how findings can be best

utilized.

• Identify the strengths and limitations of the meta-study (e.g., consider

how the quality or source or types of the data or analytic process might

support or weaken its methodological integrity).

• Describe the limits of the scope of transferability (e.g., what readers

should bear in mind when using findings across contexts).

• Consider implications for future research, policy, or practice.

LEVITT ET AL.

Table 3

Mixed Methods Article Reporting Standards (MMARS): Information Recommended for Inclusion in Manuscripts That Report the

Collection and Integration of Qualitative and Quantitative Data

Paper section or

element Description of information to be reported Recommendations for authors to consider & notes for reviewers

Title page

Title • See the JARS–Qual and JARS–Quant Standards. • Authors: Refrain from using words that are either qualitative (e.g.,

explore, understand) or quantitative (e.g., determinants, correlates),

because mixed methods stands in the middle between qualitative

and quantitative research.

• Authors: Reference the terms mixed methods or qualitative and

quantitative.

Author note • See the JARS–Qual and JARS–Quant Standards.

Abstract • See the JARS–Qual and JARS–Quant Standards.

• Indicate the mixed methods design, including types of

participants or data sources, and analytic strategy, main

results/findings, and major implications/significance.

• Authors: Specify the type of mixed methods design used. See the

note on types of designs in the methods research design overview

section below.

• Authors: Consider using one keyword that describes the type of

mixed methods design and one that describes the problem

addressed.

• Authors: Describe your approach(es) to inquiry and, if relevant,

how intersecting approaches to inquiry are combined when this

description will facilitate the review process and intelligibility of

your paper. If your work is not grounded in a specific approach(es)

to inquiry or your approach would be too complicated to explain in

the allotted word count, however, it would not be advisable to

provide explication on this point in the abstract.

Introduction

Description of

research

problems/

questions

• See the JARS–Qual and JARS–Quant Standards. • Authors: This section may convey barriers in the literature that

suggest a need for both qualitative and quantitative data.

• Reviewers: Theory or conceptual framework use in mixed methods

varies depending on the specific mixed methods design or

procedures used. Theory may be used inductively or deductively

(or both) in mixed methods research.

Study objectives/

aims/research

goals

• See the JARS–Qual and JARS–Quant Standards.

• State three types of research objectives/aims/goals: qualitative,

quantitative, and mixed methods. Order these goals to reflect

the type of mixed methods design.

• Describe the ways approaches to inquiry were combined, as it

illuminates the objectives and mixed method rationale (e.g.,

descriptive, interpretive, feminist, psychoanalytic, postpositivist,

constructivist, critical, postmodern or constructivist, or

pragmatic approaches).

• Reviewers: A mixed method objective, aim, or goal may not be

familiar to reviewers. It describes the results to be obtained from

using the mixed methods design type where “mixing” or

integration occurs (e.g., the aim is to explain quantitative survey

results with qualitative interviews in an explanatory sequential

design). For instance, the goal of a qualitative phase could be the

development of a conceptual model, the goal of a quantitative

phase might be hypothesis testing based upon that model, and the

goal of the mixed methods could be to generate integrated support

for a theory based upon quantitative and qualitative evidence.

Method

Research design

overview

• See the JARS–Qual and JARS–Quant Standards.

• Explain why mixed methods research is appropriate as a

methodology given the paper’s goals.

• Identify the type of mixed methods design used and

define it.

• Indicate the qualitative approach to inquiry and the

quantitative approach used within the mixed methods

design type (e.g., ethnography, randomized experiment)

• If multiple approaches to inquiry were combined,

describe how this was done and provide a rationale (e.g.,

descriptive, interpretive, feminist, psychoanalytic,

postpositivist, constructivist, critical, postmodern or

constructivist, or pragmatic approaches), as it is

illuminating for the mixed method in use.

• Provide a rationale or justification for the need to collect

both qualitative and quantitative data and the added

value of integrating the results (findings) from the two

databases.

• Reviewers: Because mixed methods research is a relatively new

methodology, it is helpful to provide a definition of it from a

major reference in the field.

• Reviewers: Mixed methods research involves rigorous methods,

both qualitative and quantitative. Refer to the JARS–Qual

standards (qualitative) and JARS–Quant standards (quantitative) for

details of rigor.

• Reviewers: One of the most widely discussed topics in the mixed

methods literature would be research designs. There is not a

generic mixed methods design, but multiple types of designs. At

the heart of designs would be basic, core designs, such as a

convergent design, an explanatory sequential design, and an

exploratory sequential design. Although the names and types of

designs may differ among mixed methods writers, a common

understanding is that procedures for conducting a mixed methods

study may differ from one project to another. Further, these basic

procedures can be expanded by linking mixed methods to other

designs (e.g., intervention or experimental trial mixed methods

study), theories or standpoints (e.g., a feminist mixed methods

study), or to other methodologies (e.g., a participatory action

research mixed methods study).

(table continues)

QUALITATIVE RESEARCH REPORTING STANDARDS

Table 3 (continued)

Paper section or

element Description of information to be reported Recommendations for authors to consider & notes for reviewers

Participants or

other data

sources

• See the JARS–Qual and JARS–Quant Standards.

• When data are collected from multiple sources, clearly

identify the sources of qualitative and quantitative data

(e.g., participants, text), their characteristics, as well as

the relationship between the data sets if there is one

(e.g., an embedded design).

• State the data sources in the order of procedures used in

the design type (e.g., qualitative sources first in an

exploratory sequential design followed by quantitative

sources), if a sequenced design is used in the mixed

methods study.

• Authors: Because of multiple sources of data collected, separate

descriptions of samples are needed when they differ. A table of

qualitative sources and quantitative sources is helpful. This table

could include type of data, when it was collected, and from whom

it was collected. This table might also include study aims/research

questions for each data source and anticipated outcomes of the

study. In mixed methods research, this table is often called an

implementation matrix.

• Authors: Rather than describe data as represented in numbers

versus words, it is better to describe sources of data as open-ended

information (e.g., qualitative interviews) and closed-ended

information (e.g., quantitative instruments).

Researcher

description

• See the JARS–Qual Standards. • Reviewers: It is helpful to establish in a publication the researchers’

experiences (or research teams’ experiences) with both qualitative

and quantitative research as a prerequisite for conducting mixed

methods research.

• Authors: Because mixed methods research includes qualitative

research, and reflexivity is often included in qualitative research,

we would recommend statements as to how the researchers’

backgrounds influence the research.

Participant

recruitment

Participant

sampling or

selection

• See the JARS–Qual and JARS–Quant Standards.

• Describe the qualitative and the quantitative sampling in

separate sections.

• Relate the order of the sections to the procedures used in

the mixed methods design type.

Participant

recruitment

• See the JARS–Qual and JARS–Quant Standards.

• Discuss the recruitment strategy for qualitative and quantitative

research separately in mixed methods research.

Data collection

Data collection/

identification

procedures

• See the JARS–Qual and JARS–Quant Standards.

Recording and

transforming

the data

• See the JARS–Qual Standards.

Data analysis • See the JARS–Qual and JARS–Quant Standards.

• Devote separate sections to the qualitative data analysis,

the quantitative data analysis, and the mixed methods

analysis. This mixed methods analysis consists of ways

that the quantitative and qualitative results will be

“mixed” or integrated according to the type of mixed

methods design being used (e.g., merged in a convergent

design, connected in explanatory sequential designs and

in exploratory sequential designs).

Validity,

reliability, and

methodological

integrity

• See the JARS–Qual and JARS–Quant Standards.

• Indicate methodological integrity, quantitative validity

and reliability, and mixed methods validity or

legitimacy. Further assessments of mixed methods

integrity are also indicated to show the quality of the

research process and the inferences drawn from the

intersection of the quantitative and qualitative data.

LEVITT ET AL.

Small, 2011). The basic assumption of this methodology

is that the combined qualitative findings and quantitative

results lead to additional insights not gleaned from the

qualitative or quantitative findings alone (Creswell,

2015; Greene, 2007; Tashakkori & Teddlie, 2010). In

mixed methods, value accrues from both qualitative find-

ings and quantitative results, and the integration of the

two in a thoughtful way leads to greater mining of the

data and enhanced insights. In addition, authors can

publish multiple articles from a mixed methods study,

such as a qualitative article, a quantitative article, and a

mixed method overview article.

The thoughtful and robust use of mixed methods requires

meeting the standards of both quantitative and qualitative

research methodology in the design, implementation, and

reporting stages. To this end, various mixed methods de-

signs have emerged in the literature (Creswell & Plano

Clark, 2011), and they help inform the procedures used in

reporting studies (e.g., the convergent design, the explor-

atory sequential design, the explanatory sequential design).

Although some standards and recommendations exist by

authors writing in the health sciences (e.g., Creswell, Klas-

sen, Plano Clark, & Smith, 2011) and by journal editors

(e.g., the Journal of Mixed Methods Research; Fetters &

Freshwater, 2015), reporting standards for mixed methods

research have not been advanced to date in psychology or in

APA Style.

Table 3 conveys information about reporting standards

for mixed methods. The column headings organize infor-

mation in the same manner as the JARS–Qual table.

Typically, in mixed methods research, both JARS–Qual

and JARS–Quant standards must be met, with additional

MMARS standards also needing to be met. In the pre-

sentation of qualitative and quantitative components,

the sequence should represent the order that unfolded in

the study. When these components co-occurred, authors

may use their discretion in presenting the sequencing of

studies, but are encouraged to do so in a way that presents

a logical progression of narrative as well as an audit trail

(Merriam, 2014).

Recommendations and Future Considerations

A concern of the JARS–Qual Working Group is that

the use of qualitative methods in psychology is expand-

ing rapidly, and it is likely that new approaches to re-

search will continue to emerge. Indeed, we hope that

these standards are used to support the publication of

qualitative research and to increase the methodological

integrity of research published, but that they are not used

to limit the development of new qualitative methods. We

expect that qualitative reporting standards will continue

to shift and change in relation to growth of the field and

evolving writings on these issues (e.g., Gough & Deat-

rick, 2015; Wu, Thompson, Aroian, McQuaid, & Deat-

rick, 2016). We also hope that as the reporting standards

continue to develop, they do not contribute to the mar-

ginalization of minority epistemological perspectives and

designs but support methodological pluralism in our

field.

Also, we are not suggesting that every element that we

advance is relevant in every study. We do not support the

writing of empty statements that are not related to the

research being reported. For instance, some of our recom-

mendations make sense for research on human subjects but

not for textual or other analyses. Authors, reviewers, and

Table 3 (continued)

Paper section or

element Description of information to be reported Recommendations for authors to consider & notes for reviewers

Findings/Results

subsections

• See the JARS–Qual and JARS–Quant Standards.

• Indicate how the qualitative and quantitative results were

“mixed” or integrated (e.g., discussion, tables of joint

displays, graphs, data transformation in which one form

of data is transformed to the other, such as quantitative

text, codes, themes are transformed into counts or

variables).

• Authors: In mixed methods research, the findings section typically

includes sections on qualitative findings, quantitative results, and

mixed methods results. This section should mirror the type of

mixed methods design in terms of sequence (i.e., whether

quantitative strand or qualitative strand comes first; if both are

gathered at the same time, either qualitative findings or

quantitative results could be presented first).

• Reviewers: In mixed methods Results sections (or in the Discussion

section to follow), authors are conveying their mixed methods

analysis through “joint display” tables or graphs that array the

qualitative results next to the quantitative results (e.g., categorical

or continuous data). This enables researchers to directly compare

results or to see how results from the quantitative and qualitative

strands.

Discussion

subsections

• See the JARS–Qual and JARS–Quant Standards. • Authors: Typically, the Discussion section, like the Method and

Findings/Results, mirrors in sequence the procedures used in the

type of mixed methods design. It also reflects upon the

implications of the integrated findings from across the two

methods.

QUALITATIVE RESEARCH REPORTING STANDARDS

editors should use their judgment in making decisions about

which standards are relevant for the research manuscripts at

hand.

In sum, the establishment of these reporting standards

in APA Style heralds the acceptance of qualitative meth-

ods squarely within the canon of psychological ap-

proaches to inquiry and supports authors, reviewers, and

editors in their publication. These recommendations can

aid authors as they craft manuscripts for publication and

can assist reviewers and editors as well in the evaluation

process. We have articulated features of qualitative meth-

ods that are helpful to report in the written formulations

of a study to convey with clarity the research process. At

the same time, we recommend permitting flexibility in

reporting styles to preserve and respect qualitative tradi-

tions of inquiry. As such, these recommendations are

intended to help reviewers and editors consider the dis-

tinctive and essential features of qualitative designs in the

process of research evaluation. They should help readers

appreciate the value of the findings that are presented and

enhance the quality of work in this field moving forward.

References

American Psychological Association. (2010). Publication manual of the

American Psychological Association (6th ed.). Washington, DC: Author.

American Psychological Association Presidential Task Force on Immigra-

tion. (2012). Crossroads: The psychology of immigration in the new

century. Retrieved from http://www.apa.org/topics/immigration/report

.aspx

APA Publications and Communications Board Working Group on Journal

Article Reporting Standards. (2008). Reporting standards for research in

psychology: Why do we need them? What might they be? American

Psychologist, 63, 839–851. http://dx.doi.org/10.1037/0003-066X.63.9

.839

Bamberg, M. (2012). Narrative analysis. In H. Cooper, P. M. Camic, D. L.

Long, A. T. Panter, D. Rindskopf, & K. Sher (Eds.), APA handbook of

research methods in psychology (Vol. 2, pp. 85–102). Washington, DC:

American Psychological Association.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology.

Qualitative Research in Psychology, 3, 77–101. http://dx.doi.org/10

.1191/1478088706qp063oa

Chang, D. F., & Yoon, P. (2011). Ethnic minority clients’ perceptions of

the significance of race in cross-racial therapy relationships. Psychother-

apy Research, 21, 567–582. http://dx.doi.org/10.1080/10503307.2011

.592549

Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Thousand

Oaks, CA: Sage.

Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing

among five approaches. Thousand Oaks, CA: Sage.

Creswell, J. W. (2015). A concise introduction to mixed methods research.

Thousand Oaks, CA: Sage.

Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011).

Best practices for mixed methods research in the health sciences. Wash-

ington, DC: National Institutes of Health. http://dx.doi.org/10.1037/

e566732013-001

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting

mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Danziger, K. (1979). The positivist repudiation of Wundt. Journal of the

History of the Behavioral Sciences, 15, 205–230. http://dx.doi.org/10

.1002/1520-6696(197907)15:3⬍205::AID-JHBS2300150303⬎3.0

.CO;2-P

Danziger, K. (1990). Constructing the subject: Historical origins of psy-

chological research. Cambridge, UK: Cambridge University Press.

http://dx.doi.org/10.1017/CBO9780511524059

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and

practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.),

Handbook of qualitative research (pp. 191–215). Thousand Oaks, CA:

Sage.

Fetters, M. D., & Freshwater, D. (2015). Publishing a methodological

mixed methods research article. Journal of Mixed Methods Research, 9,

203–213. http://dx.doi.org/10.1177/1558689815594687

Fine, M. (2013). Echoes of Bedford: A 20-year social psychology memoir

on participatory action research hatched behind bars. American Psychol-

ogist, 68, 687–698. http://dx.doi.org/10.1037/a0034359

Finfgeld-Connett, D. (2014). Use of content analysis to conduct

knowledge-building and theory-generating qualitative systematic re-

views. Qualitative Research, 14, 341–352. http://dx.doi.org/10.1177/

1468794113481790

Fishman, D. B., & Messer, S. B. (2013). Pragmatic case studies as a source

of unity in applied psychology. Review of General Psychology, 17,

156–161. http://dx.doi.org/10.1037/a0032927

Frost, D. M. (2011). Stigma and intimacy in same-sex relationships: A

narrative approach. Journal Of Family Psychology, 25, 1–10. http://dx

.doi.org/10.1037/a0022374

Frost, D. M., & Ouellette, S. C. (2011). A search for meaning: Recognizing

the potential of narrative research in social policy-making efforts. Sex-

uality Research & Social Policy: A Journal of the NSRC, 8, 151–161.

http://dx.doi.org/10.1007/s13178-011-0061-2

Gergen, K. J. (2014). Pursuing excellence in qualitative inquiry.

Qualita-

tive

Psychology, 1, 49–60. http://dx.doi.org/10.1037/qup0000002

Gergen, K. J., Josselson, R., & Freeman, M. (2015). The promises of

qualitative inquiry. American Psychologist, 70, 1–9. http://dx.doi.org/10

.1037/a0038597

Gergen, M. M., & Gergen, K. J. (2012). Playing with purpose: Adventures

in performative social science. Walnut Creek, CA: Left Coast Press.

Gilgun, J. F. (2005). “Grab” and good science: Writing up the results of

qualitative research. Qualitative Health Research, 15, 256–262. http://

dx.doi.org/10.1177/1049732304268796

Giorgi, A. (2009). The descriptive phenomenological method in psychol-

ogy: A modified Husserlian approach. Pittsburgh, PA: Duquesne Uni-

versity Press.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory:

Strategies for qualitative research. Chicago, IL: Aldine.

Gough, B., & Deatrick, J. A. (2015). Qualitative health psychology re-

search: Diversity, power, and impact. Health Psychology, 34, 289–292.

http://dx.doi.org/10.1037/hea0000206

Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA:

Jossey-Bass.

Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contra-

dictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln

(Eds.), Handbook of qualitative research (pp. 191–215). Thousand

Oaks, CA: Sage.

Harding, S. (1992). Rethinking standpoint epistemology: What is “strong

objectivity?” The Centennial Review, 36, 437–470. Retrieved from

http://www.jstor.org/stable/23739232

Harré, R. (2004). Staking our claim for qualitative psychology as science.

Qualitative Research in Psychology, 1, 3–14.

Haverkamp, B. E. (2005). Ethical perspectives on qualitative research in

applied psychology. Journal of Counseling Psychology, 52, 146–155.

http://dx.doi.org/10.1037/0022-0167.52.2.146

Hernández, M. G., Nguyen, J., Casanova, S., Suárez-Orozco, C., & Saeter-

moe, C. L. (2013). Doing no harm and getting it right: Guidelines for

ethical research with immigrant communities. New Directions for Child

LEVITT ET AL.

and Adolescent Development, 2013, 43–60. http://dx.doi.org/10.1002/

cad.20042

Hesse-Biber, S. (2010). Mixed methods research: Merging theory with

practice. New York, NY: Guilford Press.

Hill, C. E. (2012). Consensual qualitative research: A practical resource

for investigating social science phenomena. Washington, DC: American

Psychological Association.

Josselson, R. (2007). The ethical attitude in narrative research: Principles

and practicalities. In J. Clandinin (Ed.), The handbook of narrative

inquiry (pp. 537–566). Thousand Oaks, CA: Sage. http://dx.doi.org/10

.4135/9781452226552.n21

Josselson, R. (2011). Narrative research: Constructing, deconstructing and

reconstructing story. In F. J. Wertz., K. Charmaz, L. M. McMullen, R.

Josselson, R. Anderson, & E. McSpadden (Eds.), Five ways of doing

qualitative analysis: Phenomenological psychology, grounded theory,

discourse analysis, narrative research, and intuitive inquiry (pp. 224–

243). New York, NY: Guilford Press.

Kuckartz, U. (2014). Qualitative text analysis: A guide to methods, prac-

tice and using software. Thousand Oaks, CA: Sage. http://dx.doi.org/10

.4135/9781446288719

Levitt, H. M. (2015). Qualitative psychotherapy research: The journey so

far and future directions. Psychotherapy, 52, 31–37. http://dx.doi.org/10

.1037/a0037076

Levitt, H. M. (2016). Recommendations for reviewing qualitative research.

Retrieved from http://www.apa.org/pubs/authors/review-manuscript-ce-

video.aspx

Levitt, H. M., Bamberg, M., Frost, D., & Josselson, R. (2016, August).

APA Publications and Communications Qualitative Article Reporting

Standards Task Force. In H. M. Levitt (Chair), Qualitative reporting

standards for research in psychology. Skill-building session conducted

at the 124th American Psychological Association Annual Convention,

Colorado, CA.

Levitt, H. M., Motulsky, S. L., Wertz, F. J., Morrow, S. L., & Ponterotto,

J. G. (2017). Recommendations for designing and reviewing qualitative

research in psychology: Promoting methodological integrity. Qualitative

Psychology, 4, 2–22. http://dx.doi.org/10.1037/qup0000082

Lieblich, A., Tuval-Mashiach, R., & Zilber, T. (1998). Narrative research:

Reading, analysis, and interpretation. Thousand Oaks, CA: Sage. http://

dx.doi.org/10.4135/9781412985253

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park,

CA: Sage.

Madill, A., & Gough, B. (2008). Qualitative research and its place in

psychological science. Psychological Methods, 13, 254–271. http://dx

.doi.org/10.1037/a0013220

McLeod, J. (2011). Qualitative research in counseling and psychotherapy.

Thousand Oaks, CA: Sage.

Merriam, S. B. (2014). Qualitative research: A guide to design and

implementation (3rd ed.). San Francisco, CA: Wiley.

Mertens, D. M. (2010). Mixed methods and the politics of human research:

The transformative-emancipatory perspective. In A. Tashakkori & C.

Teddlie (Eds.), Handbook of mixed methods in social and behavioral

research (2nd ed., pp. 135–164). Thousand Oaks, CA: Sage.

Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Method-

ological implications of combining qualitative and quantitative methods.

Journal of Mixed Methods Research, 1, 48–76. http://dx.doi.org/10

.1177/2345678906292462

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in

counseling psychology. Journal of Counseling Psychology, 52, 250–

260. http://dx.doi.org/10.1037/0022-0167.52.2.250

Neale, J., & West, R. (2015). Guidance for reporting qualitative research.

Addiction, 110, 549–550. http://dx.doi.org/10.1111/add.12857

Neimeyer, R. A., Hogan, N. S., & Laurie, A. (2008). The measurement of

grief: Psychometric considerations in the assessment of reactions to

bereavement. In M. S. Stroebe, R. O. Hansson, H. Schut, & W. Stroebe

(Eds.), Handbook of bereavement research and practice: Advances in

theory and intervention (pp. 133–161). Washington, DC: American

Psychological Association. http://dx.doi.org/10.1037/14498-007

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A.

(2014). Standards for reporting qualitative research: A synthesis of

recommendations. Academic Medicine, 89, 1245–1251. http://dx.doi

.org/10.1097/ACM.0000000000000388

Onwuegbuzie, A. J. (2012). Introduction: Putting the MIXED back into

quantitative

and qualitative research in educational research and beyond:

Moving toward the radical middle. International Journal of Multiple

Research Approaches, 6, 192–219. http://dx.doi.org/10.5172/mra.2012

.6.3.192

Osbeck, L. M. (2014). Scientific reasoning as sense making: Implications

for qualitative inquiry. Qualitative Psychology, 1, 34–46. http://dx.doi

.org/10.1037/qup0000004

Parker, I. (2004). Criteria for qualitative research in psychology. Qualita-

tive Research in Psychology, 1, 95–106. http://dx.doi.org/10.1191/

1478088704qp010oa

Parker, I. (2015). Psychology after discourse analysis: Concepts, methods,

critique. New York, NY: Routledge.

Paterson, B. L., Thorne, S., Canam, C., & Jillings, C. (2001). Meta-study

of qualitative health research: A practice guide to meta-analysis and

meta-synthesis. Thousand Oaks, CA: Sage. http://dx.doi.org/10.4135/

9781412985017

Pea, R. D. (1993). Learning scientific concepts through material and social

activities: Conversational analysis meets conceptual change. Educa-

tional Psychologist, 28, 265–277. http://dx.doi.org/10.1207/s1532

6985ep2803_6

Ponterotto, J. G. (2005a). Integrating qualitative research requirements into

professional psychology training programs in North America: Rationale

and curriculum model. Qualitative Research in Psychology, 2, 97–116.

http://dx.doi.org/10.1191/1478088705qp035oa

Ponterotto, J. G. (2005b). Qualitative research in counseling psychology: A

primer on research paradigms and philosophy of science. Journal of

Counseling Psychology, 52, 126–136. http://dx.doi.org/10.1037/0022-

0167.52.2.126

Ponterotto, J. G. (2005c). Qualitative research training in counseling psy-

chology: A survey of directors of training. Teaching of Psychology, 32,

60–62. Retrieved from http://top.sagepub.com.ezproxy.lib.umb.edu/

content/32/1/47.full.pdf⫹html

Potter, J., & Wetherell, M. (1987). Discourse and social psychology.

London, UK: Sage.

Rennie, D. L. (1995). On the rhetorics of social science: Let’s not conflate

natural science and human science. The Humanistic Psychologist, 23,

321–332. http://dx.doi.org/10.1080/08873267.1995.9986833

Rennie, D. L. (2012). Qualitative research as methodical hermeneutics.

Psychological Methods, 17, 385–398. http://dx.doi.org/10.1037/

a0029250

Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing quali-

tative research. New York, NY: Springer.

Sandelowski, M., & Leeman, J. (2012). Writing usable qualitative health

research findings. Qualitative Health Research, 22, 1404–1413. http://

dx.doi.org/10.1177/1049732312450368

Schultz, W. T. (2005). Handbook of psychobiography. New York, NY:

Oxford University Press.

Small, M. L. (2011). How to conduct a mixed methods study: Recent

trends in a rapidly growing literature. Annual Review of Sociology, 37,

57–86. http://dx.doi.org/10.1146/annurev.soc.012809.102657

Smith, J. A. (2004). Reflecting on the development of interpretive phe-

nomenological analysis and its contribution to qualitative research in

psychology. Qualitative Research in Psychology, 1, 39–54. http://doi.

dx.org/10.1191/1478088704qp004oa

QUALITATIVE RESEARCH REPORTING STANDARDS

Steinberg, S. R., & Cannella, G. S. (2012). Critical qualitative research

reader. New York, NY: Peter Lang.

Stiles, W. B. (1993). Quality control in qualitative research. Clinical

Psychology Review, 13, 593–618. http://dx.doi.org/10.1016/0272-

7358(93)90048-Q

Suzuki, L. A., Ahluwalia, M. K., Mattis, J. S., & Quizon, C. A. (2005).

Ethnography in counseling psychology research: Possibilities for appli-

cation. Journal of Counseling Psychology, 52, 206–214. http://dx.doi

.org/10.1037/0022-0167.52.2.206

Tashakkori, A., & Teddlie, C. (Eds.). (2010). Handbook of mixed methods

in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

http://dx.doi.org/10.4135/9781506335193

Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012).

Enhancing transparency in reporting the synthesis of qualitative re-

search: ENTREQ. BMC Medical Research Methodology, 12, 181–189.

http://dx.doi.org/10.1186/1471-2288-12-181

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for

reporting qualitative research (COREQ): A 32-item checklist for inter-

views and focus groups. International Journal of Qualitative Health

Care, 19, 349–357. http://dx.doi.org/10.1093/intqhc/mzm042

Walsh, R. G. (2015). Making discursive space in psychology for qualitative

report-writing. Qualitative Psychology, 2, 29–49. http://dx.doi.org/10

.1037/qup0000020

Wertz, F. J. (2010). The method of eidetic analysis for psychology. In T. F.

Cloonan & C. Thiboutot (Eds.), The redirection of psychology: Essays in

honor of Amedeo P. Giorgi (pp. 26l–278). Montréal, Québec, Canada:

L’Université duq Uébec a

Montréal et Rimouski, Le Cercle Interdisci-

plinaire de Recherches Phénoménologiques.

Wertz, F. J. (2014). Qualitative inquiry in the history of psychology.

Qualitative Psychology, 1, 4–16. http://dx.doi.org/10.1037/qup0000007

Wertz, F. J., Charmaz, K., McMullen, L. M., Josselson, R., Anderson, R.,

& McSpadden, E. (2011). Five ways of doing qualitative analysis:

Phenomenological psychology, grounded theory, discourse analysis,

narrative research, and intuitive inquiry (pp. 224–243). New York, NY:

Guilford Press.

Wisdom, J. P., Cavaleri, M. A., Onwuegbuzie, A. J., & Green, C. A.

(2012). Methodological reporting in qualitative, quantitative, and mixed

methods health services research articles. Health Services Research, 47,

721–745. http://dx.doi.org/10.1111/j.1475-6773.2011.01344.x

Wolcott, H. (2010). Ethnography lessons: A primer. Walnut Creek, CA:

Left Coast Press.

Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R.

(2013). RAMESES publication standards: Realist syntheses. BMC Med-

icine, 11, 21. http://dx.doi.org/10.1186/1741-7015-11-21

Wu, Y. P., Thompson, D., Aroian, K. J., McQuaid, E. L., & Deatrick, J. A.

(2016). Commentary: Writing and evaluating qualitative research re-

ports. Journal Of Pediatric Psychology, 41, 493–505. http://dx.doi.org/

10.1093/jpepsy/jsw032

Yin, R. K. (2013). Case study research: Design and methods. Thousand

Oaks, CA: Sage.

Received September 20, 2016

Revision received March 10, 2017

Accepted March 10, 2017 䡲

LEVITT ET AL.