10/10/2017 NCI Informed Consent Template – Optional Resource on Readability Tools Page 1
Optional Resource: Using Online and Manual Readability Tools to Assess the Reading
Level of Informed Consent Documents
Note: This resource is provided to assist Informed Consent Document authors in assessing the
readability of their documents. It is an optional resource.
Readability formulas are used to estimate the reading difficulty of text.
1,2
In general, they
measure the average number of syllables in words and the average number of words in sentences.
Most formulas provide results as grade levels, such as the 8
th
grade reading level. However,
because readability depends on so many issues, achieving a certain grade level is not a guarantee
of comprehension.
Types of readability formulas: There are numerous readability formulas including Flesch-
Kincaid, Flesch Reading Ease, SMOG, Fry, Fog Index, and Dale-Chall.
3
They are generally
accurate to ± 1.5 grade levels.
• Flesch Reading Ease – This formula uses a 100-point scale based on the average number
of syllables per word and the average number of words per sentence. The higher the
Flesch Reading Ease score, the easier it is to read the document. For example, a
document that scores at 60 is easier to read than a document that scores at 40.
• Flesch-Kincaid – This formula is a modified version of the Flesch Reading Ease
Formula. It assigns a grade level to a document. For example, you might see the results
listed as 8
th
grade or 12
th
grade depending on the complexity of the text. Like the Flesch
Reading Ease, the Flesch-Kincaid also measures the average number of syllables per
word and the average number of words per sentence.
• SMOG – This formula measures the number of polysyllabic words (more than 2
syllables in a word) contained in a sample of 30 sentences. Like the Flesch-Kincaid, it
assigns a grade level to a document based on its complexity.
Readability Level Guidelines for Informed Consent Documents: The 2015 IOM Informed
Consent and Health Literacy Workshop Summary recommended that informed consent
documents be written at the 8
th
grade reading level or lower.
4
Implementing Readability Assessments: Recommendations and Considerations: There are
many online sites that perform readability analyses on a given document and provide a reading
level score. The most commonly used tool is the Microsoft Word Readability Statistics function.
However, it is important to be aware of several limitations of this function which can result in
underestimating the grade level of your document (below). To get the most accurate results from
your online tool, you need to “clean up” your document.
5
This means that before conducting the
analysis, you need to:
1
Klare, G.R. (1963). The Measurement of Readability. Ames, IA: Iowa University Press.
2
Klare, G.R. (1984). Readability. In: Pearson D, Barr R, Kamil M, Mosenthal P, eds. Handbook of Reading Research. White Plains, NY: Longman;
681-744.
3
Klare, G.R. (1984).
4
Informed Consent and Health Literacy: Workshop Summary. IOM (Institute of Medicine). Washington, DC: The National Academies Press.
2015, https://www.nap.edu/read/19019/chapter/11#136 (2015, accessed October, 2017).
5
Toolkit for Making Written Material Clear and Effective. Centers for Medicare and Medicaid Services, US Department of Health and Human
Services. Washington, DC: 2010, https://www.cms.gov/Outreach-and-Education/Outreach/WrittenMaterialsToolkit/index.html
10/10/2017 NCI Informed Consent Template – Optional Resource on Readability Tools Page 2
• Delete titles, phrases, fragments, headers and lists that are not complete sentences. This
includes lists of side effects that are not written as complete sentences (although you can
include bulleted items that are written as complete sentences).
• Delete periods that don’t mark the end of a sentence, such as numerals in a number list
(1. or 2.); abbreviations (Celeste B. Jones, M.D.); or periods used in decimals (10.3).
• Delete phone numbers and URLs.
Readability formulas were designed for use on narrative, flowing text that consists of complete
sentences. They were not designed to measure phrases, fragments or lists.
6
If you include
phrases, fragments, headers and lists that are not complete sentences, your readability software
will not give you an accurate sentence count. If you don’t remove extra periods, your software
may count more sentences than there are, giving you a lower readability score.
7
It is normal to see some variability across the tools. However, you will ideally see similar results
from different tools. You can first put your document through MS Word’s Readability Function,
and then put it through the Online-Utility readability assessment tool at
https://www.online-
utility.org/english/readability_test_and_improve.jsp. Then, compare the results.
One good way to check the accuracy of your results is to conduct both an online and manual
readability analysis of your document. NCI’s Pink Book, “Making Health Communications
Programs Work, available at
https://www.cancer.gov/publications/health-communication/pink-
book.pdf, provides instructions on how to conduct a manual SMOG readability analysis (pages
162-166). You can then compare your results to the SMOG reading level you got from the
Online-Utility tool. MS Word Readability does not give a SMOG score.
Limitations of Readability Formulas:
• There is not a one-to-one correlation between the grade level of a specific document and
a person’s reading ability. For example, if you use the Flesch-Kincaid Readability
Formula to analyze your informed consent document and you get a score of 10
th
grade, it
does not mean that all adults reading at the 10
th
grade level will understand the text.
8
• Readability formulas do not measure many factors that affect reading ease, including the
familiarity of vocabulary and concepts, clarity of writing, concept density, format and
design, cultural relevance, believability, or the reader’s readiness to learn.
9,10
• Because most readability formulas give you averages, they do not tell you which sections
of text are hardest to read. You can select some of the potentially more difficult passages,
such as paragraphs with drug names, complicated medical procedures, and very long
sentences, when doing a manual SMOG analysis.
(2010, accessed October, 2017).
6
Doak, C, Doak L, Root J. (1996). Teaching Patients with Low Literacy Skills. Philadelphia, PA: J.P. Lippincott.
7
Toolkit for Making Written Material Clear and Effective. CMS. (2010).
8
Toolkit for Making Written Material Clear and Effective. CMS. (2010).
9
Doak, C, Doak L, Root J. (1996).
10
Toolkit for Making Written Material Clear and Effective. CMS. (2010).
