spec ial
vs.
ATOS Lexile Which Readability Formula Is Best?
rep ort
• ATOS for Books Readability Formula—a grade-level scale for measuring book readability levels. This formula is used to establish levels for books used with Renaissance Learning’s popular Accelerated Reader (AR) progress-monitoring and personalized readingpractice software, and is available free to publishers and others wanting an improved readability measure for books.
Introduction Readability formulas are helpful tools used by teachers and librarians to match students with appropriate books. Today’s educators have many different formulas from which to choose. This paper compares ATOS (Advantage-TASA Open Standard), a scientifically research-based readability formula developed by Renaissance Learning, and the Lexile Framework, a readability formula developed by MetaMetrics, Inc.
• ATOS for Text Readability Formula—also a gradelevel scale for use with other reading materials such as magazine articles, textbooks, and passages from reading tests. Educators may submit passages of text for free analysis through the Renaissance Learning website: www.renlearn.com/ar/overview/atos
Key Findings ATOS is a superior tool for helping educators match books to students because it is 1. Based on extensive research
• ATOS Readability Formula Conversion Scales— formulas that convert the ATOS grade-level scale to 100- and 2000-point scales similar to those used by TASA’s Degrees of Reading Power (DRP) test and the Lexile Framework, respectively.
2. More accurate than Lexile 3. An open standard—meaning it is free and easier to use ATOS is especially well suited to teachers working with nonfiction, emergent-reader, and high-interest/lowreadability level material.
Read on to learn why ATOS is the better tool for helping educators match books to students and build a lifelong love of reading.
Background on the ATOS Readability Formula
1
In 1998, Renaissance Learning, in conjunction with Touchstone Applied Science Associates (TASA), began developing a new, improved readability formula to give teachers and students a better tool for estimating text difficulty. The goals of the project were to improve the reliability and accuracy of reading estimates, and to develop a formula specifically designed for matching books to students.
ATOS Is Based on Extensive Research ATOS was developed using comprehensive tools and data, including:
• The largest set of reading test items ever used— more than 650 leveled items from a combination of Renaissance Learning’s STAR Reading (a normreferenced, computerized reading test) and TASA’s Degrees of Reading Power norm-referenced test.
To achieve these goals, it was necessary, among other things, for the formula to deal with evaluating fiction versus nonfiction, books for emergent readers, and highinterest/ low-readability level books for older readers, all of which have posed problems for readability formulas in the past. It was also Renaissance Learning’s goal to create an “open” formula—one that would be available to the educational community free of charge, in an easy-to-use format, and able to be used with any nationally normed reading test.
• The world’s largest word-frequency corpus—474 million words representing all the text of 28,000 K–12 popular trade books in the Renaissance Learning quiz library. Many of these books were published within five years of the ATOS project, making this corpus more relevant than those used in other readability research that date back several decades and include materials other than K–12 trade books. • An improved and expanded graded-vocabulary list— starting with TASA’s existing list showing the words most often found in books of different grade levels, and expanded based on study of the Renaissance Learning corpus.
The resulting project is believed to be the largest and most comprehensive study of readability ever conducted (Milone, 2009). It produced three different formulas, each of which is used for a different purpose:
2
• The National Reading Practice Database from Renaissance Learning—containing the AR records of more than 30,000 students who read and tested on more than 950,000 actual trade books. Thus, for the first time, substantial data on actual book reading was incorporated into the design of a readability formula.
(2004) found that the benefits of a high success rate on AR quizzes held regardless of the core reading curriculum used in the classroom. Despite the strength of the evidence supporting a high comprehension rate, some reading programs, including the Lexile Framework, target a lower level of comprehension. Specifically, Lexile targets 75 percent comprehension with its book-level recommendations—a benchmark that is not supported by research and, in fact, is described by the publisher as “an arbitrary, but useful, choice of 75% comprehension” (Stenner & Stone, 2004, p. 21).
In Accelerated Reader, level of success in reading practice is measured by percent correct on reading practice quizzes, which is in effect a measure of comprehension. Renaissance Learning recommends that students average 85 percent correct or higher on these quizzes. This recommendation has been validated by several large research studies. For instance, Topping and Sanders (2000) analyzed Tennessee Value-Added Assessment System data and found a positive effect on the teacher effectiveness index when students averaged 85 percent correct or higher.
2
ATOS Is More Accurate Than Lexile
ATOS improves readability scoring of several types of books where previous formulas have been less accurate and less helpful to teachers—such as nonfiction, emergent-reader, and high-interest/lowreadability level books. It also correlates well with other readability formulas.
More recently, researchers at Renaissance Learning validated the 85-percent-correct recommendation by analyzing the latest reading practice and achievement data from the new National Reading Practice Database—which included 50,000 students and 3 million books (Paul, 2003). This research showed that percent correct on AR reading practice quizzes is more important to student reading achievement than the amount of reading practice or the readability level of books. At all ability levels, students experience greater normal curve equivalent (NCE) reading gains as percent correct increases (see Table 1).
ATOS employs the three statistics researchers have found to be most predictive of reading difficulty: number of words per sentence, number of characters per word, and average grade level of the words in the book. The formula eliminates sampling error because all books are leveled based on a computerized, full-text scan of the book. That is, ATOS calculates the readability of the entire book, not just a few samples of text within it.
Another study used hierarchical linear modeling to replicate Paul’s 2003 findings. Borman and Dowling (2004) found that a high average percent correct on AR quizzes, over the course of the school year, was associated with better reading achievement at the end of the school year. In addition, Bolt
Unlike ATOS, the Lexile Framework is originally based on average word frequency found in the American Heritage Intermediate Corpus (Carroll, Davies, & Richman, 1971) and the average number of words per
Table 1. Students Experience Greater NCE Gains as Percent Correct Increases Grades 2–12; N=45,670 Student Achievement Level
Average Percent Correct Range
(Percentile Range)
Below 65%
65%–74%
75%–84%
85%–94%
95%–100%
0–20
–2.08
–0.74
1.57
5.01
3.44
21–40
–3.13
–1.18
0.85
4.72
6.35
41–60
–4.66
–1.06
0.23
5.29
6.77
61–80
–3.95
–2.78
–0.30
3.21
6.08
81–100
–5.72
–4.78
–2.18
2.11
4.93
3
sentence. Because average word frequency is not as accurate a predictor of semantic difficulty as either word length or word grade level, Lexile is likely to be less accurate than ATOS.
Teachers match books to students using Renaissance Learning’s Goal-Setting Chart. This ATOS companion provides suggested ranges of book difficulty for students’ grade-equivalent scores in reading. Because it is based on grade-equivalent score, any norm-referenced standardized test of reading achievement can be used. (For a free copy of this chart, visit http://doc.renlearn.com/KMNet/ R003792601GF8A5B.pdf.)
Compared to other readability formulas, ATOS works especially well with books for emergent readers. Other formulas often overstate the difficulty of emergent-reader books. For example, Five Little Kittens, by Nancy Geller Jewell, is a 32-page picture book. The sentences are slightly long and the word “kitten” is used repeatedly. The publisher labels this book as appropriate for grades K–3, ages 5–8. This book has a Flesch-Kincaid level of 4.6 and a Lexile level of 970, which equates to grades 6–7. This is the same range that includes books such as Dogsong by Gary Paulsen and Exploring the Titanic by Robert Ballard.
While the Lexile Framework also provides a chart showing recommended reading ranges, the Lexile ranges are much narrower than those on Renaissance Learning’s Goal-Setting Chart, especially for grades 5 and above. This is due to Lexile’s adoption of the arbitrary 75% correct rule rather than basing their recommendations on empirical evidence, as we do with ATOS. This will likely result in frustration and lack of motivation on the part of students as Lexile ranges do not provide the flexibility to read a wide range of books. Table 2 shows a comparison of Renaissance Learning ZPD ranges and Lexile ranges.
The ATOS level for Five Little Kittens is a much more reasonable 2.6. By measuring vocabulary level and adjusting for extreme sentence length, ATOS produces a more accurate readability level than either FleschKincaid or Lexile.
The reading ranges on the Goal-Setting Chart are based on the theoretical concept of zone of proximal development, or ZPD (Vygotsky, 1978). In independent, literature-based reading, ZPD is the range of books that will challenge a student without causing frustration or
Although MetaMetrics, Inc., claims Lexile to be the only system that puts reader and text on the same scale, Renaissance Learning has for years published tables that put reader and text on the same scale.
Table 2. Renaissance Learning’s ZPD Ranges Allow Students to Read a Wider Range of Literature Grade Equivalent
Renaissance Learning ZPD Range
Lexile Range Converted to ATOS Scale for Comparison
Low
High
Low
High
5.0
3.4
5.4
3.8
6.4
5.5
3.7
5.7
4.0
6.8
6.0
4.0
6.1
4.3
7.4
6.5
4.2
6.5
4.6
8.0
7.0
4.3
7.0
4.9
8.5
7.5
4.4
7.5
5.0
8.7
8.0
4.5
8.0
5.2
9.1
9.0
4.6
9.0
5.8
10.1
10.0
4.7
10.0
6.3
10.9
11.0
4.8
11.0
6.6
11.3
12.0
4.9
12.0
6.9
11.7
4
loss of motivation. The ranges become increasingly broad as students move up in grade equivalent. This recognizes that much good literature does not have a very high reading level. Nor does it contain leveled vocabulary, so even if books are easy for students to read/comprehend, they are still exposed to new vocabulary. In addition, once students become proficient readers, it is important they read a wide variety of books. The use of wide reading ranges ensures students have access to a large selection of good books that are interesting to them.
Because Lexile is a closed standard, teachers and librarians in districts adopting this formula may be forced to use only those tests and reading materials that report readability levels according to the Lexile formula. Obviously, this arrangement limits the choices in reading materials available to teachers and students. To also help educators and in keeping with the concept of open standards, Renaissance Learning has developed conversion charts that correlate ATOS for Books with a 100-point scale like the one used by DRP and a 2000point scale like the one used by Lexile. Table 3 shows sample ATOS levels and the corresponding 100-point and 2000-point scale values. For the full conversion table, see The Development of ATOS: The Renaissance Learning Readability Formula (Milone, 2009): http://research.renlearn.com/research/pdfs/334.pdf
The ZPD ranges on the Goal-Setting Chart were developed and updated over the years using actual student reading data (Renaissance Learning, 2003). The ranges were also validated through a sophisticated analysis of reading data from more than 50,000 students (Borman & Dowling, 2004). This research showed that students who read within their ZPD ranges experience greater gains in reading achievement than students who read below their ZPD ranges. Students who read above their ZPD can also experience gains, as long as they are reading with a high level of comprehension.
3
Table 3. ATOS Conversion Chart to 100and 2000-Point Scales
ATOS Is an Open Standard
The ATOS Readability Formula is an “open standard,” meaning that teachers can use gradeequivalent scores from any nationally normed reading test to determine each student’s reading level. It also means ATOS analysis is free of charge. (Lexile charges state departments of education and test publishers for the use of the Lexile scale.) With an open standard, teachers have the flexibility to use one readability formula for textbooks and another for literature-based reading. In addition, systems such as Renaissance Learning’s ZPD tables, which express book readability on a grade-level scale, allow teachers to use their students’ grade-equivalent scores from any test to match students with appropriate books. The ATOS formula’s ease of use is especially helpful in communications with students and parents. For example, in the ATOS system, a reading level of 4.5 translates simply to “fourth year, fifth month.” The Lexile Framework—a closed standard—uses a non-grade-level scale that is more difficult to compare and understand, and makes more work for teachers and parents. For instance, a student with a reading level of 4.5 might be assigned an enigmatic “700” in Lexile.
5
ATOS Levels
ATOS 100Point Scale Values
ATOS 2000Point Scale Values
0.3
21
23
0.5
27
31
0.7
30
39
1.0
34
51
1.3
37
120
1.5
38
157
1.7
40
190
2.0
41
241
2.3
43
301
2.5
44
348
2.7
44
381
3.0
46
440
3.3
47
491
3.5
47
530
3.7
48
561
4.0
49
619
4.3
50
669
4.5
50
701
4.7
51
731
Finally, while Lexile is proprietary, ATOS is available for educators to use on any text for which a readability level is needed.
Table 3. (continued)
ATOS Levels
ATOS 100Point Scale Values
ATOS 2000Point Scale Values
5.0
52
781
5.3
53
829
5.5
53
860
5.7
54
888
6.0
55
921
6.3
56
951
6.5
57
979
6.7
57
999
7.0
58
1029
7.3
60
1061
7.5
60
1080
7.7
61
1098
8.0
61
1121
8.3
62
1149
8.5
62
1161
8.7
62
1178
9.0
63
1201
9.3
63
1237
9.5
64
1258
9.7
64
1272
10.0
65
1293
10.3
65
1314
10.5
65
1328
10.7
66
1342
11.0
66
1364
11.3
66
1385
11.5
67
1399
11.7
67
1413
12.0
67
1434
12.3
68
1455
12.5
68
1469
12.7
68
1483
13.0
69
1504
Conclusion Readability formulas are important tools. They give teachers and librarians a place to start in their task of matching books to students, and we hope, creating a love of reading in all students. The ATOS for Books Readability Formula, designed especially to accurately level trade books, is a superior tool for accomplishing this task—especially for teachers working with nonfiction, emergent-reader, or high-interest/lowreadability material.
Learn More About ATOS Download a copy of The Development of ATOS: The Renaissance Learning Readability Formula from http://research.renlearn.com/research/pdfs/334.pdf, or contact Renaissance Learning for a print copy.
6
References Bolt, D. (2004). HLM analysis of effect of Reading Renaissance implementation on various reading curricula. Unpublished manuscript, University of Wisconsin–Madison. Borman, G. D., & Dowling, N. M. (2004). Testing the Reading Renaissance program theory: A multilevel analysis of student and classroom effects on reading achievement. Unpublished manuscript, University of Wisconsin–Madison. Available online from http://www.education.wisc.edu/elpa/people/faculty/Borman/BormanDowling2004_RdgRenProg.pdf Carroll, J. B., Davies, P., & Richman, B. (1971). The American heritage intermediate corpus. New York: American Heritage Publishing Company. Milone, M. (2009). The development of ATOS: The Renaissance readability formula. Wisconsin Rapids, WI: Renaissance Learning, Inc. Paul, T. D. (2003). Guided independent reading: An examination of the Reading Practice Database and the scientific research supporting guided independent reading as implemented in Reading Renaissance. Wisonsin Rapids, WI: Renaissance Learning, Inc. Available online from http://research.renlearn.com/research/pdfs/165.pdf Renaissance Learning. (2003). The research foundation for Accelerated Reader goal-setting practices. Wisconsin Rapids, WI: Author. Stenner, A. J., & Stone, M. H. (2004, May). Does the reader comprehend the text because the reader is able or because the text is easy? Paper presented at the International Reading Association, Reno-Tahoe, NV. Durham, NC: Metametrics. Available online from http://www.lexile.com/m/resources/materials/Stenner_Does_the_Reader_Comprehend_the_Text....pdf Topping, K. J., & Sanders, W. L. (2000). Teacher effectiveness and computer assessment of reading: Relating value-added and learning information systems data. School Effectiveness and School Improvement, 11(3), 305–337. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
7
For more information, or for additional copies of this report, contact: Educational Research Department PO Box 8036 • Wisconsin Rapids, WI 54495-8036 (800) 338-4204 • www.renlearn.com © 2006 Renaissance Learning, Inc. All rights reserved. AR, ATOS, Accelerated Reader, Advanced Technology for Data-Driven Schools, Renaissance, Renaissance Learning, and STAR Reading are trademarks of Renaissance Learning, Inc., and its subsidiaries, registered, common law, or pending registration in the United States and other countries. Degrees of Reading Power, DRP, and TASA are registered trademarks of Touchstone Applied Science Associates, Inc. The Lexile Framework is a registered trademark of MetaMetrics, Inc.
1
L2211.1209.NS.1M R35200