Lexile Scores: the Mismeasurement of Reading

kids-read

Recently my 14-year-old asked me, “What is the highest Lexile score you’ve ever heard of?”

I am not a fan of kids (or adults, for that matter) comparing themselves on the basis of test scores or numbers, so I deliberately gave her a nonsense answer: “Eleventy jillion.”

“No, what’s really the highest number?” she persisted.

“Why do you want to know?” I asked her.

“No reason, it’s just we’ve been finding out what our Lexile scores are. There’s this kid named Trevor* [not his real name] who got 1600.” She waited for me to be impressed.

“Mm-hm,” I said as noncommittally as I could.

“He and this girl named Amanda Chu* [not her real name] are widely known as being really smart, and they compete a lot, so when she got 1580 she was really sad.” My daughter mimed shrinking in on herself.

“So what’s the reason you are taking Lexile tests? What’s the purpose of them?”  I asked.

“I don’t really know,” she admitted.  “Maybe it’s for what we’re supposed to read?”

“And what would you teachers think about you telling each other your scores? Are you supposed to, or are they supposed to be private?”

“They wouldn’t care–nobody really cares!”  she said.

Except they do, apparently. The girl in her class (“widely known as being really smart”) who scored second felt really sad. How about the kids who scored toward the bottom of the class?

And does the Lexile score really tell teachers what they don’t already know: that this child has a large vocabulary, and another is challenged by reading? Is there a meaningful difference between the child who scored 1600 and the one who scored 1580? And why on earth would you let the students know their scores, knowing full well the irresistible urge to share and compare?  What is it all about?

metametrics

 

The Lexile Framework (as well as the Quantile Framework for math) was developed by the MetaMetrics Corporation.  Their slogan is “Linking Assessment With Instruction”:

“MetaMetrics® is focused on improving education for learners of all ages. We develop scientific measures of academic achievement and complementary technologies that link assessment results with real-world instruction.

“For more than twenty years, our work has been increasingly recognized worldwide for its distinct value in differentiating instruction and personalizing learning. In the U.S., our research on postsecondary reading demands informed the Common Core State Standards for college- and career-readiness.

“Our products and services for reading, mathematics and writing provide unique insights about academic ability and the potential for growth, enabling individuals to achieve their goals at every stage of development.”  [Emphasis mine.]

Lexile

What kind of “scientific measures” are these?  According to lexile.com, the Lexile Framework assesses both readers and texts.  (Note that the this measurement is made by using MetaMetrics’ “proprietary algorithm.”  That means they own this instrument of assessment, and get paid for its use.)  When a reader’s “measure” matches a text’s “measure,” “a targeted reading experience can occur.”  What is a “targeted reading experience”?  One in which “[t]he reader will encounter some level of difficulty with the text, but not enough to get frustrated.”

That’s a seemingly simple and maybe even laudable goal, but Lexile scores have been criticized for being both overly simplistic and wildly inaccurate.  For instance, author Mike Mullin writes about his encounter with a woman who was looking for a book for her daughter, who said her daughter was not allowed to read anything below a 1,000 Lexile measure by her school.  The woman seemed uninterested in the content of his apocalyptic novel, ignoring his warning it might be too violent for her 6th grade child, but rejected his book on the basis that it scored a 750, which would be “too easy” for her child.

However, when Mullin looked up Ernest Hemingway’s classic novel A Farewell to Arms at the Lexile book search page, it scored a scant 730.  As Mullin notes, “Is my work more difficult, more sophisticated, or more appropriate for older readers than that of Mr. Hemingway, a Nobel Laureate in literature? Of course not! Think about it: If this poor student stays in her school system, she’ll NEVER be allowed to read A Farewell to Arms. It’s allegedly too easy for her.”

Similarly, Blaine Greteman of The New Republic notes that Slaughterhouse Five by Kurt Vonnegut scores a Lexile measure of 870, while children’s book classic Mr. Popper’s Penguins by Richard and Florence Atwater is “deemed more complex” with a Lexile score of 910.  According to the article, Mr. Popper’s Penguins also outscores To Kill a Mockingbird (790) and The Grapes of Wrath (680).

PopperCover

These writers (and many others) are making the point that vocabulary and grammatical complexity are far from the most accurate or appropriate measures of reading material.  Indeed, according to the Common Core Standards (which were based in part on MetaMetrics research), “text complexity consists of three equally important parts” (emphasis mine):

  • “qualitative dimensions” that are “best measured or only measurable by an attentive human reader, such as levels of meaning or purpose; structure; language conventionality and clarity; and knowledge demands”;
  • “quantitative dimensions…such as word length or frequency, sentence length, and text cohesion, that are difficult if not impossible for a human reader to evaluate efficiently, especially in long texts, and are thus today typically measured by computer software; and
  • “Reader and task considerations… [that are] variables specific to particular readers (such as motivation, knowledge, and experiences) and to particular tasks (such as purpose and the complexity of the task assigned and the questions posed) [which] must also be considered when determining whether a text is appropriate for a given student. Such assessments are best made by teachers employing their professional judgment, experience, and knowledge of their students and the subject.” [All emphases mine.]
text_complexity

Common Core Standards Three-Part Model for Measuring Text Complexity

Content, context, background knowledge, and ideas are at least as important as the “quantitative dimensions” that are the only things that the Lexile Framework measures, but the Lexile Framework ignores those, as well as “human readers” and “teachers implying their professional judgement,” and touts itself as “the GOLD STANDARD for college and career readiness.”

Stephen Krashen of the California School Library Association calls the Lexile Framework “unnecessary and potentially harmful,” noting that it “attempts to solve a problem that doesn’t exist.”  Krashen says that there is a “much easier way for readers to select texts: Are they comprehensible and interesting? It doesn’t take long for a reader to determine this: All it takes is sampling a little of the text (reading it).”  Teachers and librarians can also help the reader to find interesting and appropriate reading material.  Furthermore, the Lexile Framework’s “restriction of reading… will needlessly limit readers’ choices, keeping readers in a narrow range of texts.”

Krashen further calls the Lexile Framework “a waste of money”:

“The research cost of the Lexile Framework was approximately two million dollars and the research was supported by the National Institute of Child Health and Human Development (Metametrics, no date). This money would have been much better invested in our school and classroom libraries. While prices are not mentioned in the literature I have seen, the full kit comes with the Lexile Framework Map, “with examples of books, magazines, tests, and educational levels,” software (the Lexile Analyzer), and “an item bank for measuring reading performance, conversion formulas for commonly used reading texts, and a technology for linking existing reading tests to the Lexile Framework.” In addition to the cost of this material, one must also consider the time invested in making sure all texts have a lexile rating and making sure that we know at every moment each student’s lexile rating!”

Books in the real world outside schools are not grouped by Lexile scores, (as much as MetaMetrics and its Lexile enthusiasts might wish it were so).   This does not stop the occasional teacher from sending students into school libraries looking for books by their Lexile score; or parents (as in Mullin’s example) from asking public librarians where a particular Lexile score section is.  It has to be explained to some people that libraries do not group books by these measures.

As Mullin notes, “What should you do? If you’re a school administrator, teacher, or librarian, quit using Lexiles….If you’re a parent, let your child pick books the way you do–based on interest and need. Ask your school to dump the Lexile system. The last thing we need is an expensive program that makes the great work parents, teachers, and librarians do–educating our children–more difficult.”

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Lexile Scores: the Mismeasurement of Reading

  1. Annie R says:

    Thank you for sharing your post, I found it incredibly interesting! I was only vaguely aware of the Lexile scoring system before reading your post, which I became increasingly thankful for the more I read. The Lexile system seems to be a more frustrating and illogical version of AR books, which I did not think was possible. It baffles me to hear that schools will not allow students to read books based on a somewhat arbitrary (in my opinion) ranking system. While I do understand the need to make sure students are improving their grammar and understanding sentence structure, it frustrates me when restrictive quantitative measurements are placed on books. I had an amazing experience with books growing up. My local library became my second home for most of middle and high school. I explored the shelves and read anything that looked remotely interesting. I read some amazing books, but also some books that were not as good. But because of my experience reading many different books, I grew as a reader and was able to determine the difference between good writing and great writing. Mike Mullin wrote that the Lexile system associates books with longer sentences and longer words with the designations of complexity and obscurity, yet “the best writers never use two words where one will do, and they choose their words with precision.” It is incredibly concerning to read this because this means “students forced to use the Lexile system in their reading are being taught to be bad writers.” We have discussed in our class about when we learned to read. I honestly do not remember learning to read in school, but nor do I remember learning to write. I was always considered a good writer, from elementary school to high school, yet I do not know most of the technical terms and grammatically correct ways to structure sentences. I just write what sounds right, based on the many books I have read. I worry that with restrictive reading programs, such as the Lexile system, students are losing their passion for reading and, in turn, not growing as writers.

    Like

    • juniperonjupiter says:

      Thanks for your comment, Annie! I wish more kids had the rewarding experience with books and reading you had when you were growing up. Reading measurement systems like AR and Lexile are frustrating and illogical, as you say, because they mistake a piece of reading for the whole. It is like judging a painting on how many colors are used, or how many brushstrokes per square inch.

      This is not to say that studying those things is not a valid tool of art analysis and criticism, but to focus only on those aspects is missing the point spectacularly. Similarly, studies of vocabulary and sentence complexity are troubling in their reductiveness and their claim to be conclusive measures of writing and reading.

      Another thing you mentioned was that these systems tend to teach students that longer sentences and longer words are “better.” That is something the old essay section of the SATs was famous for. The more words the test-taker could cram in, the higher the score. This is a system that rewarded long-windedness rather than depth of thought or clarity of writing. Take a look at this 2005 NY Times article, “SAT Essay Test Rewards Length and Ignores Errors” (http://www.nytimes.com/2005/05/04/education/sat-essay-test-rewards-length-and-ignores-errors.html?_r=0). In it, Les Perelman, a director of undergraduate writing at MIT, studied graded SAT essays that were made public:

      “He was stunned by how complete the correlation was between length and score. ‘I have never found a quantifiable predictor in 25 years of grading that was anywhere near as strong as this one,’ he said. ‘If you just graded them based on length without ever reading them, you’d be right over 90 percent of the time.’ The shortest essays, typically 100 words, got the lowest grade of one. The longest, about 400 words, got the top grade of six. In between, there was virtually a direct match between length and grade.

      “How to prepare for such an essay? ‘I would advise writing as long as possible,’ said Dr. Perelman, ‘and include lots of facts, even if they’re made up.’ [Incorrect facts were not penalized in the essay portion of the test.] This, of course, is not what he teaches his M.I.T. students. ‘It’s exactly what we don’t want to teach our kids,’ he said.”

      It remains to be seen whether the new, “optional” SAT essay is an improvement, but I am not holding my breath. Thoughtful and meaningful analysis of writing is much harder than MetaMetrics/Lexile and their ilk admit.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s