A Consumer Reports-style review of math instructional materials that called out nearly all the curricula evaluated for failing to align to the common-core standards is now coming under attack for its methodology.
The nonprofit EdReports.org posted online its first round of reviews, which focused on K-8 mathematics materials from widely used publishers such as Pearson, McGraw-Hill, and Houghton Mifflin Harcourt, at the beginning of the month. The group found that, contrary to their publishers’ claims, 17 of 20 math series evaluated failed to meet criteria for alignment to the Common Core State Standards.
Publishers and at least one outside math expert, however, argue that the teacher-led reviews were tainted by shoddy methodology and are misleading. One publisher of a primarily digital curriculum that was found unaligned even claims that some reviewers spent little or no time logged into its materials, making the review “inadequate and inconsistent with rigorous standards.”
If widely accepted, EdReports’ reviews could have profound effects for adoption decisions, curriculum experts said.
“In general, the results are pretty bad for all the publishers,” said Morgan Polikoff, an assistant professor of education at the University of Southern California, in Los Angeles, who was not part of the project. “I think people really will pay attention to this.” The outcomes echo previous alignment studies conducted by Mr. Polikoff and others.
But publishers have since sharply criticized the EdReports.org methodology, especially the use of “gateways,” or thresholds, that curricula needed to pass through to continue in the review process.
Jay A. Diskey, the executive director of the Association of American Publishers’ P-12 learning group, based in Washington, said the use of such gateways resulted in “a very shallow, incomplete review.”
Diane J. Briars, the president of the National Council of Teachers of Mathematics, also has “very serious reservations” about the process. “I think they ended up with a lot of false negatives,” she said.
Just one curriculum series stood out from the pack. Eureka Math, published by Great Minds, a small Washington-based nonprofit organization, was found to be aligned to the common core for all grades, K-8.
EdReports.org, spearheaded by Maria M. Klawe, the president of Harvey Mudd College, in Claremont, Calif., was launched in August. The project is underwritten primarily by $3 million in grants from the Seattle-based Bill & Melinda Gates Foundation, which also was a major backer of the development of the common core, the William and Flora Hewlett Foundation of Menlo Park, Calif., and the Leona M. and Harry B. Helmsley Charitable Trust in New York City. (The Gates and Hewlett foundations also support, respectively, coverage of college- and career-ready standards and deeper learning in Education Week.)
The 46 reviewers, half of whom are practicing teachers, worked in teams of four over several months to review the K-5 or 6-8 instructional series. Team members combed through the curricula independently—up to a dozen textbooks apiece—and then met in a weekly videoconference to discuss their findings.
The curricula were first evaluated on whether they met the common core’s expectations for focus and coherence—that is, whether they stuck to grade-level content and followed a logical sequence for math learning. If a text passed that first gateway—and a majority did not—the reviewers then moved along to gateway two, which looked at whether the curriculum met the expectations for rigor. The third and final gateway measured usability.
Reviewers and EdReports.org representatives said in October that team members were spending three or four hours a week looking over the texts on their own.
But Philip Uri Treisman, the founder and executive director of the University of Texas at Austin’s Charles A. Dana Center, which helped develop the Agile Mind curricula, said that his group tracked the amount of time reviewers spent logged into Agile Mind’s digital materials. It found that two of the four reviewers “logged into the digital materials for little or no time at all, depending on the grade level of the program,” and that “less than 23 percent of the total content were reviewed by more than one panelist for an hour or more,” he wrote in an email. Some units were reviewed for less than 10 minutes, he said.
Eric Hirsch, the executive director of EdReports.org, responded that not all the logins worked, and that teams came together and completed their individual reviews from a projector screen. “It may have only taken 10 minutes for reviewers to look or assess whether a unit or lesson were to cover the major work of the grade sufficiently,” said Mr. Hirsch. “We stand by the work our educators did on the review.”
The EdReports.org website uses a three-tiered rating system—"meets criteria,” “partially meets criteria,” or “does not meet criteria"—for each gateway. The site also has more-detailed reports for each textbook, which include documentation on how the reviewers reached each score.
Among the highlights:
• My Math, a K-5 instructional series by McGraw-Hill, was deemed fully aligned for grades 4 and 5.
• Of the seven Houghton Mifflin Harcourt instructional series reviewed, four partially met alignment criteria for at least one grade level.
• One of the four Pearson texts reviewed was judged to be partially aligned for at least one grade level.
• All texts by Agile Mind, Big Ideas Learning, Edgenuity, Kendall Hunt, and TPS Publishing Inc. were considered not aligned to the common standards.
More than 40 states have adopted the common core, the set of English/language arts and math standards released in 2010 by the Council of Chief State School Officers and the National Governors Association.
Publishers had two weeks to write responses of up to 1,500 words, which were posted with the reviews on the EdReports.org website.
Pearson wrote that the reviewers “applied a very narrow standard for measuring focus.” Big Ideas said that the gateway process “discredits” the evaluation, because it allowed reviewers to come to “the generalized conclusion that Big Ideas Math fails two other gateways, rigor and usability, without even a perfunctory analysis” of those characteristics.
In an email, John Hartz, a spokesman for the Scottsdale, Ariz.-based Edgenuity, said that the gateway process “will mislead readers.”
Ms. Briars, the NCTM president, agreed with many of those critiques. Gateway one, she said, had a “fatal flaw” in that, in order to pass through it, instructional material could not assess content from future grades. “If that gets a zero, the whole program is thrown out,” she said. “But the assessment tasks are separate from what the kids see every day. And I might want to assess informally what students know and how in-depth their knowledge is.”
Several publishers noted in emails and interviews that their curricula had been adopted by state and district review teams.
“It has to be asked whether some of the differences here have to do with the type of rubric and alignment tool that was used” by EdReports.org, said Mr. Diskey of the AAP.
But USC’s Mr. Polikoff said EdReports.org’s review process made sense. “There can always be methodological quibbles,” he said. “It would be useful to look at all three gateways for all the books, but this seems to me a perfectly reasonable way to constrain the task.”
Mr. Hirsch said the gateway process came out of discussions with educators, who consistently concluded that “if you’re not teaching the right math, these other things won’t matter as much.” Even so, he said, “we view every publisher response as an opportunity for learning.”
Alignment studies conducted by researchers William Schmidt, the co-director of the education policy center at Michigan State University, in East Lansing, and Mr. Polikoff came to the same conclusion as EdReports.org: Claims of common-core alignment are generally unfounded.
For Denise Walston, the director of mathematics for the Washington-based Council of the Great City Schools, the review has the potential to improve what’s going on in classrooms. “Districts considering buying, I think they need to pause and wait a little bit,” she said. “But for those that have bought materials and have some programs that only partially met expectations from EdReports, now’s the time to look at what’s there and try to figure out ways they can supplement. ... The textbook is what [teachers] depend on, so you’ve got to make sure what you put in their hands is right.”
EdReports.org plans to eventually move on to secondary math and K-12 English/language arts curricula and to institute a rolling review process for K-8 math materials, said Mr. Hirsch.
A version of this article appeared in the March 18, 2015 edition of Education Week as Backlash Brews Over Critical Review of Math Materials
Eureka Math is a Common Core math. Eureka Math's framework is entirely built on the Common Core Learning Standards and Progressions for the Common Core State Standards in Mathematics.What is the best way to review math? ›
- Make a study schedule.
- Maintain a mathematics notebook.
- Read your textbook prior to class.
- Do textbook examples.
- Write the mathematical procedures.
- Re-visit previously-studied concepts.
- Summarize concepts and procedures.
- Re-read prior to a quiz or test.
Eureka Math is a Common Core math. Eureka Math's framework is entirely built on the Common Core Learning Standards and Progressions for the Common Core State Standards in Mathematics.Is Eureka math evidence based? ›
Our Eureka Math² teacher–writers and experts used a research-based approach to craft a new curriculum that ensures students build enduring knowledge.Why is Eureka math so hard? ›
Eureka focuses on *how* you get to the answer, not just getting there. The tricky part is a lot of this math is visual, because many kids learn better that way. For parents though, it's weird to see so many drawings for a simple math problem. Some parents may say it's complicated.What is a mathematical review? ›
Mathematical Reviews is a journal published by the American Mathematical Society (AMS) that contains brief synopses, and in some cases evaluations, of many articles in mathematics, statistics, and theoretical computer science.Why is math review important? ›
Cumulative review helps students practice going back to everything that they have learned and recall an old skill that they have previously acquired to solve the problem, or as Rohrer (2009) coins it, “retrieval practice.” Review also allows students to determine what they remember and what they do not remember and ...Which math curriculum is not Common Core? ›
Horizons Math is not aligned with Common Core Math Standards and often goes above the expected level for the child's grade.Is Eureka Math developmentally appropriate? ›
Eureka Math is a math curriculum developed by a team of math master teachers and mathematicians to ensure that the quality of the curriculum builds upon students' prior-knowledge and teaches mathematical concepts and skills in a coherent and developmentally-appropriate way.What states use Eureka math? ›
California districts using the curriculum include Vista Unified School District, Montebello Unified School District, Riverside Unified School District, Palmdale Unified School District, Pomona Unified School District, Alameda Unified School District, Lincoln Unified School District, Mountain View Whisman School ...
Eureka Math exhibits unparalleled rigor throughout the grades. Students develop conceptual understanding and practice procedural skills and fluency. They also have opportunities to connect their learning with real-life application problems.What grade levels is Eureka math? ›
Eureka Math® is a holistic Prekindergarten through Grade 12 curriculum that carefully sequences mathematical progressions in expertly crafted modules, making math a joy to teach and learn.Is Eureka math standards aligned? ›
Eureka Math is the only curriculum found by EdReports.org to align fully with the Common Core State Standards for Mathematics for all grades, Kindergarten through Grade 8. Great Minds offers detailed analyses which demonstrate how each grade of Eureka Math aligns with specific state standards.What do teachers think about Eureka math? ›
Positive Eureka Math reviews tend to focus on the fact that the curriculum is 100% aligned to State Standards, provides easy to teach lesson plans that reduce time in test prep, and improves testing scores.Is Eureka Math successful? ›
EdReports gave Eureka Math2 top marks across all evaluated categories: focus and coherence, rigor and mathematical practices, and usability in the classroom. Their review focuses on whether materials meet rigorous indicators and criteria in these three distinct gateways.How much does Eureka math cost? ›
Is Eureka Math free? Yes. Anyone can download the entire PK–12 Eureka Math curriculum, along with a variety of instructional materials and support resources, for free.What is the best time to review math? ›
Various studies show that the best time to study math is at the beginning of the day, either in the first or second class period. In fact, students in early morning math classes tend to perform better than those in afternoon classes.