Ulrich Boser | Vox

Sorry, kids: schools need more testing, not less

2017-05-31T09:10:04-04:00

“Testing for testing’s sake”? Or a powerful educational tool? | John Nordell/Christian Science Monitor/Getty

In education circles, testing has become the villain of the day. Kids declare exams to be a waste of time while educators argue that the anxiety around tests produces a “toxic environment.” Families loathe exams, too, as I learned when doing some research on assessments, with parents often viewing tests as either a distraction from more important activities or as “testing for testing’s sake.”

But when it comes to learning, it turns out, the best research shows that exams help learning rather than harm it, and most schools and universities actually should be doing more testing, not less of it. A large and growing body of studies indicates that assessments help students learn. More — and better — testing programs can also help teachers teach.

Cutting against the grain of all the negative chatter about tests, some cognitive psychologists, including Yana Weinstein at University of Massachusetts Lowell, have declared themselves to be “champions” of testing.

The catch is that the tests have to be the right kinds of tests. The exams that spark learning typically tend to have lower stakes, they have more open-ended questions, and they are given often enough to provide clear feedback to teachers and students. But if they’re well designed, tests have been proven to help students achieve mastery of the subject material — not just evaluate progress.

To be sure, there are good reasons that tests have a tarnished brand. For years, assessment programs have been weak. Many schools and universities have relied heavily on multiple-choice exams, which often do little to measure (or instill) richer forms of understanding. Up until a few years ago, a few states, including North Carolina, gave English exams that didn’t include any writing tasks, which is a bit like administering a driving test without the student ever driving a car.

For many years, testing programs also didn’t provide enough feedback. For instance, final exams often help to determine a student’s grade. But that kind of assessment happens too infrequently — and, almost by definition, too late — to provide useful information. The high-stakes nature of such tests is also all but guaranteed to spark unproductive anxiety.

The exams that some students have to pass to get a high school diploma are the epitome of a high-stakes test. Not surprisingly, these exams can cause cheating and stress. Many of these formal, high-stakes exams also take weeks — if not months — to grade. While the exams might make a high school diploma appear more meaningful to colleges and employers, the results typically come far too late to shape what happens in specific classrooms in specific years.

Quizzes provide instant feedback, allowing teachers to fine-tune their lessons

This all stands in stark contrast to the kinds of tests most firmly backed by the evidence. These research-supported exams, of various kinds, do a few different things. First, they can provide frequent checks on student understanding. By using them, educators can fine-tune their lessons to match the learning pace of the students in a classroom.

I saw an instructor take this sort of quiz-based approach to teaching in a large physics class at the University of Maryland College Park. It was the first day after spring break, and the professor, Ben Dreyfus, gave a little quiz about electrical charges. The students responded using hand-held “clickers.” Many students had forgotten the material Dreyfus asked them about, and on a few questions, half the class gave the wrong response.

So Dreyfus spent the rest of the class time reviewing the material, to ensure that they’d absorbed it before moving on to new material that built on that knowledge.

A recent academic article on the use of clicker-based quizzes in large lectures drives home just how useful such temperature-taking can be. One biology instructor quoted in it recalled being amazed that while 90 percent of students recalled a certain rule in genetics, only 48 percent could apply it accurately:

This was a moment of revelation. … for the first time in over 20 years of lecturing I knew… that over half the class didn’t ‘get it’…. Because I had already explained the phenomenon as clearly as I could, I simply asked the students to debate briefly with their neighbors and see who could convince whom about which answer was correct. The class erupted into animated conversation. After a few minutes, I asked for a revote, and now over 90 percent gave the correct answer.

How tests teach: the “retrieval practice effect”

Even more exciting, if counterintuitive, low-stakes quizzes and exams do not just test mastery, they help create it. When we’re quizzed, the act of attempting to express what we know about a subject helps to solidify our understanding of the subject at hand. Psychologists call this mechanism the “retrieval practice effect,” or, more simply, the “testing effect.” (Students can use a version of this technique by quizzing themselves when studying for bigger tests.)

Researchers Jeffrey Karpicke and Janell Blunt, of Purdue University, have shown that quizzing promotes conceptual knowledge more than some other approaches once thought to be promising — like concept mapping, which involves summarizing what you know by graphical means, on paper or a whiteboard, and connecting related ideas with lines and arrows. On its own, it appears, quizzing helps students systematize their ideas and create a rich “structure” of knowledge.

These insights apply to lengthier tests, too — whether in math or English or social studies. As Logan Fiorella of the University of Georgia, and Richard Mayer of the University of California Santa Barbara argue in their book Learning as a Generative Activity, “students will learn more deeply if they are able to effectively select main ideas, organize them, and then restate them in their own words.”

In sum, testing “shouldn’t be a white-knuckle finale to a semester’s work,” writes the University of Washington psychologist Henry L. Roediger III, “but the means by which students progress from the start of a semester to its finish, locking in learning along the way.”

Still skeptical about the power of testing in learning? Consider a paper released earlier this year. The authors evaluated more than 250 experiments with more than 10,000 participants, in which students had prepared for exams by either rereading material they’d been exposed to — a classic approach — or being quizzed about it. The researchers found that quizzing sparked far better outcomes than rereading. They concluded: “An overwhelming amount of evidence reviewed in this meta-analysis suggests that [quizzing] increases achievement.”

If you are out of school, you can still make use of these insights. Whenever you’re trying to grasp a new subject, experts recommend that you ask yourself a lot of “why” questions. Yes, the Fed tends to raise interest rates when inflation rises — but why?

There’s been progress on the ground: a shift from bad tests to research-backed exams

In recent years, America’s schools have taken some important steps forward when it comes to tests. In K-12 schools, the exams associated with the new Common Core standards are better than previous ones, including a lot more open-ended and essay items.

Policymakers at the state and federal level have also been lowering the stakes on exams. Fewer states use a single exam to determine if a student should get a diploma, for instance. As the result of a new federal education law, states are taking a broader approach to evaluating schools, moving beyond the results of a single test-score to assess school performance.

More and more educators are using “exit slips,” a type of end-of-the class quiz to test student knowledge — the K-12 equivalent of those college clicker quizzes.

No doubt, a lot of bad testing still goes on in high-school and college classrooms, and even well-constructed, research-based tests are never going to inspire love letters. But in the end, the proof is in the outcomes, and students who experience more regular quizzing end up posting higher grades on their final exams.

Some students have come to this realization on their own. On the day that I visited Ben Dreyfus’s physics classrooms at the University of Maryland, I sat next to a junior named Brandon Fish. As he pressed the buttons to answer Dreyfus’s questions, he was sometimes surprised by what he remembered, or had forgotten. It could be distressing to discover holes in his knowledge, Fish said, but “it’s helpful to know what you don’t know.”

Near the end of our interview, Fish said something that I thought that I’d never hear from a college student, especially from a generation that has supposedly been tested to death. “I like the quizzes,” he said.

Ulrich Boser is a senior fellow at the Center for American Progress. This article was adapted from his new book, Learn Better.

The Big Idea is Vox’s home for smart discussion of the most important issues and ideas in politics, science, and culture — typically by outside contributors. If you have an idea for a piece, pitch us at thebigidea@vox.com.

I learned how to do math with the ancient abacus — and it changed my life

2017-03-06T08:00:07-05:00

A few years ago, I stood in a small basement classroom just outside of New York City, watching a high schooler named Serena Stevenson answer math questions in rapid succession.

An instructor read out numbers —

74,470

70,809

98,402

— and Stevenson added them in her head. For each question, she closed her eyes, and then the fingers of her right hand began to twitch, a progression of plucks and jerks. The movements were fast and exact.

For almost an hour, she used the abacus-based approach to solve math problems. Sometimes she would get problems wrong and smile and shrug. But she also answered many of the problems correctly, including the addition of multiple five-digit numbers in her head.

The key to her success was an ancient technology called the abacus. As I discovered while reporting on a book on the science of learning, the typical abacus has small discs that move up and down on thin posts. The small discs have different values, and the four beads on the bottom have a value of 1. The discs at the top have a value of 5. To calculate a problem, you move the discs up and down until you get to a solution.

For much of that evening, Stevenson used a practice called “mental abacus,” envisioning the abacus in her mind and then using her fingers to work through the problem.

From watching Stevenson, I knew that gaining skills on the abacus was more than a matter of counting beads, and so I decided to enroll myself and my two daughters in an abacus course to see if we could also hone our math skills. Along the way, I learned surprising insights into how people gain new skills.

The abacus: an ancient technology with modern relevance

As a technology, the abacus predates the making of glass and the invention of the alphabet. The Romans had some sort of counting device with beads. So did the early Greeks. The word “calculate” comes from the expression “drawing pebbles,” basically using some sort of abacus-like device to do math.

Researchers from Harvard to China have studied the device, showing that abacus students often learn more than students who use more modern approaches.

UC San Diego psychologist David Barner led one of the studies, and he argues that abacus training can significantly boost math skills with effects potentially lasting for decades.

“Based on everything we know about early math education and its long-term effects, I’ll make the prediction that children who thrive with abacus will have higher math scores later in life, perhaps even on the SAT,” Barner told me.

These sorts of conclusions have inspired groups of abacus devotees, and schools devoted to the practice have been popping up everywhere Los Angeles to New Jersey. My sister, Katharina, counts herself among the converts. A technology teacher in Maryland, she started using the tool to teach her students math a few years ago. Now she gives abacus workshops and has half a dozen different abacus apps to help her students hone their skills on the tool.

To learn something new, your brain needs to be fully engaged

When first I watched high school abacus whiz Serena Stevenson, her hand gestures seemed like a pretentious affect, like people who wear polka-dot bow ties. But it turned out that her finger movements weren’t really all that dramatic, and on YouTube, I watched students with even more theatrical gesticulations. What’s more, the hand movements turned out to be at the heart of the practice, and without any arm or finger motions, accuracy can drop by more than half.

Part of the explanation for the power of the gestures goes to the mind-body connection. But just as important is the fact that abacus makes learning a matter of doing. It’s an active, engaging process. As one student told me, abacus is like “intellectual powerlifting.”

Psychologist Rich Mayer has written a lot about this idea, and in study after study he has shown that people gain expertise by actively producing what they know. As he told me: “Learning is a generative activity.”

The power of mentally doing is clear in memory tasks. Want to remember the French word for home, “maison,” for instance? People are far more likely to recall the word “maison” if a letter is missing from the word — e.g., “mais_n.” When people add the “o,” they’re more engaged and thus learn more.

This idea also extends to more difficult cognitive tasks. Take something like reading. If we push ourselves to dream up a mental image of what we’re reading, we retain a lot more knowledge. By creating a type of “mind movie,” we’re building more cognitive connections — and making the learning more durable.

Short-term memory is crucial to learning but is easily overwhelmed

After seeing abacus students like Stevenson in action, I did more research and soon discovered another reason for the abacus’s success. As an approach to learning math, abacus reduces demands on short-term memory. When people use the beads on an abacus, they’re using the device to keep track of digits, which allows them to execute more complex calculations.

This is important because short-term memory is important. Researchers now believe that everything that we learn needs to first be processed in short-term memory before the material becomes stored in long-term memory and thus learned.

The problem is that short-term memory is pretty short, and we are only able to juggle around half a dozen items at a time. This explains why we can’t multitask while learning. Music, driving, Twitter — they all drag on short-term memory, and thus keep us from understanding.

The abacus appears to have evolved over the centuries to put less demand on short-term memory, and the five beads on each post lines up pretty well with the number of items that people can retain in working memory. “You might say that the abacus makes best use of what we’ve got in terms of cognitive capacity,” UCSD’s Barner told me. “It fits the limits of human cognition.”

In this regard, the abacus provides some important take-homes when it comes to learning. We often overestimate how much information we can store in short-term memory. More precisely, people often try to learn too much at a time, taking an all-you-can-eat-style approach to gaining expertise. People will think, for instance, that they can learn from a lecture while chatting with a friend. (They can’t.) Or people will try to understand a big, complicated idea in a single sitting. (They can’t.)

Events often suffer from much the same problem. Long talks, lengthy meetings, and prolonged lectures can all erode short-term memory, crowding the limited pathway to long-term memory. For this reason, experts such as Ruth Colvin Clark argue that classes should never go longer than 90 minutes. We simply don’t have the mental stamina to continue learning for much longer.

When I interviewed psychologist John Sweller, who studies the role of short-term memory in learning, he gave the example of foreign language programs that try to instruct people in history or literature. By combining the two topics, people learn a lot less, he argues.

“You’re going to learn neither,” Sweller told me. “It’s cognitive overload.”

To learn something well, people need to know the underlying system within that area of expertise

Shortly after enrolling my daughters and myself in an abacus class, we discovered that the practice relies on a math strategy known as decomposition, which makes computation easier by breaking numbers down into their component parts. So students are encouraged to think about how certain numbers have “complements” or “partners.” For instance, 10 is made by partnering 7 plus 3 or by partnering 6 plus 4.

For an actual math problem, consider 5 plus 8. On the abacus, you would not add those actual figures. Instead, you would “decompose” the numbers and add 10 to the 5 and take away 2 — or the partner of 8 — in order to get to the answer: 13.

It can take a little longer to learn math in this way. Certainly it took me a little while to fully grasp this approach. But decomposition gives people a better underlying sense of how the math actually works. (Interestingly, my kids didn’t find the approach all that novel, since a decomposition approach is embedded in the new Common Core math standards.)

Tom Sato, Stevenson’s instructor, has taught abacus for more than a decade, and he argues that the abacus’s more systemic approach is one of the key benefits of the practice. “I see a lot of kids who struggle with math because they know that 1 and 1 is 2. But when they see 2 plus 2, they don’t know what to do,” Sato told me. “The way we teach it, we try to make a framework for students to work in, and the ones who are successful are the ones who understand the system.”

This idea extends well past math, and today a growing number of experts believe that understanding systems knowledge is key to richer forms of learning. Cognitive scientist Lindsey Richland, for instance, argues that to build concepts, to solve problems, to engage in any sort of critical thought, people need to grapple with patterns within an area of expertise. “The underpinnings of the ability to do higher-order thinking really comes down to reasoning about relationships,” she says.

As an example, take learning about the ocean. To develop reasoning, to create a systems understanding, Richland argues that people shouldn’t dwell overly on standalone facts. Rather, they should examine questions like: What happens to the ocean if the level of salt goes up? What’s the difference between oceans and lakes? How do reefs impact ocean currents?

We can do this ourselves. If you’re learning something new, ask yourself hypothetical questions. By asking “what if” questions, people gain more of a systems understanding. So if people want to know more interior design, they might ask themselves: How would I design a bathroom if my client were wealthy and loved gold? How would I design a bathroom if my client were young and disabled? How would I design a bathroom with a nautical motif?

Take any bit of literature as another example. People can gain a lot by discussing the implications of counterfactuals. Want to better understand Romeo and Juliet? Then consider what would have happened if the young lovers had not died. Would the Capulets and Montagues have continued their feud? Would the lovers have lived happily ever after?

In this sense, the ancient abacus is nothing new. As Sato told me, “The big question is: Do you just memorize certain things?” he said. “Or do you try and see how it all fits together?”

Learning runs on confidence

Some aspects of the abacus classes were not surprising, like the fact that my kids hated them. For my children, Sunday mornings were about watching cartoons, not doing math. “Boring, boring, boring,” my daughter once told me.

What I didn’t expect was the boost in mathematical self-esteem. I’m one of the many who harbor some math doubts, and whether it is trigonometry or number theory, I’ll feel a touch of fear. My typical solution is avoidance, and if I have to calculate something like percentage change, I’ll go online. One of my phone apps is a tip calculator.

Like every anxiety, there’s a lot of irrationality, and in a way, that’s why I found abacus so helpful. It gave me a dedicated way to succeed, and after a few classes — and a good amount of practice — math seemed a little less frightening. I hadn’t become Euclid. But the practice knocked my numerical trepidations down a notch.

This is a time-demonstrated power of the abacus. Self-assurance grows easily on the device, and abacus students are almost 30 percentage points less likely to be nervous about an upcoming math test, according to one study. Part of the reason, it seems, is that practice and outcomes appear to move in near lockstep, with dedication correlating highly with performance.

What’s more, abacus training underscores the importance of fluency, and most abacus courses demand students do calculations at lightning speed. And while this aspect of the practice can make math seem a little mindless, there’s a good amount of evidence behind the idea of overlearning.

My kids were glad once the classes were over, and it was clear that they, too, had gained some math confidence. My youngest daughter would increasingly beam after getting a math problem correct, while my older child brought her abacus into school to show to her teacher. Later, one of my kids even started wearing an abacus necklace. These were small successes. Just a moment of self-triumph. But that’s how people ultimately develop confidence.

Ulrich Boser is a senior fellow at the Center for American Progress. This article was adapted from his new book, Learn Better.

Adapted from Learn Better by Ulrich Boser. By permission of Rodale Books.

First Person is Vox’s home for compelling, provocative narrative essays. Do you have a story to share? Read our submission guidelines, and pitch us at firstperson@vox.com.