Nathan's Reviews > Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement
Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement
by John Hattie
by John Hattie
John Hattie's a genius: looking at over 800 meta-analyses of over 50,000 studies, he's isolated the effect of different variables on the achievements of kids in school. There's so much here, I can dig in again and again for parenting and teaching inspiration. Here's what I took away before my brain ranneth over. (d=0.3 is a measure of how effective the intervention is; almost every intervention works, but 0.4 is the mean benefit so it's only a really GOOD intervention if it scores more than 0.4).
one of the major results presented in this book relates to increasing the amount of feedback because it is an important correlate of student achievement. However, one should not immediately start providing more feedback and then await the magical increases in achievement.As will be seen below,increasing the amount of feedback in order to have a positive effect on student achievement requires a change in the conception of what it means to be a teacher; it is the feedback to the teacher about what students can and cannot do that is more powerful than feedback to the student, and it necessitates a different way of interacting and respecting students (but more on this later). It would be an incorrect interpretation of the power of feedback if a teacher were to encourage students to provide more feedback. As Nuthall (2007) has shown, 80% of feedback a student receives about his or her work in elementary (primary) school is from other students.But 80% of this student- provided feedback is incorrect! It is important to be concerned about the climate of the classroom before increasing the amount of feedback (to the student or teacher) because it is critical to ensure that “errors” are welcomed, as they are key levers for enhancing learning. It is critical to have appropriately challenging goals as then the amount and directedness of feedback is maximized. Simply applying a recipe (e.g.,“providing more feedback”) will not work in our busy, multifaceted, culturally invested, and changing classrooms.
Marzano (2000) argued that 80 percent of the variance in achievement could be accounted for by student effects, 7 percent by school effects, and 13 percent by teacher effects. He then used these estimates to evaluate the effects on student achievement of an ineffective, an average, and an exceptional teacher in an ineffective, an average, and an exceptional school respectively.Average schools and average teachers, although he said they did little harm, also did little to influence students’ relative position on the distribution of achievement for all students; ineffective teachers, no matter how effective the school, had a negative impact on the standings of all students, whereas students of exceptional teachers, even in ineffective schools, either maintained or increased achievement, many quite substantially.“Exceptional performance on the part of teachers not only compensates for average performance at the school level, but even ineffective performance at the school level” (Marzano, 2000, p. 81).
Indeed, one of the fascinating discoveries throughout my research for this book is discovering that many of the most debated issues are the ones with the least effects. It is a powerful question to ask why such issues as class size, tracking, retention (that is, holding a student back a grade), school choice, summer schools, and school uniforms command such heated discussion and strong claims. Such cosmetic or “coat of paint” reforms are too common. So many structural claims involve the parents (more homework), lead to more rules (and therefore more rule breakers), have hints of cultural imperatives (quietness and conformity is desired), and often include appeals to common sense (reducing class size is obviously a good thing!). However, the most powerful effects of the school relate to features within schools, such as the climate of the classroom, peer influences, and the lack of disruptive students in the classroom—all of which allow students and teachers to make errors and develop reputations as learners, and which provide an invitation to learn.
What a child brings to the classroom each year is very much related to their achievement in previous years—brighter children tend to achieve more and not so bright children achieve less.This should not be surprising given that the correlation between ability and achievement is very high. Hattie and Hansford (1982) reported an average correlation of r = 0.51 between measures of intelligence and achievement (an effect size, d = 1.19).This high relationship accounts for what many researchers call (usually with a sense of surprise) the “Matthew effect”, which is based on the biblical notion that the rich get richer and the poorer get poorer or do not gain as much. Prior achievement predicts success from preschool to the first years of schooling (Duncan et al., 2007; La Paro & Pianta, 2000; Schuler, Funke, & Baron-Boldt, 1990), between high school and college or university grades (Kuncel, Hezlett, & Ones, 2001), between college and adult success (Bretz, 1989; Samson, Graue, Weinstein, & Walberg, 1984), and between grades in school and later job performance (Roth, BeVier, Switzer, & Schippmann, 1996).
About 70–80 percent of families have two parents in mostWestern countries,about 10–20 percent of families are single-parent, and about 2–10 percent are other than these struc- tures. Pong, Dronkers, and Hampden-Thompson (2003) found that single parenthood is associated with lower mathematics and science achievement (although the effects are quite small). They also noted that countries with more generous welfare policies, like Austria, showed the smallest gaps.The greatest gaps were in countries such as the United States and New Zealand, who, they claimed, lagged behind other industrialized countries in providing financial assistance, universal child benefits, tax benefits and maternity leave benefits to single and poorer families.They concluded that “to some extent the investment in national family policies explains why Australia ranks at the top but the United States and New Zealand rank last in the academic resilience of children from single-parent homes” (p. 695).
Hong and Ho (2005) concluded that parent aspirations were the most important influence on their children’s achievement, whereas parental supervision in the forms of monitoring students’ homework, time watching television, and time going out with friends appeared to have a negative effect on the educational aspirations of adolescent students. Similarly, Rosenzweig (2000) noted that the relationships between student achievement and parental participation (d = 0.56) and supportive parenting (d = 0.43) were much higher than with homework supervision (d = 0.19), participation in school activities (d = 0.14), communication with school and teachers (d = 0.14), monitoring school progress (d = 0.12), providing structure in the home (d = 0.00), and controlling and disciplining parental style (d = –0.09).These effects were the highest in high SES families, in elementary compared to high schools, and in Asian and Latino compared to white and African American families. Of as much interest are those family variables that negatively relate to achievement. These factors included external rewards, homework surveillance, negative control, and restrictions for unsatisfactory grades. Overall, “the higher the hopes and expectations of parents with respect to the educational attainment of their child, the higher the student’s own educational expectations and, ultimately, the greater the student’s academic achievement” (Hong & Ho, 2005, p. 40).These high expectations are assisted by greater parent-student communication and the student’s control over their own studies (see also Fan and Chen, 2001).
As an example of its use, Konstantopoulos (2005) found that a substantial proportion of the variation in student achievement lies within schools and not between schools. If the variance is within, this means that factors such as teacher variability have a relatively larger effect on student achievement than do school effects.“It appears that the teachers students are assigned to may be more important than the schools they attend” (p. 36).
New Zealand, as an example, has among the lowest percentage of between-school variance (about four percent and thus the within school variance is much greater.) Using data from the Second International Mathematics Study, Scheerens, Vermeulen, and Pelgrum (1989) found that school effects were undetectable as a source of variance in New Zealand, whereas between-teacher or between-class variance was 42 percent. Harker and Nash (1996; Nash & Harker, 1997) found that the school effect in New Zealand high school performance accounted for between five to ten percent of the variance in mathematics, nine to ten percent of the variance in English, and five to seven percent of the variance in science.The message is that, if you take two students of the same ability,it matters not which school they attend,but it may matter greatly who their teacher is. It is not so much that teachers matter, as that the variance within schools indicates that some teachers matter more than others!
A major reason difficult goals are more effective is that they lead to a clearer notion of success and direct the student’s attention to relevant behaviors or outcomes, whereas “doing your best” can fit with a very wide range of goals. It is not the specificity of the goals but the difficulty that is crucial to success.There is a direct linear relationship between the degree of goal difficulty and performance.There are five meta-analyses rela- tive to this contention (Table 9.2) and the overall effect size is a large d = 0.67 (these are not all achievement outcomes and so are not included in the Appendices of this book). The performances of the students who have the most challenging goals are over 250 percent higher than the performances of the subjects with the easiest goals (Wood & Locke, 1997).
Also, difficult goals are much better than “do your best” or no assigned goals. Any school with the motto “do your best” should immediately change it to “face your challenges” or “strive to the highest”.The following five meta-analyses relate to this contention (Table 9.3). This is because “do your best” goals are easily attained—in one sense, anything you do can be defined as your best. Instead, teachers and learners should be setting challenging goals.
Goals have a self-energizing effect if they are appropriately challenging for the student, as they can motivate students to exert effort in line with the difficulty or demands of the goal. Commitment to the goals helps, but is not necessary for goal attainment—except for special education students, where commitment makes a major difference. Klein,Wesson, Hollenbeck, and Alge (1999) found a high relationship (d = 0.47) between goal commit- ment and subsequent performance, and the effect between commitment and outcome increased as a function of goal difficulty. Donovan and Radosevich (1998) found lower effects of commitment to goals than they expected, but these were still quite high (d = 0.36).
When I completed the first synthesis of 134 meta-analyses of all possible influences on achievement (Hattie, 1992) it soon became clear that feedback was among the most powerful influences on achievement. Most programs and methods that worked best were based on heavy dollops of feedback.When I was presenting these early results in Hong Kong, a questioner asked what was meant by feedback, and I have struggled to under- stand the concept of feedback ever since. I have spent many hours in classrooms (noting its absence, despite the claims of the best of teachers that they are constantly engaged in providing feedback), worked with students to increase self-helping (with little success), and have tried different methods of providing feedback.The mistake I was making was seeing feedback as something teachers provided to students—they typically did not, although they made claims that they did it all the time, and most of the feedback they did provide was social and behavioral. It was only when I discovered that feedback was most powerful when it is from the student to the teacher that I started to understand it better.When teachers seek, or at least are open to, feedback from students as to what students know, what they understand, where they make errors, when they have misconceptions, when they are not engaged—then teaching and learning can be synchronized and powerful. Feedback to teachers helps make learning visible.
Programmed instruction, praise, punishment, and extrinsic rewards were the least effec- tive forms of feedback for enhancing achievement. Indeed, it is doubtful whether rewards should be thought of as feedback at all. Deci, Koestner, and Ryan (1999) have described tangible rewards (stickers, awards, and so on) as contingencies to activities rather than feed- back because they contain so little task information. In their meta-analysis of the effects of feedback on motivation, these authors found a negative correlation between extrinsic
The contributions from teaching approaches—part I 175
rewards and task performance (d = –0.34).Tangible rewards significantly undermined intrinsic motivation, particularly for interesting tasks (d = –0.68) compared to uninter- esting tasks (d = 0.18). In addition, when the feedback was administered in a controlling manner (e.g., saying that the student performed as they “should” have performed), the effects were even worse (d = –0.78). Thus, Deci et al. concluded that extrinsic rewards are typically negative because they “undermine people’s taking responsibility for motivating or regulating themselves” (Deci et al., 1999, p. 659). Rather, extrinsic rewards are a controlling strategy that often leads to greater surveillance, evaluation, and competition, all of which have been found to undermine enhanced engagement and regulation (Deci & Ryan, 1985).
Another form of feedback is repeated testing, but this is only effective if there is feedback from the tests to teachers such that they modify their instruction to attend to the strengths and gaps in student performance.Although performance is increased with more frequent testing, the amount of improvement in achievement diminishes as the number of tests increase (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991). Students taking at least one test during a 15-week term scored about half a standard deviation higher in criterion examinations than students taking no tests.When two groups answered identical test items, superior performance was obtained from students who answered the questions on a large number of short tests rather than on a small number of long tests.The caution is that it may not be the frequency of test taking but that frequent test taking made the learning intentions and success criteria more specific and transparent. Clariana and Koul (2006) found that multiple-try feedback was less effective for surface outcomes (d = –0.22) but more effective for higher-order outcomes (d = 0.10).
Many states in the United States have high-stakes testing and there is also much testing embedded in the No Child Left Behind imperatives.There have been arguments that such frequent testing is akin to a coaching effect, whereas others consider that any gains are because of narrowing the curriculum, teaching to the test, and because too many students are excluded who may not perform so well. Amrein and Berliner (2002) raised much debate with their analysis of the performance of 18 states with high-stakes testing systems and found little effect of these systems on student achievement.This conclusion was contested (e.g., Braun, 2004; Raymond & Hanushek, 2003; Rosenshine, 2003). Lee (2006) used meta-analysis to compare different state policies on the National Assess- ment of Educational Progress examination. He found six studies favored high-stakes testing states, five were mixed, and one favored low-stakes testing states. The effects were extremely varied (d = –0.67 to d = 1.24), although it made no difference as to the focus of the accountability—that is, whether the focus is a combination of schools and students d = 0.38, for schools alone d = 0.39, or for students alone d = 0.31.The effects on mathematics (d = 0.38) are slightly higher than on reading (d = 0.29), and higher for elementary (d = 0.44) and middle schools (d = 0.35) than for high schools (d = 0.03).
Sign into Goodreads to see if any of your friends have read Visible Learning.sign in »