Understandings and Misunderstandings about RCTs

angus-deatonPolicy makers and the media have shown a remarkable preference for Randomized Controlled Trials or RCTs in recent times. After their breakthrough in medicine, they are increasingly hailed as a way to bring human sciences into the realm of ‘evidence’-based policy. RCTs are believed to be accurate, objective and independent of the expert knowledge that is so widely distrusted these days. Policy makers are attracted by the seemingly ideology-free and theory-free focus on ‘what works’ in the RCT discourse.

Part of the appeal of RCTs lies in their simplicity.  Trials are easily explained along the lines that random selection generates two otherwise identical groups, one treated and one not. All we need is to compare two averages.  Unlike other methods, RCTs don’t require specialized understanding of the subject matter or prior knowledge. As such, it seems a truly general tool that works in the same way in agriculture, medicine, economics and education.

Deaton cautions against this view of RCTs as the magic bullet in social research. In a lengthy but well readable NBER paper he outlines a range of misunderstandings with RCTs. These broadly fall into two categories: problems with the running of RCTs and problems with their interpretation.

Firstly, RCTs require minimal assumptions, prior knowledge or insight in the context. They are non-parametric and no information is needed about the underlying nature of the data (no assumptions about covariates, heterogeneous treatment effects or shape of statistical distributions of the variables).  A crucial disadvantage of this simplicity is that precision is reduced, because no prior knowledge or theories can be used to design a more refined research hypothesis.  Precision is not the same as a lack of bias.  In RCTs treatment and control groups come from the same underlying distribution. Randomization guarantees that the net average balance of other causes (error term) is zero, but only when the RCT is repeated many times on the same population (which is rarely done). I hadn’t realized this before and it’s almost never mentioned in reports.  But it makes sense. In any one trial, the difference in means will be equal to the average treatment effect plus a term that reflects the imbalance in the net effects of the other causes. We do not know the size of this error term, but there is nothing in the randomization that limits its size.

RCTs are based on the fact that the difference in two means is the mean of the individual differences, i.e. the treatment effects.  This is not valid for medians. This focus on the mean makes them sensitive to outliers in the data and to asymmetrical distributions. Deaton shows how an RCT can yield completely different results depending on whether an outlier falls in the treatment or control group.  Many treatment effects are asymmetric, especially when money or health is involved. In a micro-financing scheme, a few talented, but credit-constrained entrepreneurs may experience a large and positive effect, while there is no effect for the majority of borrowers. Similarly, a health intervention may have no effect on the majority, but a large effect on a small group of people.

A key argument in favour of randomization is the ability to blind both those receiving the treatment and those administering it.  In social science, blinding is rarely possible though. Subjects usually know whether they are receiving the treatment or not and can react to their assignment in ways that can affect the outcome other than through the operation of the treatment. This is problematic, not only because of selection bias. Concerns about the placebo, Pygmalion, Hawthorne and John Henry effects are serious.

Deaton recognizes that RCTs have their use within social sciences. When combined with other methods, including conceptual and theoretical development, they can contribute to discovering not “what works,” but why things work.

Unless we are prepared to make assumptions, and to stand on what we know, making statements that will be incredible to some, all the credibility of RCTs is for naught.

Also in cases where there is good reason to doubt the good faith of experimenters, as in some pharmaceutical trials, randomization will be the appropriate response. However, ignoring the prior knowledge in the field should be resisted as a general prescription for scientific research.  Thirdly, an RCT may disprove a general theoretical proposition to which it provides a counterexample. Finally, an RCT, by demonstrating causality in some population can be thought of as proof of concept, that the treatment is capable of working somewhere.

Economists and other social scientists know a great deal, and there are many areas of theory and prior knowledge that are jointly endorsed by large numbers of knowledgeable researchers.  Such information needs to be built on and incorporated into new knowledge, not discarded in the face of aggressive know-nothing ignorance.

The conclusions of RTCs are often wrongly applied to other contexts. RCTs do not have external validity.  Establishing causality does nothing in and of itself to guarantee generalizability. Their results are not applicable outside the trial population. That doesn’t mean that RCTs are useless in other contexts. We can often learn much from coming to understand why replication failed and use that knowledge to make appropriate use of the original findings by looking for how the factors that caused the original result might be expected to operate differently in different settings. However, generalizability can only be obtained by thinking through the causal chain that has generated the RCT result, the underlying structures that support this causal chain, whether that causal chain might operate in a new setting and how it would do so with different joint distributions of the causal variables; we need to know why and whether that why will apply elsewhere.

Bertrand Russell’s chicken provides an excellent example of the limitations to straightforward extrapolation from repeated successful replication.

The bird infers, based on multiple repeated evidence, that when the farmer comes in the morning, he feeds her. The inference serves her well until Christmas morning, when he wrings her neck and serves her for Christmas dinner. Of course, our chicken did not base her inference on an RCT. But had we constructed one for her, we would have obtained exactly the same result.

The results of RCTs must be integrated with other knowledge, including the
practical wisdom of policy makers if they are to be usable outside the context in which they were constructed.

Another limitation of the results of RCTs relates to their scalability. As with other research methods, failure of trial results to replicate at a larger scale is likely to be the rule rather than the exception. Using RCT results is not the same as assuming the same results holds in all circumstances.  Giving one child a voucher to go to private school might improve her future, but doing so for everyone can decrease the quality of education for those children who are left in the public schools.

Knowing “what works” in a trial population is of limited value without understanding the political and institutional environment in which it is set. Jean Drèze notes, based on extensive experience in India, “when a foreign agency comes in with its heavy boots and suitcases of dollars to administer a `treatment,’ whether through a local NGO or government or whatever, there is a lot going on other than the treatment.” There is also the suspicion that a treatment that works does so because of the presence of the “treators,” often from abroad, rather than because of the people who will be called to work it in reality. Unfortunately, there are few RCTs which are replicated after the pilot on the scaled-up version of the experiment.

This readable paper from one of the foremost experts in development economics provides a valuable counterweight to the often unnuanced admiration for everything RCTs.  In a previous post, I discussed Poor Economics from “randomistas” Duflo and Banerjee. For those who want to know more, there is an excellent debate online between Abhijit Banerjee (J-PAL, MIT) and Angus Deaton on the merits of RCTs.

Is Boys’ Underachievement Caused by Too Much Feminization in Education?

Feminization in education refers to the increasing dominance of females within the teaching profession, especially in early childhood education and primary education, and its consequences.  Various arguments are being given on why this is generally a bad thing. The first argument is that it deprives boys and girls from male role models.  In South Africa, with a sizable share of one-parent and zero-parent households, this could have a significant effect.  Secondly, when teachers are increasingly recruited from only half of the population, there is a higher chance on qualified teacher shortages.

The third argument is potentially the strongest, that increasing feminisation has negative effects on learning outcomes of boys.  PISA results have consistently shown that boys are more likely than girls to be overall low-achievers, meaning that they are more likely than girls to perform below the baseline level of proficiency in all three of the subjects that are tested in PISA: reading, mathematics and science.  Moreover, boys in OECD countries are twice as likely as girls to report that school is a waste of time, and are 5 percentage points more likely than girls to agree or strongly agree that school has done little to prepare them for adult life when they leave school.

This underachievement and these negative attitudes seem to be strongly related to how girls and boys absorb society’s notions of “masculine” and “feminine” behaviour and pursuits as they grow up. For example, several research studies suggest that, for many boys, it is not acceptable to be seen to be interested in school work. Boys adopt a concept of masculinity that includes a disregard for authority, academic work and formal achievement. For these boys, academic achievement is not “cool” (Salisbury et al., 1999). Although an individual boy may understand how important it is to study and achieve at school, he will choose to do neither for fear of being excluded from the society of his male classmates. Indeed, some studies have suggested that boys’ motivation at school dissipates from the age of eight onwards and that by the age of 10 or 11, 40% of boys belong to one of three groups: the “disaffected”, the “disappointed” and the “disappeared”. Members of the latter group either drop out of the education system or are thrown out. Meanwhile, studies show that girls seem to “allow” their female peers to work hard at school, as long as they are also perceived as “cool” outside of school. Other studies suggest that girls get greater intrinsic satisfaction from doing well at school than boys do.  Boys are more likely than girls, on average, to be disruptive, test boundaries and be physically active – in other words, to have less self-regulation. As boys and girls mature, gender differences grow even wider as boys start withdrawing in class and becoming disengaged.

These findings seem to suggest that traditional school settings are more challenging for boys than for girls.  Current school environments may inadvertently disadvantage boys with its emphasis on coursework and downplaying of competition. A lack of male teachers may increase the impression among boys that schools are something ‘for girls’. Secondly, male teachers may be more sensitive to and able to deal with these challenges.

Migration and Development

downloadA very interesting interview with Michael Clemens, migration expert from the Centre for Global Development, a US think thank.  What would happen if Europe (quite utopically) would decide to open up its borders?  Some key extracts:

On why people migrate:

Safety and opportunity depend mostly on what country you live in, and 97 percent of humanity lives in the country they were born in. For those of us born in safe, prosperous countries, such a random lottery seems quite satisfactory. Most migrants are people who have simply decided that they will not let lottery results enforced by others determine the course of their lives.

On the impact of migration on the host country:

I would go as far as to say that this is a consensus opinion among economists. That is saying a lot, because economists are known for putting caveats on everything. But all the serious evidence we have points to large gains in overall economic activity from reduced barriers to labor mobility. Ninety-six percent of American labor economists agree that the economic benefits of US immigration exceed the losses.

Unfortunately, this research is rarely used as the basis for a debate.  Instead, nasty arguments like ” if we allow them, many more will come”, tend to be accepted as truth.  What I didn’t know, is that research shows that immigration increases wages for low-skilled labour, rather than depressing them:

Research has shown that natives acquire more skill when immigration rises [by specializing in occupations requiring more complex tasks and less manual labor]. And firms adjust their investments when immigrants are present, shifting away from technologies that eliminate low-skill jobs for both low-skill immigrants and low-skill natives. Most simply of all, foreign workers are not just workers, they are also consumers. Immigrants at low wages tend to consume products, like fast food and budget clothing, that are made and sold by other low-wage workers.

This relates to the work of Ricardo on comparative advantages in trade. Countries with a lot of labor relative to capital, for example, will tend to have a comparative advantage in labor intensive goods production. Apart from human rights and social justice arguments, letting in more migrants makes economic sense, in particular in greying Europe.  Rather than fearing that immigrants will plunder social security coffers, the question is whether European welfare systems will collapse without immigrants:

A comprehensive review by the independent OECD in 2013 found that the average immigrant household in Europe contributed over £2,000 [$3,000] more in taxes than it took in benefits.

The most interesting argument in my eyes, is how often a mindset that considers migrants as people who somehow not belong where they are, is used in case of cultural tensions.  The tensions are explained as a result of migration, rather than as a responsibility of the host country.  The mindset becomes a self-fulfilling prophecy.   Clemens clarifies the point with a good analogy:

Suppose a woman is attacked by men on the street, as she walks to work. What caused the attack? It depends on your assumptions. Many people in the world do not believe that women have the unqualified right to work or to walk down any street. These people might say that the cause of the attack was that the woman’s family allowed her to take a job and walk around unguarded. If you believe that women’s rights to work and travel are beyond question, you might identify a different cause of the attack: The cause of the attack was that men decided to attack her.

Likewise, when activists hold rallies to unmistakably threaten immigrants with violence, many might describe this as social conflict “arising” from immigration. This view requires you to already have decided that migrants don’t have the right to be there—for the same reasons that saying attacks against women arise from their presence on the sidewalk requires you to have already decided that women don’t have the right to walk on the sidewalk.

Finally, how about the effects on the origin countries?  Brain drains leaving countries with few qualified doctors and engineers.  Or, the important role of remittances, many times bigger than aid, on development? Clemens underlines that immigration is different from actively recruiting people in developing countries:

So if we’re talking about immigration policy, the question “Does migration substantially harm low-income countries?” is the same as the question, “Does forcibly stopping people from leaving low income countries substantially help those countries?” To put it mildly, social science has absolutely no evidence of such a effect.

People in developed countries often see the wealth and opportunities in their country as a right, rather than as a stroke of luck, as if they have any credit in being born in a rich country.  Given the fact that countries such as Lebanon (4 million people) and Jordan (6,5 million people) each can take one million Syrian refugees, a good starting point for European countries would be to increase the numbers they’re willing to accept each year.  Articles and arguments such as this deserve to be widely read.

Eradication of Poverty on Hold?

A long piece in The Economist recently on the evolution in purchasing power parity between economies of developed and emerging countries. Up until a few years ago, it looked as if convergence would be reached within 30 years, even if excluding Chinese growth.  Hundreds of millions of people were drawn out of poverty.  Voices have been calling for the post-2015 global development goals to include the eradication of poverty by 2030.

However, the pace of economic growth has been slowing in emerging economies, not just in China, which is managing a difficult transition from low-wage, export-based manufacturing towards an economy dominated by services and internal consumption.  However, at the current pace, it will take 150 years to catch up (using as indicator GDP/ person in PPP as % of US GDP).

em_cathcing up_1

 

 

 

 

 

 

 

Convergence was foreseen by economists like Robert Solow.   As the main drivers he identified capital influx (as a result of higher interest rates offered by developing countries) and technological progress (enabling emerging economies to leapfrog development stages).  Pietra Rivoli saw a ‘race to the bottom’ by poor countries as a way to attract labour-intensive industries, allowing people to abandon agriculture, get access to better services, creating a virtuous spiral.

The main reasons why the convergence has grinded to a near standstill are:

  1. The peak of manufacturing in a country’s development occurs earlier and is lower than previously. Dani Rodrik attributes this to the growing role of technology, reducing demand for low-wage manufacturing jobs, lowering the incentive for companies to seek out regions with low wages and lowering the share of manufacturing in the total value chain of a product.
  2. The previous decade was a period of exceptional hyperglobalisation, spurred by strong demand for natural resources, China’s accession to the WTO and strong growth in trade (also outside China).

Rather than the optimistic scenario foreseeing income convergence within a generation, it looks we’re back at the slow grind towards convergence, driven by incremental progress in geography (infrastructure, see work of Jared Diamond), institutions (see work of Daren Acemoglu) and trade (e.g. regional agreements on trade in services).

The article is rather pessimistic in tone, as it considered the gains in poverty reduction as an exceptional feat not likely to be repeated soon. It raises critical questions for countries like India and Bangladesh which are looking to benefit from their demographic dividend and take over some of China’s low-wage industry.  It also underlines the need for investments in education.

Unfair Trade

FairTradeCoffee“A CUP of “fair-trade” coffee used to make mud taste good, and you could buy it only in churches and charity shops (The Economist).

A recently published 4-year study of The School of Oriental and African Studies (SOAS) in London which uncovered some uncomfortable findings about the fair trade industry in Ethiopia and Uganda, may make Fairtrade coffee even taste less good.  It has raised a flurry of reviews (The Guardian, The Economist).  Some extracts.

From The Guardian

“Our research took four years and involved a great deal of fieldwork in Africa. We carried out detailed surveys, we collected oral histories, we talked to managers of co-operatives, to owners of flower companies, to traders and government officials, to auditors, to very young children working for wages instead of going to school, to people who had done fairly well out of Fairtrade, and to people who appeared not to have benefited.”

“One of our interviewees, James in Uganda, is desperately poor and lives with his elderly father in an inadequate shack very close to a tea factory supported by Fairtrade. Despite the fact that his father was once a worker at the tea factory, James is charged fees at the factory’s Fairtrade health clinic. He cannot afford them and instead has to make his way on one leg to a government clinic more than 5km away to get free treatment.”

Some main findings:

  • Fair Trade agricultural seasonal and casual workers often earned lower incomes than those working for non-FT employers.
  • Social community services intended as a by-product of Fair Trade are often not accessible for FT workers.
  • More concern with the incomes of producers than with wage workers’ earnings.
  • Differences could not be attributed to the fact that Fairtrade cooperatives were based in areas with higher or particular disadvantages.

The study raises some questions about Fairtrade, to say the least.  It may not fit with our view of helping the poor, but workers may be better off working with large producers, offering higher wages, better facilities and more days of work.  Moreover, most of the organisations that are certified tend to come from richer, more diversified developing countries, such as Mexico and South Africa, rather than the poorer ones that are mostly dependent on exporting one crop.  As one of the researchers writes:

” If we are interested in what makes a difference to extremely poor people, it is important to compare areas with Fairtrade organisations not only with other smallholder producing areas, which we did, but also with areas where producers are much larger. If larger farmers can pay better and offer more days of work, this is surely an important thing to understand.”

The findings may have some parallels with wider issues with development:

  • the convenience of small efforts that show solidarity with the poor
  • charging a premium for easing one’s conscience
  • a stereotypical view of ‘the poor’  and what they prefer
  • the attractiveness of simple, straightforward solutions
  • a proliferation of labels and organisations, harnessing this desire ‘to do good’
  • a romanticized ‘small is beautiful’ view on development

 

Why We Disagree on Climate Change (M.Hulme)

hulme coverMike Hulme is a professor of climate change and director of the prestigious and controversial research Tyndall Centre for Climate Change Research  .  The book is not a book on the science of climate change.  If it were only about the science, Hulme argues, reaching agreement would be straightforward.   Rather, disagreements on the science of climate change are symptoms of much more fundamental differences in human nature.

Such underlying reasons include:

  • Different ways people look at science and understand its limitations
  • Different ways people see the relation between (scientific) knowledge and policy
  • Different ways people consider risk;
  • Different ethical positions people take about their responsibility for future generations and are willing to invest for future welfare (quantified in the ‘discount rate’)
  • Different beliefs in our duty to others, who’s responsible for climate change and our religions
  • Different world views, described by Douglas and Wildavsky’s Cultural Theory
  • Different ways people communicate about climate (no message is neutral) and interpret those messages
  • Different ways people think climate change should be governed.
Four Rationalities in Cultural Theory

Four Rationalities in Cultural Theory

Cultural Theory distinguishes four dominant world views, expressed by the intensity of bonds between people (individualist vs collectivist) and the extent to which people believe that few or many rules are necessary to control behaviour.

  • Fatalists: nature is a lottery, there is nothing we can do
  • Hierarchists: experts can manage the outcomes if we get behind them
  • Individualists: it’s down to individuals and markets
  • Egalitarians: altruism and common effort, built on social justice, are the way forward

The cultural dimension of climate change is also revealed by the language and images we use.  Expressions like ‘dangerous’ climate change, ‘save’ the climate, ‘tipping points’ and ‘fragile’ climate express different views on climate.  Greenpeace has used a range of images of the years, from the ‘climate change bomb’  to the polar bear to communicate climate change to different target groups.

In discussion on avoiding ‘dangerous’ climate change, questions such as whose climate (small island states, Western Europe, Saharan Africa…), dangerous for whom and what climate is desired (pre-industrial, Holocene…) are often glossed over.

These underlying differences have created various ‘stories’ about climate change, inspired by 4 basic human desires:

  • nostalgia for a lost ‘ideal’ climate
  • fear for the unknown future
  • mastering and controlling nature
  • call to respond to social injustice

Climate change has become a central concept in development as well.  Climate change prominently features in calls for proposals or as a transversal theme in programmes.  At first sight, this might seem strange.  Does it make sense to invest funds in the welfare of future generations if current generations lack basic amenities such as basic healthcare and education.  Investing in the economic development of the poor will enable them to protect themselves better against any natural disasters.  Limiting fertility rates (as a result of economic development) is likely to do more for climate change than any planned ‘development’ intervention (as China likes to point out).  Climate change mitigation and development are frequently in conflict each other.  Projects under the Clean Development Mechanism rarely benefit the poorest.  Increasing trade is bad for the carbon level in the atmosphere but has lifted millions out of poverty.  The European decision to subsidize biofuels has pushed up food prices.

However, if you look at climate change as an idea (like ‘democracy’ or ‘terrorism’) that has as much to do with science as with culture, it makes more sense.  Climate change as an idea carries a variety of meanings and interpretations.  Then, climate change becomes a tool to advance development agendas of social justice, better functioning markets or public welfare.

As solution, the book gives some recipes from complexity science.  Rather than aiming to find one global solution, a variety of approaches catering to different world views, ideas about governance, science etc. stands a better chance at curbing climate change.  Climate change derives from various other problems, such as population growth, unsustainable energy, endemic poverty, food security, deforestation, biodiversity loss…  Rather than framing climate change as a mega-problem, requiring a mega-solution, Hulme argues that disentangling the issue, moving climate change to the background, is more likely to yield effect.

If you like an intelligent take on why climate change is about so much more than debates on science, then this book is for you.  For development people, discussions on the IPCC, the Stern Report and the disagreements between North and South are all there.  Duncan Green from Oxfam also has a fine review of the book.

Discussion ‘Seeing Like a State’ (James Scott)

I recently finished James Scott’s ‘Seeing Like a State, possibly the most broadly venturing book on development I’ve ever read.  I ordered the book after reading a raving review by Duncan Green on FP2P.

What’s wrong with seeing like a state? States have been seeking to expand their power by increasing information they collect about their citizens and territory. This process from “blindness” towards ever increasing “legibility” involves a simplification of complex natural and social situations and processes, retaining only “useful” information.  Ever increasing state capacity allows pursuing ever wider ambitions to appropriate, control and manipulate (levying taxes, collecting conscripts, prevent undesired behaviour…).  Historical examples are the introduction of permanent last names and the standardization of units (for length, weight…).

Scott discusses a few cases in detail.  The introduction of German forestry science transformed chaotic “old” forests into standardized, geometric wood-production factories and enabled centralized management by state bureaucrats.   The modernist urban vision of Le Corbusier ignored both history and local people’s desires and made abstraction of the complex micro-sociology that turns areas into vibrant and attractive neighbourhoods.   Lenin’s socialist vision aimed at re-making society, ignoring political and social complexities and realities.

All those examples share a naive optimism in the ‘make-ability’of society, inspired by an erroneous belief in the scientific method.  Rather than questioning assumptions and adopting the uncertainty, characteristic for science, they overestimate the power of science and show contempt for local knowledge.  The fatal overestimation of human capability to design society and the importance of local and practical knowledge are highlighted by critics such as Jane Jacobs (on Le Corbusier’s plans) and Rosa Luxemburg (on Lenin’s plans).

Scott stresses that states can use more ‘legibility’ both to improve and curtail freedom.  It can be used to improve public welfare, introduce literacy or vaccination campaigns or offer a bulwark against global homogenisation.  However, it can just as well lead to disaster.  Administrative reordering of society and nature is not sufficient to generate social disaster.  Combined with a high modernist ideology, an authoritarian state and a weak civil society, it can have lethal consequences.

This process of legibility also affects power relations.  By discarding local knowledge, simplification of processes benefits state bureaucrats.   The language of policies, procedures and permits de-skills people with local and practical knowledge such as farmers and teachers.

“The capital has its order, the village its customs” (p.33)

This double-edged character of a legible state raises dilemmas for development agencies, as well-intended efforts to increase institutional capacity to improve citizens’ literacy may later just as well be used to suppress them.  The development community often still sees the world as something that can be planned top down, captured in a log frame.  A body of knowledge and jargon impenetrable to outsiders systematizes and simplifies a wide variety of local situations and provides an aura of authority.  The drive towards ‘evidence-based’ interventions based on ‘what works’ similarly assumes that reality can be reduced to a few variables and that only knowledge that is acquired through tightly controlled experiments is valid.  The archetypical ‘flying in’ of experts shows contempt for the history of place and domain.

A good example is poly-culture in West-Africa. It was dismissed by agriculture experts in favour of more efficient mono-culture.  However, yield maximization is not the only consideration for small African farmers.  For some, steady yields are more important than occasionally higher ones.  Constraints on land, skills, tools and labour also play a role, as well as elements of culture and group identity.

Scott suggests that ‘practical knowledge -friendly’ institutions should be flexible and adaptable to changing local realities, although he doesn’t explicitly writes it implies decentralisation.  His recommendation resembles in the discourse on complexity and development.  Secondly, development interventions should be analysed to what degree they enhance the skills, knowledge and responsibility of those who are part of it.  He pleas for polyvalent institutions that foster reasoning and negotiation among its members, distinguishing ‘canned’ institutions that permits little or no modification from those that are largely open to the development and application of local knowledge.

The book has plenty of lessons for education and development.  Supporting ministries in teacher training, curriculum design or national assessment all increase state capacity, often to the detriment of local and practical knowledge.  Traditionally, the role of the state in education has focused on the input (curriculum, textbooks) and the output (assessment, accreditation) of education, whereas the how was referred to as the ‘black box’ of education, the domain of the teacher and largely illegible to the state.  The teacher was recognized as a professional, equipped with a rich arsenal of knowledge about his/ her students, the local context and pedagogical content knowledge.  It was accepted that the teacher would know how best to translate the intended learning outcomes into powerful learning activities.  However, this zero-sum game between ‘education experts’ (academics, bureaucrats) and teachers is at play as well.  Experts use (evidence-based) education  research, large-scale assessments, control over professional development and an avalanche of reporting requirements to encroach on the ‘how’ of the teaching, characterised by ever more detailed curricular descriptions and focus on assessments.

On the other hand, some countries struggle to attract qualified teachers, in particular in areas such as maths and science.  These subjects are taught by teachers without the expertise, experience or motivation to provide qualitative education.  Stronger control by means of curriculum and assessment may be a necessary way to guarantee a minimum quality to children.  However, it might just as well kill the love and passion for teaching