What is an RCT? – Balloon Ventures

What is an RCT?

An RCT is a specific type of evaluation which aims to reduce biases in estimating the impact of a programme and allow for causal claims (i.e. X caused Y). Let’s look at an example to understand how.

Say we run a nursery school and want to increase the number of parents who send their children to the nursery. To do this, we buy a new swing set. We compare the number of children attending the school before the installation of the swing set and after and here’s what we find:

There are more children enrolled after the installation of the swing set, therefore we can conclude that the swing set had an impact, right?

Well, maybe. Something did, it could have been the swing set or it could have been something else. For example, what could we conclude if we learnt that on the same day as the installation of the swing set, the local council started an incentive scheme to support parents in taking their children to nursery. It could either be the swing set or the new scheme.

Let’s say there is an identical nursery school next door and we also counted number of children at their school. This might help us figure it out. If we got the following data what could we conclude:

Since both went up an equal amount, it seems that the swing set had no impact on student numbers (there is no reason why our new swing set increased number of children at the other school). Instead, it seems that something which affected both schools is driving the change, this must be the local council’s new scheme. Let’s look at another situation:

Both nurseries experience some increase in children attending, so we know the policy had some effect (difference ‘a’ in graph above), but we can clearly see that our nursery has had a greater jump, this would suggest that the swing set had an impact above and beyond the scheme (difference ‘b’ in the graph above).

The second nursery in this example would be what an RCT calls a ‘control group’ (i.e. an example of the group you’re studying which doesn’t receive the ‘programme’ you’re studying). Control groups are important because they allow you to isolate the effect of a specific factor, in this case the addition of a new swing set. I.e. we wouldn’t have been able to figure out what was happening if we didn’t have the data from the other school. In other words, if you have two identical groups of people/nurseries/etc., and something happens to one group but not the other, any change in outcomes would be down to that one thing.

Control groups are all well and good, but how do you create a control group that is identical to the group you’re interested in? That’s where randomisation comes in. Say you have 1,000 people and want to create two groups of 500. If you randomly allocated people to either of these two groups, you could be confident that the two groups were basically the same (i.e. same number of men and women, same average age, etc.).   

Balloon Ventures is running an RCT of our enterprise programme in collaboration with Stanford University, find out more here. 

If you want to learn more about RCTs check out this resource: https://www.povertyactionlab.org/research-resources/introduction-evaluations