Thames Valley Speed Cameras

An independent report        

RANDOMISED CONTROLLED TRIALS

 

RCTs (Randomised Controlled Trials) are scientific trials that are standard practice for finding out what effect an "intervention" has in a complex environment. In other words, RCTs would directly answer the question: "What effect do speed cameras have on the number and severity of casualties on the roads where they operate?"

 

Any competent engineer selecting speed camera sites would obviously use an RCT in order to find out what effect the speed cameras have on road safety, yet it is surprising that to this day (over 20 years later) no scientific trials have ever been run!

 

2.1 The idea behind RCTs is remarkably simple.

2.2 What is wrong with simply comparing collision rates before and after speed cameras were deployed?

2.3 How might an RCT be performed to evaluate the effect of 10 new speed cameras?

2.4 Why do the authorities refuse to perform RCTs?

2.5 RCTs can be run both when installing speed cameras, and when removing them

2.6 How might an RCT be used when removing speed cameras?

 

2.1 The idea behind RCTs is remarkably simple.

 

By randomly selecting ‘test’ and ‘control’ sections of road, the effects of all factors should be evenly distributed between the ‘test’ and ‘control’ groups, with the exception of the effect of the speed cameras at the test sites. This should leave the speed cameras as the sole cause of any difference between the ‘test’ and ‘control’ sections of road. Note: RCTs can be used for any site-based intervention, not just speed cameras, such as lower speed limits, traffic calming etc.

 

RCTs are most commonly used to test new medical drugs in a particular form of RCT called a double blind trial, wikipedia.

 

2.2 What is wrong with simply comparing collision rates before and after speed cameras were deployed?

 

There are many factors that could cause collision rates to change at speed camera sites therefore the initial problem of a simple before/after comparison is that the results are not the effect of the speed cameras, they are the effect of all the factors combined. The problem then is that, in order to evaluate the effect of the actual speed cameras, all the other factors have to be evaluated and this is far more complicated than simply running RCTs in the first place. What turns this into a major problem, though, is the fact that the authorities have been unable or unwilling to fully account for the other factors. The result of this is that the effect of speed cameras has never been officially established.

 

Collision rates at speed camera sites may change due to:

 

2.2.1 The speed cameras (the effect we're trying to measure)
2.2.2 RTM (Regression To the Mean, an effect of the site-selection process)
2.2.3 General influences (aka "trend”, such as increasing traffic flow, improved vehicle safety, cost of fuel etc)
2.2.4 Diversion of traffic

 

These 4 factors are mentioned in the government's largest report on speed cameras, the 4YE (Four Year Evaluation), but there are other factors as well such as other site-selection effects and co-intervention (other measures also installed).

 

The most important factor to consider is RTM. The 4YE estimated that RTM had a larger influence than all other factors combined on KSI (Killed or Seriously Injured) collision rates at speed camera sites. Furthermore, at mobile speed camera sites in Thames Valley, the RTM effect was greater than the entire KSI reduction that occurred. This meant that, after RTM was excluded from the results, KSI were found to have increased following the deployment of mobile speed cameras.

 

Furthermore, there have been 2 systematic reviews of speed camera reports (1) (2), both of which searched world-wide for the best quality reports. Both reviews concluded that the quality of evidence needed to be improved and both reviews recommended running speed cameras in RCTs.

 

2.3 How might an RCT be performed to evaluate the effect of 10 new speed cameras?

 

2.3.1 Select 20 potential new sites in the usual way

2.3.2 Install any interventions that are deemed necessary in all 20 sites (other than the speed cameras, of course)

2.3.3 Arrange the sites into matched pairs

2.3.4 RANDOMLY select 1 of each pair to be a "test" site, the other will be a "control" site

2.3.5 Operate speed cameras at the 10 test sites

2.3.6 After some time (usually 3 years), compare the changes in collision rates at the test and control sites

 

We might find that KSI collisions have reduced at the speed camera sites by say 42% (national average) but we don't stop there. We then evaluate the identically prepared “control” sites. If there had been a smaller reduction at the control sites (say 32%), then speed cameras have provided a benefit. But if, on the other hand, there was a larger reduction at the control sites (say 52%), then speed cameras have contributed to more collisions.

 

How the RCT works should be obvious. Collision rates at the test and control sites should be influenced by all the same factors except for the speed cameras, which should only influence the test sites. Therefore any difference between the test and control sites should only be due to the effect of the speed cameras (plus some random variation). The effect of random variation can be reduced by including more sites, or by performing further RCTs and combining the results, or both.

 

Apart from a possible issue of some traffic diverting away from the routes with speed cameras (which would need to be evaluated), the differences between the speed camera sites and the control sites would give a scientific answer to the question: “What effect do speed cameras have on road safety on the roads where they operate?

___________________________________________

 

Note: The importance of the RANDOM nature of the selection (2.3.4) cannot be underestimated. Without the randomised selection then “selection bias” may be introduced and the comparison sites could not be used as controls.

 

Note: Because speed cameras have become so political, the randomised selection (2.3.4) would have to be witnessed and verified by independent engineers and the start of the trial would have to be announced in advance. It would not now be acceptable to publish results of a trial in which it was claimed that the randomised selection was competently performed.

 

Note, the results of a trial of only 10 speed cameras may not be statistically significant and preferably there would be many more speed cameras in the trial, but even a 10 speed camera RCT would be a start. If further RCTs were performed, and if each conformed to a national standard, then every new trial would add to the evidence base.

___________________________________________

 

2.4 Why do the authorities refuse to perform RCTs?

 

2.4.1 The government's largest report on speed cameras, the 4YE (Four Year Evaluation) gives one reason. The 4YE states that RCTs are not run (p51): "Because this approach entails non-treatment of eligible sites, it can be difficult to justify, especially in large-scale studies of beneficial treatments."

 

This demonstrates a worrying lack of understanding:

 

2.4.1a How can they know that speed cameras are "beneficial" if they refuse to perform RCTs to find out (catch 22)?

 

2.4.1b RCTs do allow the use of other interventions therefore no site needs to be left without treatment (2.3.2).

 

2.4.1c There are thousands more sites that meet requirements for speed cameras than there are speed cameras to install (or resources to treat in other ways) therefore eligible sites are already being left without treatment. The 4YE actually acknowledges this (p51): "...studies sometimes use a group of comparison sites that are similar to the study sites..." and even cites a report that (p51) "(...used nearby untreated sites with similar collision frequencies for comparison)".

 

2.4.1d Similar reasons to refuse to run scientific trials could be used for new medical drugs. How can the medical profession hold back the "beneficial treatment" of a new medical drug from patients while lengthy scientific trials are run and still find justification for that delay? Scientific trials are essential because we need to know, not only that the drug has genuine positive benefits, but that these outwiegh any potential negative side effects. Would you give a child a new drug that hasn't been properly tested? We should expect similar standards for interventions that affect our safety when we use the roads. Just like new medical drugs, speed cameras should not be used without proper testing.

 

2.4.2 A second reason not to use RCTs is that they could cost more. For instance, it may cost more to evaluate 20 sites than 10 and there is also the cost of at least one independent engineer to oversee the trial.

 

2.4.2a Cluster analysis is a standard procedure that LAs (Local Authorities) perform. They get the collision data recorded by the Police, plot these on a map of their area, look for clusters of collisions and analyse them. Therefore the cost of finding sites suitable for speed cameras and other interventions is generally covered, whether RCTs are run or not.

 

2.4.2b When scientific trials are not run, it can be very difficult to assess the effects of speed cameras due to the effects of other factors. It is possible that almost as much has been spent attempting to account for all the other factors as it would have cost to run scientific trials in the first place.

 

2.4.2c Running an RCT is a "one-off" cost (the speed cameras are a continuing cost year after year).

 

2.4.2d Speed cameras are very expensive. In Thames Valley alone, they cost society £6.8 million each and every year. Therefore, if there are any extra costs (site evaluation and independent oversight), these would be minimal in comparison to the total costs and could be paid for from the fine revenue or speed awareness course fees.

 

2.5 RCTs can be run both when installing speed cameras, and when removing them

 

RCTs should have been performed when speed cameras were installed but, if the authorities continue to refuse to do that, RCTs can be run in 'reverse' instead when speed cameras are removed.

 

TVSRP state, "If we took out cameras like this, we would expect to see the number of crashes rise again" but, since the largest government report shows that speed cameras may not have saved any lives, nor prevented any serious injuries, it seems unlikely that removing them would make crashes rise. But isn't finding this out the whole point of the RCT?

 

It should be clear that, if speed cameras were removed following a lower than usual crash rate, then the crash rate may well rise simply due to RTM but, if we removed speed cameras after a higher than usual crash rate, then the crash rate could be expected to reduce. In other words, choosing which speed cameras to remove would tell us nothing about the effect of the speed cameras, because the effect of RTM could well be greater. This is why speed cameras must not be removed unless this is done within an RCT.

 

2.6 How might an RCT be used when removing speed cameras?

 

Very easily. For example, there are 212 active fixed speed camera sites in Thames Valley and they could be matched into 106 pairs according to their collision histories. All other site-treatment and maintenance would be completed and then one of each pair would be chosen by RANDOM selection to be removed.

 

If the change in collision rates at the sites where the speed cameras were removed was different to the sites where speed cameras remained, then we would have clear evidence of the effect of speed cameras, at least on those sections of road.

 

Note: This comparison only works when the speed cameras that are removed are selected at RANDOM within an RCT.

 

Politically, we need to create the basis for agreement therefore speed camera supporters and anti speed camera campaigners need to be brought together BEFORE any RCT trial starts in order to approve the parameters of the RCT. This could, for the 1st time, provide the basis for agreement in both the evidence and the resulting policies.