Thames Valley Speed Cameras

An independent report        

In the final analysis, the evidence presented in the Four Year Evaluation suggests that speed cameras made very little difference to the number of fatal or serious injuries. Speed cameras might prevent a few deaths, or contribute to a few more occurring, therefore, with public safety at stake, the conclusion must surely be that far more accurate methods of assessment are required. Although I developed the most accurate method to evaluate the effect of speed cameras (the FTP method) there is no substitute for the gold standard: RCTs (Randomised Controlled Trials).

 

This report recommends that speed cameras should only be operated within Randomised Controlled Trials.

 

3.6 Understanding RTM (Regression To the Mean)

 

Because RTM seems so difficult for the authorities to understand, it may be worth a quick explanation of how it occurs. Let us suppose that speed cameras do absolutely nothing for road safety and that all sections of road have a mean (or normal) rate of 1.7 KSI (Killed or Seriously Injured) every 3 years. A collision either occurs or doesn't therefore there can never actually be 1.7 KSI at any particular site, the nearest would be either 1 or 2. Further, due to the random nature of collisions, a range of perhaps 0 to 4 would not be surprising on any particular section of road.

 

If speed cameras were only deployed after 3 or 4 KSI, then the average prior to speed cameras might be 3.3 KSI (which is the case at speed camera sites nationally). Remember that the mean rate is still 1.7 KSI at these sites, it's just that they were chosen after KSI were "unusually high". Therefore the average with speed cameras in operation would still be the mean rate of 1.7 KSI. This is a reduction of 48%, with the speed cameras having played no part in producing this reduction.

 

Therefore, in this example, the 48% reduction is entirely due to RTM (as a result of the site-selection process), with no effect by the actual speed cameras. You could do anything at these sites such as release a red balloon, stand on your head, deploy a speed camera, or do nothing at all. Any of these would still get a 48% reduction and this is the effect known as RTM.

 

3.7 The Four-year Evaluation is not unusual

 

The vast majority of official reports on speed cameras are "before/after" studies without controls for effects such as RTM. Reductions in fatal and serious collisions occur at speed camera sites but the entire reduction could simply be due to RTM, trend and other factors, with the speed cameras producing no benefit whatsoever. There are reports, like the 4YE, that contain estimates for RTM but none contain measurements and none have scientific trials. The evidence in favour of speed cameras could accurately be described as "weak" or "poor" at best. Speed cameras cost millions of pounds to install, operate and maintain and millions of drivers are prosecuted. They affect the behavour of nearly all drivers, not just those who are prosecuted or those who will crash, therefore we need to determine the overall effect that speed cameras are having.

 

3.8 Why do the DfT and other authorities refuse to perform RCTs (scientific trials)?

 

The 4YE states that scientific trials (RCTs) might not be run (p51): "Because this approach entails non-treatment of eligible sites, it can be difficult to justify, especially in large-scale studies of beneficial treatments."

 

This demonstrates a worrying lack of understanding:

 

3.8.1 How can they know that speed cameras are "beneficial" if they refuse to perform RCTs to find out (catch 22)?

 

3.8.2 RCTs do allow the use of other interventions therefore no site needs to be left without treatment (2.3.2).

 

3.8.3 There are thousands more sites that meet requirements for speed cameras than there are speed cameras to install (or resources to treat in other ways) therefore eligible sites are already being left without treatment. The 4YE actually acknowledges this (p51): "...studies sometimes use a group of comparison sites that are similar to the study sites..." and even cites a report that (p51) "(...used nearby untreated sites with similar collision frequencies for comparison)".

 

3.8.4 Similar reasons to refuse to run scientific trials could be used for new medical drugs. How can the medical profession hold back the "beneficial treatment" of a new medical drug from patients while lengthy scientific trials are run and still find justification for that delay? Scientific trials are essential because we need to know, not only that the drug has genuine positive benefits, but that these outwiegh any potential negative side effects. Would you give a child a new drug that hadn't been properly tested? We should expect similar standards for interventions that affect our safety when we use the roads. Just like new medical drugs, speed cameras should not be used without proper testing.

 

Our safety on the roads relies on the competence and honesty of the DfT, yet they are not doing their job. They refuse to subject speed cameras to scientific trials, they don't measure RTM, they don't even attempt to use the RTM estimates they have obtained to evaluate the overall effects of speed cameras and they are being dishonest with the public. This would be completely unacceptable (and possibly illegal) with any other "safety" device, so why is this allowed with speed cameras?

 

The author recommends that speed cameras must only be deployed within scientific trials (RCTs).

 

 

3.9 References

 

The Four-year Evaluation can be downloaded direct from official website, or from here: The Four-year Evaluation.

 

Details of calculations are on this spreadsheet, 4ye_rtm.xls.

 

Note1: From the spreadsheet, effect of speed cameras is 19.3% of reduction in FSCs therefore, out of a total of 100 fewer fatalities pa, the speed cameras resulted in 19 fewer fatalities pa at the speed camera sites (In other words, there would have been 81 fewer fatalities pa anyway, even without the speed cameras).

 

From the spreadsheet, the 1,745 fewer KSI pa was over and above trend which, before trend, was a total reduction of 2,224 pa. Since 100 of these were fatal, that means there were 2,124 fewer serious injuries pa. If 19.3% of these were due to the speed cameras, that's a reduction of 410 serious injuries pa.

 

Note2: p46 Table 3.10: There are 3,752 speed camera sites resulting in 19 fewer fatalities pa (from note1 above), therefore this is a reduction of 1 fatality every 194 years per site on average (3,752/19.3).

 

Note3: p35 Table 3.2: There are 4,172 speed camera sites resulting in 410 fewer serious injuries per year (from note 1 above), therefore this is a reduction of 1 serious injury every 10.2 years per site on average (4,172/410).

 

There appear to be 420 speed camera sites where the fatality figures were not obtained.

It is unclear why the authors of the 4YE failed to collect all the data.

 

Note4: From spreadsheet, there were 212.5 fatalities pa after speed cameras started operating at 3,752 sites which works out at 1 fatalty every 17.7 years per site on average.

 

Note5: From spreadsheet, there were 2191 serious injuries pa after speed cameras started operating at 4,172 sites which works out at 1 serious injury every 1.9 years per site on average.

WHY NOT JUST READ THE GOVERNMENT REPORTS ON SPEED CAMERAS?

 

This page is an examination of the Government's "Four-year Evaluation" report.

 

The 4YE (4 year Evaluation) is the largest and most comprehensive speed camera report published by the UK government and it is considered to be the foundation of the entire UK speed camera programme. But the 4YE needs to be explained as it is a very deceptive report. Some say that it has been "sexed up" and even called it "the dodgy dossier of speed cameras".

 

The 4YE "Executive summary" states:

p4: "Overall 42% fewer people were killed or seriously injured. At camera sites, there was also a reduction of over 100 fatalities per annum (32% fewer). There were 1,745 fewer people killed or seriously injured..."

 

But these reductions were not caused by the speed cameras. Analysis contained at the end of the 4YE estimates that the vast majority of these reductions would have occured anyway, even if no speed cameras had been deployed. In fact, the 4YE does not state how many lives were saved or how many serious injuries were prevented but, when the effects of speed cameras are calculated (only using information contained within the 4YE), the rate of reductions is found to be:

 

  • Speed cameras prevent on average one fatality within each speed camera site once every 194 years,

  • Speed cameras prevent on average one serious injury within each speed camera site once every 10.2 years,

 

But the evidence in the 4YE suggests even this may be an over estimate of the benefits of speed cameras and, furthermore, the 4YE states that some road users may divert to routes without speed cameras. Any collisions involving the traffic that diverts will then occur on other roads instead therefore speed cameras may simply change where deaths and serious injuries occur, rather than actually prevent them. As a result, speed cameras may not have saved any lives, and may not have prevented any serious injuries overall.

 

The main conclusions:

 

  • The Four-year Evaluation does not establish that speed cameras saved any lives or prevented any serious injuries.

 

  • The Four-year Evaluation is incomplete and its conclusions are unsubstantiated.

 

  • Considering that lives are at stake, we must demand FAR higher standards of testing, analysis and reporting.

 

Recommendations:

 

  • In order to provide the highest quality of evidence, the author recommends that speed cameras in the UK must only be operated within scientific trials which should take the form of independently organised and independently supervised rigorous Randomised Controlled Trials. All trials should conform to a national standard so that the results can be combined into a national evidence database. In this way, the effects of speed cameras could be demonstrated and quantified scientifically.

 

  • Another new policy should be adopted to ensure that area-wide effects of speed cameras are independently monitored, analysed and reported.

Contents

 

3.1 Introduction
3.2 The Four Year Evaluation
3.3 How large was the RTM effect?
3.4 Does the 4YE present evidence that speed cameras saved lives or prevented serious injuries?
3.5 Is the estimate of RTM effects in the 4YE accurate?
3.6 Understanding RTM (Regression To the Mean)
3.7 The Four-year Evaluation is not unusual
3.8 Why do the DfT and other authorities refuse to perform RCTs (scientific trials)?
3.9 References

 

3.1 Introduction

 

Official speed camera reports consistently find large reductions in numbers of fatal or serious collisions at speed camera sites but, because speed cameras might often be deployed at sites following unusual spikes in collisions, large reductions could be expected afterwards anyway, even if speed cameras had not been deployed. This effect is called RTM (regression to the mean) but no official report has managed to fully exclude the effect of RTM from its results. Official reports, therefore, have been unable to establish what effect the actual speed cameras have had and some may find it surprising how poor the quality of official evidence is of the effect of speed cameras on fatal and serious injury rates. It is clearly impossible to examine every report here so what follows is an examination of the largest government report on speed cameras in Britain:

 

3.2 The Four Year Evaluation

 

The 4YE (Four Year Evaluation) report can be downloaded direct from official website, or here: The Four-year Evaluation.


The entire 4YE is full of dubious assumptions and incomplete analysis but this examination will focus on just two simple questions:

 

1) How many lives do speed cameras save?
2) How many serious injuries do speed cameras prevent?

 

We might expect that answering these two questions would be the main purpose of a national report on speed cameras but the 4YE does not answer either of these directly. The information is in the 4YE, but the answers have to be worked out. We start by observing that the 4YE contains none of the following:

 

1) No scientific trials of speed cameras (no Randomised Controlled Trials)

2) No measurement of the RTM effect (Regression To the Mean), possibly the most important factor at speed camera sites

3) No comparison or control sites

4) No estimated overall effect of the speed cameras in the Executive summary

5) No estimated area-wide (or national) effect of the speed cameras

 

What the 4YE does contain is an estimate of RTM at a small sample of speed camera sites but the 4YE does not use this to calculate the overall effect of speed cameras on fatalities, or on serious injuries. Therefore what follows here are the necessary calculations and results, only using information contained within the 4YE. Starting with the "Executive summary":

 

4YE (p4) "Overall 42% fewer people were killed or seriously injured. At camera sites, there was also a reduction of over 100 fatalities per annum (32% fewer). There were 1,745 fewer people killed or seriously injured...".

 

These reductions were not due to the speed cameras, though, because there are several factors that influence collision rates at speed camera sites. The most important factor to consider is RTM (regression to the mean).

 

3.3 How large was the RTM effect?

 

The key to understanding the effects of speed cameras is this general statement of the RTM effect:

 

4YE (p157) "Thus RTM accounts for about three fifths of the observed reduction in FSCs (Fatal and Serious Collisions) with the effects of the cameras and trend each accounting for a fifth".

 

RTM is therefore estimated to be larger than all other factors combined (around 60% of the reduction in fatal and serious collisions). RTM occurs due to speed cameras often being located at sites following an unusual spate of collisions that are unlikely to reoccur. Collision numbers would therefore reduce anyway, even if speed cameras had not been deployed.

 

And Trend is estimated to be responsible for a further 20% of the reductions. Trend is the name given to the effect of all of the general influences on collision rates (including rising traffic volumes and safer vehicle design). The recorded numbers of serious injury collisions reduced over the entire road network, the vast majority of which had no speed cameras, therefore similar reductions would be expected at the speed camera sites, even if speed cameras had not been deployed.

 

Therefore, when the 4YE executive summary states there were 100 fewer deaths and 1,645 fewer people seriously injured, the vast majority of those reductions would have occurred anyway, even without the speed cameras. Because only around a fifth (actually 19.3%) of the reduction in fatal or serious collisions at the sites was due to the speed cameras, the actual speed cameras resulted in around 19 fewer deaths and 410 fewer serious injuries per year1 at the speed camera sites. The 4YE states that these reductions occurred at around 4,000 speed camera sites across Britain therefore the rate of reductions are:

 

  • Speed cameras prevent on average one fatality within each speed camera site once every 194 years2.

  • Speed cameras prevent on average one serious injury within each speed camera site once every 10.2 years3.

 

(To put these figures into context, even after speed cameras were deployed there were 213 fatalities and 2,191 serious injuries at the sites every year. That's one fatality every 17.7 years4 and one serious injury every 1.9 years5 per site).

 

In order to calculate the above effects of speed cameras, the following assumptions had to be made:

 

3.3.1 the estimates of the effects of speed cameras in Appendix H of the Four-year evaluation are accurate,
3.3.2 the estimates accurately represent the effects of all speed cameras nationally,
3.3.3 the effects are the same for casualties as they are for collisions,
3.3.4 the effects are the same for fatalities as they are for serious injuries.

 

These assumptions have to be made otherwise what was the point of estimating RTM in the first place? Having included the estimate for RTM, why has the 4YE not gone on to estimate how many lives may have been saved or serious injuries prevented and made this clear in their "Executive summary"? This clearly demonstrates that the 4YE is incomplete.

 

3.4 Does the 4YE present evidence that speed cameras saved lives or prevented serious injuries?

 

If we assume that the RTM estimate in the 4YE is accurate and that there were, therefore, 19 fewer deaths and 410 fewer serious injuries at the sites each year as a result of the speed cameras, were these deaths and injuries actually prevented?

 

The 4YE states (p156): "... part of the reduction in collisions attributable to the cameras may be due to diversion of traffic away from routes with cameras".

 

The number of motorists that divert may be very low, but those that do may be in higher risk groups. For example, new drivers can now be banned if they get two minor speeding convictions within their first two years and thrill seekers on sports motorcycles could prefer routes without speed cameras. Small diversions of high-risk traffic may result in disproportionately high reductions in fatal or serious injuries at the speed camera sites, only for those injuries to occur on the roads where that traffic went. Unfortunately the 4YE makes no estimate of the value of this effect but, if each speed camera moves the location of just one fatality within 194 years on average, then the speed cameras didn't actually save any lives overall.

 

Similarly, if each speed camera moves the location of just one serious injury within 10.2 years on average, then the speed cameras did not actually prevent any serious injuries either.

 

Ultimately, the evidence presented in the 4YE suggests that speed cameras have a relatively small effect and may not have saved any lives or prevented any serious injuries whatsoever. If the estimate of RTM in the 4YE is accurate, the 4YE provides no real evidence that speed cameras have produced a genuine benefit.

3.5 Is the estimate of RTM effects in the 4YE accurate?

 

Note: KSI means "Killed and Seriously Injured" casualties in road traffic accidents

 

Figure 3.1 shows the RTM estimate in the 4YE (data from Table H9 on p158). The blue column shows the number of KSI collisions per site per year before speed cameras were deployed. The red column shows the estimate of the number that would have occurred afterwards had speed cameras not been deployed, and the green column shows the number that did occur after speed cameras were deployed.

 

So the difference between the blue column and the red column is the combined effect of RTM and trend. This is the reduction that would have occurred anyway, even without any speed cameras.

 

The difference between the red column and the green column is therefore the effect of the speed cameras (a 19.3% reduction).

 

But a figure as high as 19.3% might seem unlikely when we consider that fewer than 8% of fatal and serious collisions involve a vehicle that was speeding in the first place (Table 1.2). Furthermore, like all safety devices, speed cameras may have negative side effects as well as positive benefits.

 

So how was the estimate of a 19.3% reduction arrived at?

 

The KSI collision rate had speed cameras not been deployed (the red column in Figure 3.1) was estimated using the EB (Empirical Bayes) method. The EB method uses a prediction model involving complicated formulae and data derived from a group of sites thought to be similar to speed camera sites. Although the complex EB model was specifically designed to accurately estimate collision rates, the red column in Figure 3.1 is not the EB model estimate, it's the EB model estimate plus an amount of the baseline data. The reason given for increasing the estimate was: (p149) “because the high observed collision frequency (in the baseline) will in part tend to arise because of site features which are not included in the model".

 

But this is not necessarily true and no evidence is presented in the 4YE to support this. Simply increasing the estimate will not correct for inaccuracies in the EB model, but what it will do is introduce some of the RTM effect into the final estimate, despite the fact that the whole point of applying the model was precisely to prevent that happening.

 

For example, if the equivalent of 10% of the baseline had been added to the EB model estimate, then the EB model estimate would have been as shown in the red column in Figure 3.2 (4ye_rtm.xls).

 

It is certainly a possibility that the EB model did produce an accurate estimate and that fatal and serious collision rates with and without speed cameras were found to be much the same.


The 4YE does not state whether more or less than 10% of the baseline was factored in, but the red column in Figure 3.1 is almost certainly an over estimate of what would have occurred had speed cameras not been deployed. Therefore the estimate of a 19.3% reduction due to the speed cameras is also almost certainly an over estimate.

 

This conclusion is not only supported by Police investigations into the factors that contribute to collisions, but their investigations strongly suggest that it is a substantial over estimate.