ADI Pass Rates: the simple steps to ensure you’re top of the league

The MSA GB has always fought against the publication of individual Approved Driving Instructor (ADI) pass rates, on the basis that the data differences make both correlation and therefore sensible comparison, of data, virtually impossible.

Considering the relatively small number of tests per instructor it is unlikely, from a statistical point of view, that it would be possible to produce reliable statistics even if a complicated algorithm were to be developed to try to cover all the variables.

As I understand it the chi-squared test, used to keep a check on examiner variances relies on a large sample. In a year a driving examiner probably conducts around one thousand five hundred driving tests. A driver trainer is unlikely to take more than fifty pupils for test in a year and probably less.

Age of candidate, gender of candidate, auto or manual gears, location of test centre; all these make for large differences in the stats recorded – as do unknown variables such as a pupil who ISN’T ready for the test exercising their democratic right to put themselves in for it anyway, as they are off to uni in a couple of weeks and they’d like to get it sorted out now, thanks very much.

Computer geeks – pull together an algorithm for that one

In the examples below I have rounded the decimals. The data is all extracted from Department for Transport (DfT) and Driver and Vehicle Testing Agency (DVSA) data sets and from a Freedom of Information (FOI) request. Each example illustrates the difficulty of providing a fair rating for an instructor based on available data.

I have also made some suggestions (IN BOLD) as to actions an instructor might take to improve their pupil pass rate.

 

___________________________

Current overall pass rate 47% Male 51% Female 44%

Current overall pass rate (automatic) 40% Male 44% Female 38%

So, don’t teach Female pupils in automatics.

___________________________

Current overall pass rate 17 year olds 57% Male 58% Female 55%

Current overall pass rate 25 year olds 46% Male 51% Female 43%

So, don’t teach anyone over 17 years old.

___________________________

Highest overall pass rates for one group – 17-year olds taking their fourth test.

So, advertise for 17 year old clients who have already failed three driving tests.

___________________________

These centres are all in North London and are within about 5 miles of each other

DTC	First time Pass Rate
Wanstead	32%
Chingford	36%
Barnet	38%
Wood Green	38%
Enfield	40%
Mill Hill	43%

So, if you live in North London take your pupils to Mill Hill.

___________________________

These centres in Greater Manchester are either inside the M60 or less than a couple of miles outside it.

DTC	First time Pass Rate
Cheetham Hill	30%
Failsworth	36%
West Didsbury	43%
Bredbury	48%
Sale	50%
Hyde	62%

So, if you live in Greater Manchester and usually use Cheetham Hill stop, take your pupils for test in Hyde they are twice as likely to pass first time. 

___________________________

Although there is nothing the ADI can do about it, the route taken for each test can affect the pass rate by as much as 13% as this information extracted from an FOI request shows.

The table below shows the pass and fail rates by test route for car practical driving tests conducted at Darlington driving test centre between 1 January 2012 and 31 December 2014.  This data is only for routes where more than 10 tests have been conducted.

Route Number	No. tests conducted	Failed tests	%Pass
7	896	505	43.6%
8	724	406	43.9%
15	697	380	45.5%
6	838	451	46.2%
14	672	359	46.6%
16	30	16	46.7%
1	940	494	47.4%
4	767	400	47.8%
5	736	379	48.5%
10	716	364	49.2%
11	641	313	51.2%
12	666	320	52.0%
99	25	12	52.0%
2	837	398	52.4%
13	654	296	54.7%
9	750	337	55.1%
3	838	368	56.1%

___________________________

Then of course there is the luck of the draw as to which examiner you take as this second extract from the same FOI request indicates

The table below shows the pass rate statistics for examiners who are permanently based at Darlington test centre between 1 January 2012 and 31 December 2014.

This has been supplied in an anonymised format as we do not release pass and fail rates by named examiner. Information about named individuals is exempt from release under section 36(2)(c) on the grounds that its disclosure would prejudice the effective conduct of public affairs, a view certified by the Minister and then subject to the public interest test as to whether the balance lies in favour of withholding or disclosing the information.  We have concluded, applying a presumption in favour of disclosure, and in all the circumstances of the case, the public interest in maintaining the exemption outweighs the public interest in disclosing the information. 

Examiner	No. tests conducted	Failed tests	%Pass
Examiner 3	882	481	45.5%
Examiner 4	2,508	1,347	46.3%
Examiner 7	2,313	1,237	46.5%
Examiner 1	1,561	780	50.0%
Examiner 8	229	112	51.1%
Examiner 5	1,837	872	52.5%
Examiner 6	1,631	768	52.9%
Examiner 2	233	95	59.2%

 ___________________________

Publication of ADI pupil pass rates would be as meaningless as publishing individual examiner pass rates or the pass rate per route at a particular test center.

Now would be a good time for the DVSA to make it very clear that they do not intend to publish ADI  pass rates.