Fleet and safety: The highs and lows of being risk assessed

Online assessments have become a standard part of driver risk management, but what determines a determines a driver’s overall rating? Elizabeth Howlett reports.

Online risk assessments are designed to be a cost-effective way to determine whether a driver needs in-car training or further learning.

A series of online assessments, typically covering driving history, knowledge and hazard perception, aim to measure a driver’s potential exposure to risk.

There is a range of options on the market, and each provider has varying methods to establish whether a driver is high-, medium- or low-risk.

I completed online risk assessments from IAM RoadSmart, TTC Group, E-Training World, DriverMetrics and DriveTech to understand the methods by which each provider would assess my capabilities – and what factors would contribute to my overall results.

Some providers have developed their own software while others use third parties.

To deliver its latest product, IAM RoadSmart, for example, has partnered with licence checking provider Licence Bureau, which uses DriverMetrics software.

TTC uses DriverMetrics, Imagitech and RoadMarque systems within its assessments to give an “all-round approach”, according to Adrian Hide, senior consultant at TTC.

E-Training World is a third party provider and says 80% of its business comes through companies with branded systems.

These overlaps and alliances are worth fleet managers considering when choosing and implementing a driver risk assessment as one company may well provide the software for another – and vice versa.

Perhaps surprisingly, I was not given the same risk rating by all five providers – it was mixed bag of low, medium and high which made me even more keen to discover how my exposure to risk was calculated.

At a glance, my knowledge and attitude to driving let me down on all five assessments. This makes perfect sense as my driving background does not align with that of a seasoned fleet driver.

In 2011, on my third attempt, I successfully passed my driving test but have not been behind the wheel since. My licence may be clean, but my knowledge is rusty at best.

Back then, I passed my theory test with a score of 98% and, thanks to hours of practice, I was no stranger to hazard perception tests.

These were definitely reflected in my results. Of those providers that gave detailed breakdowns, I was scored as either low- or medium-risk for hazard perception.

The DVSA’s method (back in 2011) was to click for a potential hazard (like approaching a crossroads); again if a further hazard manifests itself (say, a vehicle approaches from either side); and a third time if that vehicle crosses in front unexpectedly. So up to three clicks for one hazard.

An example from my test was that when travelling in a built-up residential area, refuse bins at the tops of drives could indicate a refuse collection truck further ahead, so the test wanted me to click on the wheelie bins as a potential hazard, and then click again when the truck was in sight.

E-Training World used the standard click method of hazard perception testing and scored me as low risk.

I felt a strong sense of déjà vu when an introductory video of a fleet driver using the same dustbin example popped up, and I immediately knew how I was meant to conduct the test.

“We effectively take traditional methods of assessing drivers in vehicle and place them online. That is our business ethos in terms of how we assess drivers,” explains Jonathan Mosley, director of sales and training at E-Training World.

“It’s not the hazard itself, but pre-empting the hazard arriving. If a driver has ignored the signs then they have ignored the clues to the hazard.”

DriveTech gave three short video clips as part of the ‘anticipation section’ of the test which measured my awareness of my surroundings, looking at details and scanning for hazards. The results from this would fall under skill, for which I scored ‘medium’.

I was instructed to pause the clip and then click on a hazard. After each video I was also asked questions about what I had observed such as speed limits and road signs.

IAM RoadSmart also used video to test my awareness and concentration. Its videos featured an interactive and realistic in-car view, prompting me to check my rear view and door mirrors for potential hazards, just as I would when driving normally.

A click-based hazard perception was included, and I scored ‘low’ in this section.

TTC was the only provider to encompass hazard perception, observation, distance and reaction tests within its assessment and provided me with an overall score for each section.

It used the traditional method of hazard perception click-testing, and I gained an average score.

Adrian Hide explains that the videos all use built-in interactive algorithms that will calculate the result.

I scored above average for concentration, which within a set timeframe tested my ability to notice things such as road signs and colours of oncoming vehicles on still images.

To test my reactions, it gave me a flashcards exercise for which I was graded as average. I was given the shape of a blue diamond and had to click on the matching blue diamond whenever it appeared.

The main shape changed every few seconds, meaning I had to concentrate on the moving flashcards and on the shape I was matching.

The only provider to not include a hazard perception test was DriverMetrics as it uses a psychometric approach called the Driver Risk Index.

I scored low on the hazard awareness section, featuring likelihood-based questions prompting me to give an honest response to how I would react to a hazard.

An example reply would be, ‘I often tailgate drivers travelling under the speed limit’.

“Most risk assessments will focus on skill, hazard perception and knowledge, but they aren’t the reasons drivers crash,” says Richard Hill, commercial director at DriverMetrics.

“There is, therefore, a disconnect (with standard online risk assessments) between the assessment process and the resulting coaching.

"Our programme’s focus is on behavioural factors that are astringently linked to crash involvement, and they are the areas that research has validated.”

It is safe to say that while my ability to spot hazards may make me low risk, my lack of knowledge and understanding of the rules of the road contributed to my higher scores.

DriveTech, TTC and E-Training World all marked my knowledge section as high risk and this is a self-confessed shortcoming due to lack of experience.

According to Mosley, the majority of people are rated high risk on knowledge-based questions, which made me feel slightly reassured.

Contrary to the others, IAM RoadSmart scored my knowledge as low risk.

A contributing factor could be that I completed this risk assessment last and possibly knew some of the answers from previous tests.

Another factor is that this particular assessment asked a series of first aid questions, and I am a qualified first aider – so I would suggest that my low score perhaps is due to luck of the draw, rather than an inefficiency within the assessment.

It is worth noting that although IAM RoadSmart uses DriverMetrics software. It uses its Duty of Care product not the Driver Risk Index.

I had identified gaps within my own knowledge and the assessments have correctly pointed them out.

But I wanted to ascertain what factors, if any, automatically make drivers high risk – and, most importantly, which providers had these algorithms within their products.

DriveTech’s fleet training manager for its on-road academy, Keith Freeman, explains that its risk assessment is based on statistics.

“The key age risk areas are 17-25 and 55-plus,” Freeman says.

“Statistically, lower risk age brackets are between 30 and 50 years and we can only work on statistics. Other factors, based purely on statistical information include long periods driving in the dark and high annual mileage.”

TTC also has factors within its risk assessment that would automatically make a driver high risk, which once again include age and mileage.

“Driving on the road network is considered a dangerous pastime due to the multitude of people on the roads,” explains Hide.

“The longer you spend on the road network the more at risk you are – by default. Considerations will be linked to mileage, if we think your exposure to risk is related to the amount you drive.

“Age and gender will be a risk indicator, too. If you are a 19-24-year-old male, your risk factor for that age category would be greater than a female of equivalent age.

“Rural driving will make you higher risk as motorways are statistically safer. Certain vehicles could make you high risk, such as a high performance car.”

I was curious to know what statistics and information informed TTC’s algorithms as it sounded similar to insurance claims.

Hide confirmed that it pools questions and risk factors based on insurance, but that it also has access to knowledge and data from Transport for London and the Department for Transport.

Hide concludes: “Risk originates from many areas and no one particular area is a sufficient indicator in its own right to identify risk. We have to look at it as a whole.”

IAM RoadSmart, Driver Metrics and E-Training World – all of which marked me as low- to medium-risk – do not have automatic high risk scoring answers.

Hill says: “Most assessments work on that (automatic high-risk algorithm) basis, but not ours. It does not matter how many miles you drive, it is how you drive the miles.”

Costs are difficult to establish as some providers work on a volume-based quotation basis. TTC was reticent to quote a price as it usually works with large volumes of drivers.

E-Training World says that large groups will drive its prices down but it generally starts from £15 for the online assessment and a further £5 for each module.

DriverMetrics charges £17.50 for the assessment and six e-learning modules, while DriveTech gave an example of an online driver assessment with on-road training as £200 per driver but suggested there may be flexibility for large groups.

IAM RoadSmart’s yet to be named new product will be a fixed-fee driver risk, audit and compliance tracker starting at £4 per driver per month. I was given pre-launch access to it.

Each provider has a different way of assessing risk, and there is clearly not a universal solution to determine if a driver needs in-car training or further learning.

Instead, it is perhaps advisable for a fleet manager to decide what system best fits in with their overall driver risk management and what is appropriate for their drivers.

If a driver suffers from a lack of confidence, for example, then DriverMetrics could be considered to target the root cause of the risk.

However, if a driver simply falls short on knowledge or rules of the road then a more ‘traditional’ online assessment could be the best way forward.

Management training

Many of the leading providers offer management training for fleet decision-makers.

TTC offers a managers audit to benchmark a business against the HSE guidelines ‘Driving at work – managing work-related road safety’.

DriveTech offers line manager workshops designed to further encourage line managers to prioritise on-road safety over simply “getting the job done” – and breeding a safety culture within a company from the management down.

DriverMetrics has a range of courses aimed at all levels of management, including a dedicated two-day senior management workshop at Cranfield University, on how to implement a behavioural-based approach to risk management.

E-Training World does not offer a specific course for managers but, instead, explores the data and liaises closely with management on the right training, or any recommended training.

Online risk assessment results