SOTIF & AI – Rise of the Little Brother

On 20^th of January 2021 the voting for the new version of the ISO/DIS 21448 has started. It is the evolution of the ISO PAS 21448.

Till now mainly the ISO 26262 was the big challenge. The focus lies in identifying all hazards by systematic or random failures and mitigating them. The functionality itself was secondary. 

AI in the spotlight

But with the increasing importance of Artificial Intelligence (AI) caused by the “Automation levels” the functionality has stepped into the spotlight. AI is not deterministic anymore. It is not the sequential “This happens so therefore that will happen”. Especially for level 4 (high driving automation) and level 5 (full driving automation) AI replaces the driver completely.

Therefore, the definition of the behaviour of the system has become essential. 

So, taking in mind, that AI might be unpredictable, why should someone use AI at all? 

The challenge is to set up a system that can handle all kinds of situations. And with a deterministic system it is not possible to construct every situation. Furthermore, the system needs to be adaptive.  It should not matter whether the person crossing the road in front of the (autonomous) vehicle is wearing a blue jacket or red trousers. The important part is “there is a person that might be run over”.

Therefore, AI is one promising option. For the given task it is possible to train the system to recognize a person. Given a fitting training set all possible sets of persons are trained. Kids, adults, women, people in wheelchairs – all are classified as “worthy of protection” by the system, simply by being identified as “person”.

Intended Functionality

Now, everything seems to be fine. The (driverless) vehicle is trained, notices every person and every hazard is eliminated. Doesn’t it? As always the devil is in the detail. First, the training is essential for the AI. If you don’t train the correct data, the AI cannot react properly. This is commonly referenced as “garbage in/garbage out”. And secondly, what exactly do you expect from your system? What function is intended after all? How should the system react? Is there an abuse possible? Many questions for an easy task like “do not run over people”.

In the current version of the SOTIF standard listed in chapter 8 are listed the first restrictions and situations where a living person has to take over. This leads to the conclusion that to the according standard level 4 and level 5 of automated driving is not possible at all.

So, is there any hope left for automated driving or self-driving cars? I think, there is. It is crucial to define the reactions or better said the functionality clearly. Therefore, much more energy and brainpower need to be put into the definition of it. And here ISO 21448 comes into play. Or easier: know your system and its intended functionality.

By Gerrit Steinöcker, Functional Safety Consultant