What are we measuring with metrics?
Understanding Safety Metrics behind IEC 61508 and ISO 26262
As a consultancy we spend a fair amount of our time working with hardware metrics both for safety and reliability engineering. A few key questions crop up here:
- What is the objective of hardware metrics?
- What do the numbers tell us?
- What benefit do the metrics bring and what should we do with them?
Two standards where we are active on a regular basis have quite different objectives for hardware metrics. In this blog we look at the strategies used both in IEC 61508 and ISO 26262 when it comes to numbers behind safety.
Objective behind deriving hardware metrics
Utilized across different industries, hardware metrics rely on reliability data in the form of failure in time rates (FIT) which are taken as 10E-9 failures per operational hour. In the world of IEC 61508 this usage helps yield statistics for both reliability and for safety, while in standards such ISO 26262 the focus is purely on safety.
Such metrics have different objectives and a direct comparison between IEC 61508 and ISO 26262 is not really possible as we will see below.
The main question here is whether we are looking at a repairable system and, if so, how long the system will remain unavailable following a failure.This is very much the approach of IEC 61508 and similar related standards that are focussed on availability.
Taking architecture into account
In the calculations of metrics for both standards listed the architectural metric and the estimation of diagnostic coverage play a big part – we are ultimately focussed on the percentage of failures that can be detected by our safety mechanisms.
We multiply each FIT rate by the percentage diagnostic coverage of the safety mechanism to yield the FIT rates that are dangerous (in different categories) or safe:
In both cases the metric is similar, and provides a useful measure of the quality of the architecture.
What does PMHF actually tell us?
In ISO 26262 there is a second metric – probabilistic metric for random hardware failures (PMHF). Part 10 of ISO 26262 provides a good explanation of the use of PMHF and clarifies that it does not represent a failure rate. Part 5 of ISO 26262 (section 9.4.2.2) goes further to tell us that the PMHF has no absolute significance other than to enable a comparison of a new design with an existing one. This is always a difficult point with customers. People like to have a definitive statement with clear process to follow, and many customers find this point confusing.
Our focus with the PMHF metric is to estimate the probability of violating the safety goal over the operational lifetime of the item. The result is expressed as an average probability per hour and is not a failure rate.
The PMHF calculation in ISO 26262 provides guidance on how to estimate the instantaneous failure rate of violating the safety goal within the vehicle lifetime. This is a simplified approach, and as we look at IEC 61508, we can see how a more comprehensive estimate can be achieved.
Alastair Walker, Owner / Consultant
What’s dangerous and when can we detect it? PFDavg, PFH and Dangerous Failures in IEC 61508
The IEC 61508 approach uses two different quantities, depending on the demand placed on the specific safety function. PFDavg is the average probability of dangerous failure on demand and is used when low demands are placed on the safety function. PFH is the average frequency of dangerous failure and is used for higher demand rates of the safety function or continuous operation.
The resulting PFDavg or PFH value provides an indication of the safety integrity level (SIL) that can be claimed for the product.
IEC 61508 also divides failures into four groups: safe detected, safe undetected, dangerous detected and dangerous undetected. For this discussion, we are mainly concerned with the two dangerous failure groups.
A dangerous undetected failure is a failure that is not covered by our diagnostic coverage. A dangerous detected failure, on the other hand, is detected by the diagnostics, but can still become a concern if there is not sufficient time to repair the system.
If the demand rate for the safety function is low (less than once a year), it is likely that a dangerous detected failure can be repaired before the next demand on the safety function. In this case, the dangerous detected failure can be disregarded from the calculation.
The complexity of the PFD calculation depends on several factors, including the redundancy strategy (e.g. 1 out of 2, 2 out of 3), whether the device is repairable, and the relationship of common cause failures.
The IEC 61508 approach ultimately provides a comprehensive summary of the probability of failure on demand or, in the case of PFH, the average frequency of dangerous failure.
Conclusion
Whichever measures we use as a yardstick for the quality of our system, and more specifically the hardware, we need to be clear about the context and what we are trying to estimate. Whether the device is repairable or not is one key question (and not a consideration for ISO 26262). The quality of the diagnostics in the architecture is another key point.
Ultimately, the more complex the system, the higher the failure rate will be. However, the level of diagnostic coverage also needs to be considered when determining whether the resulting metric is acceptable.
We do not have apples-to-apples comparisons across industries and standards. Ultimately, architectural metrics provide useful data on the quality of the design, while PFD or PFH gives us a comprehensive figure for failure in IEC 61508. ISO 26262 PMHF provides more of a benchmark for comparing different items and is less useful for defining a quantitative figure for the overall failure rate.
By Alastair Walker, Owner / Consultant
