1.1 Why do we need to qualify the tools?
Given the complexity of the ECUs that we are presently working on, the development of an item will not be possible without the use of a variety of software tools. The software could be something that is used by the systems team to capture the requirements, a software used by the hardware team to create the PCB and so on. The key question that the tool qualification tries to address is: What happens if the SW tool that I am relying on to develop the item, malfunctions? Will the malfunction of the tool lead to a safety goal violation? A SW tool qualification is thus a systematic analysis of each of the SW tools used in development of the item and ensuring that additional steps are followed, in case the tool malfunction could lead to safety goal violation.
1.2 Which tools should be qualified?
At a broad level (With a pinch of oversimplification!), consider the qualification for 2 categories of tools:
- Any tool that is used for generating some kind of output that feeds as an input to a subsequent step in the development (Ex: An automatic code generator tool like MATLAB, a compiler etc.)
- Any tool that is used for verification or validation of the outputs generated (Ex: A testing tool like MATELO, a simulator of the vehicle environment like CANoe etc.)
1.2.1 Exceptions
One of the most frequently asked question is on whether we need to qualify tools like MS word, MS excel, the PC environment that we are using etc. Unfortunately, the answer is not so straightforward. Our thought would be that we need to still go back to the 2 broad categories that have been outlined above and see whether it fits into one of these categories. In the exceptional scenario that the tool doesn’t fall into any of these categories, we need to go back to our fundamentals namely:
Does the malfunction of this tool cause a safety goal violation? For example, a software tool that is used for flashing ECUs at the manufacturing plant could malfunction & potentially cause a safety goal violation by corrupting the flashed software. In such a scenario, this tool needs to be considered for tools qualification.
Do we have any mitigation/detection activities already in place in our process? The mitigation action could be as simple as a manual review of the tool output or a process which automatically checks the correctness of the output. Continuing on the example from the previous point, if there is a step in the manufacturing process that checks whether the SW is correctly flashed or not into the ECU by taking a SW dump, then this detection mechanism could be used in the TD step (Details about TD are given below).
1.3 How should the qualification be done?
Outlined below is a systematic yet simple way to perform the actual tool qualification for your projects. Even though the tools qualification seems to be a daunting task, it could be easily simplified by using the below outlined steps:
- Prepare a list of all the tools that are used in the program along with its version number, type of the tool (Development or verification/validation. Refer previous section). Ensure that in-house tools are considered as well while preparing the tools list.
- List down the inputs & outputs of each of the tools. These are essential in deriving the use cases of the tool.
- Derive the various scenarios in which the tool is going to be used. From these use cases, only consider the ones in which the erroneous functioning of the tool leads to safety goal violations. For example, a tool could be used in 5 different use cases and yet only one of them could be a safety relevant use case. The existence of even 1 use case that could potentially lead to a safety goal violation means that the tool has to be rated safety relevant and further analysis needs to be performed.
- Once we have the use case(s) for the tool, use the matrix given below to arrive at the TI & TD values.
- TI1 means that the tool malfunction does not cause any safety goal violations while a TI2 means that the tool malfunction leads to a safety goal violation. TD1 means that we have high confidence in the tool to prevent/detect any possible errors, TD2 & TD3 means medium and low confidence.
- With the TI & TD values, derive the TCL for each of tool use case(s). Repeat the determination of TCL for all of the use cases of a tool & as well for all the tools that are utilized in the system development.
- In case of a tool with multiple use cases, the highest TCL should be considered as the TCL for the tool (Ex: if use case #1 gets TCL2 & use case #2 gets TCL3, the tool TCL is TCL3).
- The last (But surely not the least) step in the process is the determination of the kind of qualification step that should be utilized for the tool. The diagram below explains the mechanism that needs to be followed:
Even though the standard defines these 4 methods for qualifying any tool that are classified as TCL2 or TCL3, there exists an alternative approach. The alternate approach that could be taken is to identify the existing detection mechanisms (TD) for the tool & identify whether any additional detection mechanisms could be added for the tool. The critical idea behind this is to move the tool from TD2 or TD3 to a TD1, thus eliminating the need for qualification activity itself.
A word of caution: This methodology that we have stated above is not given in the standard and should be carefully considered before utilizing it in the safety development.
1.4 Who should do the qualification?
Even though the standard does not explicitly talk about the ‘Who’ part, our thought is that the skills teams should do the tools qualification with help from the safety team and the review should be completed by a safety engineer. The rationale for this approach is that the skills team know all the possible use cases of the tool and hence they are in the best position to report and classify the tools they are using. A safety engineer will be able to provide direction regarding TI & TD and be able to help with qualification activities in case that is required for any of the tools.
1.5 Conclusion
To wrap it up, we have seen the reason for performing a qualification, the tools that should be considered for qualification, the methodology to be followed and who should be involved in performing tool qualification. With this increased knowledge, our hope is that you will be better off when handling the tool qualification activity in your programs.