How Many Tools Does It Take to Verify Your Software?

This reminds me of the joke asking, “How many engineers does it take to change a light bulb?”

Many of our customers, especially those in the automotive industry, have used more than one static analysis tool as part of their software development and verification process.

One reason for the use of multiple tools is that, traditionally, the adoption of static analysis was fragmented into different activities such as coding rule compliance, bug finding, and so on. The development organization may have adopted a lint tool to perform local bug finding and a rule-checking tool to verify compliance to standards such as MISRA, while the quality assurance department may have adopted tools for code metrics such as code coverage, comment density, and cyclomatic complexity.

It is important to understand how a development organization uses its tools in order to determine if those tools add value to the process. The use of multiple tools, each with its unique user interface, certainly adds complexity to the process. For example, multiple tools may create inefficiencies in terms of:

  • Licensing and maintenance costs for multiple tools: These are nontrivial costs, especially for those with complex licensing structures.
  • The resources needed to integrate tools from different vendors, including custom scripts that need to be written to import or export data from one tool to the other and different interfaces and APIs.
  • Maintenance to confirm that these tools continue to integrate and work together, such as ensuring software compatibility between different releases.
  • The need to learn multiple tools.
  • The higher tool certification or qualification effort required by functional safety standards.

At MathWorks, we strive to continuously evolve Polyspace® into a more complete solution, helping to minimize the number of tools you must use. Polyspace is an integrated environment that can help you at various stages of an embedded software development process. The following example workflow illustrates some of the benefits of such an integrated environment. You can: 

  • Find and fix different defects right within your IDE, and justify any left over issues by annotating the code.
  • Check for coding rule violations or coding standards while simultaneously finding bugs, and provide justification for deviations from standards such as MISRA.
  • Use these results as part of your code review – control flow and data flow to support discussions about addressing the key violations.
  • Ensure the absence of errors for critical modules. Reduce unnecessary robustness tests by eliminating what has already proven by Polyspace (e.g., Proof that pointer arithmetic does not cause a run time error).
  • Check for integration issues – launching Polyspace as part of the build process, automatically import the justifications, and highlight only the newly-found integration issues.
  • Generate artifacts for DO-178C, ISO 26262, and IEC 62304, and document all the artifacts in one report.
Annotating a defect to review later as part of the code review and testing activity
Annotating a defect to review later as part of the code review and testing activity.

In addition, Polyspace products support C/C++ and Ada programming languages. These are the most popular languages that are used in critical embedded systems. A single static analysis tool that spans both these languages improves the efficiency of the development organization (i.e., you are able to benefit from using the same tool across different projects and groups organization-wide).