MP Associates, Inc.

THURSDAY March 01, 2:00pm - 3:30pm | Sierra

Using Mutation Coverage For Advanced Bug Hunting

Nicolae Tusinschi - OneSpin Solutions GmbH
Vladislav Palfy - OneSpin Solutions GmbH
Tom Anderson - OneSpin Solutions GmbH
Modern verification methodologies incorporate multiple coverage solutions. These range from functional to structural coverage, leverage various coverage models and operate using varied technologies in both the simulation and formal process. The main purpose of these coverage solutions is to establish a signoff metric that indicates when enough verification has been performed. However, as coverage approaches have evolved, new use models have emerged for these tools that increases their value in the verification process. The ability of these tools to provide guidance to areas insufficiently tested, or uncover buggy scenarios is still being explored, to great effect. Mutation coverage is a relatively new technique pioneered by formal and simulation providers. The approach is to pose the question: if my design is fully covered by a specific testbench or assertion set, then if I change something in the design, the tests should detect this change. If it doesn’t then the area of change is not covered. Now if all the code segments across a block are changed, one by one, a very precise coverage metric may be established. Of course, how the design is changed, or mutated, and how these mutations are implemented has a dramatic affect on the effectiveness and performance of the solution, but the approach is widely regarded as a highly effective method for establishing both structural and functional coverage, and is used in design flows where high degrees of coverage are mandatory. Leveraging this mutation technique with formal tools provides some significant technology benefits, which accelerates the operation of the tool. In addition, the exhaustive nature of formal further improves the precision of the established coverage metric and allows for useful information on the source of uncovered scenarios to be obtained. The performance and informative nature of formal-based mutation coverage has opened up a new use model in the area of bug hunting, where the tools may be used to analyze complex operational scenarios and detect extremely well hidden error conditions, otherwise hard-to-find using more traditional solutions. This tutorial will focus on this bug hunting process. Leveraging a classic case study notorious for complex bugs, the attendees will be guided through a range of techniques applicable to many design applications, and shown how unusual bugs can manifest themselves and be detected using this technique. The workshop agenda will consist of: Introduction to mutation coverage techniques Mutation coverage with formal – it’s not so tough! Sign-off with mutation coverage Overview of bug hunting techniques with formal Bug hunting using mutation coverage Bug hunting on a practical design example Overview of potential improvements using this technique

Thank You to Our Sponsor: