I often joke with customers that, although I realize they have to work on large designs, I do my best work on designs with just 2 or 3 instances. That's because I'm often trying to replicate an issue they've observed on their design and I'm attempting to reproduce that behavior in a smaller circuit.
I've found tiny testcases to be extremely efficient ways to gain quick clarity on tool behaviors which can then be more effectively applied to the real design. But it's not just having a tiny testcase that's most useful. It's the act of creating the small testcase where most insight is gained.
But it's not always easy to create a tiny testcase. Do you know how to write a syntactically correct gate-level Verilog netlist from scratch with a text editor? Do you know how to contrive complicated timing scenarios? Do you know to modify .lib/LEFs to replicate the things likely present in a design in a smaller setting? These are non-trivial things for someone who works in many different aspects of a design flow as designers are tasked with understanding an ever-increasing breadth of tools.
I recall a quote from a colleague of mine, Thad McCracken (interview here), a few years ago. We were working on a benchmark and were having a hard time reproducing some of the issues we were observing in a small testcase. He said "If we can't reproduce it in a small testcase it tells me we fundamentally don't understand the problem well enough."
Isn't that the truth of it? With the exception of run-time/memory related issues, nearly every issue we run into can be replicated in a tiny representative testcase -- but doing so can be very difficult.
Customers often joke with me that the oldest tricks in the EDA Applications Engineering books are (1) to ask if you're using the latest tool version and (2) ask for a testcase. Sure it's easy to get a self-contained testcase with Encounter (see the link for saveTestcase at the end of this post). But a better question to ask is, "Do you have any hunches on the likely circumstances causing the issue?"
Tiny testcases can be powerful in the user community as well. Some of the most disciplined and effective project teams and CAD groups I've worked with relentlessly stress the tool at the onset of a project in ways they're going to need it to perform during crunch time.
Sure, some things only come to light in the context of real designs. But creating tiny testcases can efficiently flush out the gaps between project requirements and tool requirements to give everyone a chance to find ways to get the software to do what it needs to do to meet project requirements.
In our busy schedules we often feel like we don't have time to create small testcases to triage a situation. And indeed, sometimes we just need to send a testcase in to R&D for resolution. But next time you run into an issue I'd encourage you to consider whether a tiny testcase might shed more light on the situation.
To borrow a line from Paul Cunningham: "Sometimes we need to slow down to speed up."
Bob Dwyer
Related Reading: