Testing. It’s the most important part of engineering, whether its software or hardware. Yet all too often, it ends up getting sidestepped to get product out the door. It is woefully under-estimated in budgets and schedules. Then once the schedule starts slipping, engineers have trouble fighting the urge to skimp on testing to get things “done” faster. Most frustrating of all is the pressure from management to cut down testing hours in project plan estimates, because it “costs too much”, “takes too long”, or “doesn’t add enough value.”
By day, I’m an FPGA developer and write mostly VHDL. Despite what some software developers might think about FPGA developers, we do quite a bit of unit testing, albeit without a formal framework. And that’s the problem. As schedules start slipping, the stress and anxiety of missing a deadline cause testing rigor to decrease. Ironically, if the project had a formal unit test suite that could be relied upon, stress and anxiety would be lessened because engineers wouldn’t have to fear breaking something in the frantic push to ship on time. They could simply launch their regression tests and review the results.
The most difficult part with engineering that takes place inside a text editor is that once our creations “live” in the physical world, they become a lot more difficult to test and diagnose problems.
Suppose you’re an analog engineer designing a bandpass filter using discrete components. You just got the manufactured boards back. How do you know the filter works? Do you just look at it and say “well, it looks like a bandpass filter, so it probably works.” Of course not. You connect a signal generator up to the input, place a spectrum analyzer on the output, and you sweep across input frequencies and measure the output signal to confirm your filter has the correct attenuation for out-of-band frequencies, etc. If you were to discover a problem, you’d grab a Fluke and check to confirm that the passive components have the proper quantities. This is in essence, a unit test.
Test suites written with frameworks like JUnit give you that warm, fuzzy feeling that all of the little pieces that make up your design are working correctly. This reduces the stress on your engineers, which I believe has a strong, direct correlation with product quality.
VHDL desperately needs an open-source unit testing framework modeled after JUnit. In general, FPGA and ASIC development methodologies have severely lagged innovations in the software world. Kent Beck, the father of Test Driven Development, wrote his landmark Simple Smalltalk Testing: With Patterns paper in 1989. It’s now 21 years later, and there is no vhdlUnit.
So, who wants to get started?