This last week has reminded me how important it is to do proper, thorough testing as part of your regular development plan. What I’m getting at here, is the kind of testing that checks an application comprehensively in all states. Here are two great examples of why you need to do more than just a couple of unit tests around the specific functionality you’ve just implemented:
Example 1. In the application we’re building at the moment, we finally got rid of all compilation warnings and one of the developers decided to turn on "Treat compilation warnings as errors" so that, moving forward, our code would be really clean from that point of view.
He tested this by deleting all of his source, getting a fresh copy from SourceSafe and doing a complete rebuild of the solution. All went well so he checked it in. Other developers through the course of the day then retrieved this change and worked with it, none having a problem.
3pm came around and it was time for the daily build into the Release configuration. It was only at that point we discovered that there were indeed more warnings but that they were being suppressed in the Debug configuration (the specific warning was ensuring that you had XML comments for every public member). Because the project had been inherited from someone else a year ago, no one realised that the warnings were being suppressed and we had been blithely continuing on for twelve months.
Admittedly, it didn’t take a great deal to fix for the Release build – we quickly changed the option back to treat warnings as, well, warnings and moved on. But we then had to deal with all the new warnings in the Debug build as well.
Moral: You must test your Release build process as well as your Debug build.
Example 2: We’re building a web application and at times need to resort to the FindControl method to find and deal with specific controls on a page. A lot of our code makes sure that the control that was being sought exists too (ie. not null), so that we don’t try to do actions if the control wasn’t found. I implemented several of these kinds of logic functions early last week. Later on in the week, someone else worked on the same page and changed the page in a way that in certain page modes the control didn’t exist. He tested his changes thoroughly but neglected to double-check every function in the page. The changes were committed into the build process and the testers began to QA.
However, all was not well and soon enough came the discovery that certain functionality was no longer working as expected. After debugging the application the control changes were determined to be the culprit and again we fixed it fairly easily (when the page was in edit mode, the control to use was different).
Moral: You must test the extended functionality of the area you’re working on, not just the bit you specifically changed.