Unit tests (Part 2)

Working definition

Unit testing is a procedure used to validate if “units” of source code are working properly. More technically one should consider that a unit is the smallest testable part of an application. In an Object Oriented program, the smallest unit is a Class or more interesting, its operations.

Unit testing is done by the developers and not by end-users or separate QA. A unit test is a piece of code that calls the unit under test in particular way in order to find errors during development. Each Unit test verifies if it returns the predicted answer.

Automation is key in the success of unit testing because it gives the developers a way to create structured unit tests that can be executed easily and repeatable and can be verified through assertions in the test code.

Benefits

The goal of unit testing is show that the individual parts of the code are working correctly. In doing unit testing you gain several benefits

• Raise level QA mindset of the developer
• Raise confidence in the quality of the code
• Tests are run repeatedly
• Early warning detection system for programming and/or design errors or flaws
• Living documentation on the API of the “units”
• Aid for regression testing
• Code refactoring is more stimulated
  
  
  

When writing unit tests you should think like a tester, not just a developer. The time you take to design your unit tests will help reduce the time spent resolving defects later. Focus on the details of your objects: How is data transferred between them? Who consumes them? How easy can you break the object? What happens if I "do this"?

When a developer is programming some functionality he usually writes in some way or another some testing code to see whether his implementation does what is supposed to do and if it doesn’t produce errors. The developer then moves on to program the next piece of functionality. Again he assures that run as expected. Maybe the developer moves on other functionalities without coming back to test previous piece of code until release data. With automated unit testing, the developer has a method so the test can be run repeatedly as the code is developed.

When you have a suite of unit test that effectively test your code, you can be confident that your code will have a low likelihood of errors. The confidence in the quality of the code can be increased.

The act of writing tests often uncovers design or implementation problems. The unit tests serve as the first users of your system and will frequently identify design issues or functionality that is lacking.

Once a unit test is written, it serves as a form of documentation for the use of the target system. Other developers can look to unit tests to see example calls into various classes and members.

Perhaps one of the most important benefits is that a well-written test suite provides the original developer with the freedom to pass the system off to other developers for maintenance and further enhancement. Should those developers introduce a bug in the original functionality, there is a strong likelihood that those unit tests will detect that failure and help diagnose the issue. Meanwhile, the original developer can focus on current tasks.

It takes the typical developer time and practice to become comfortable with unit testing. Once a developer has been saved enough time by unit tests, he or she will latch on to them as an indispensable part of the development process.

Unit testing does require more explicit coding, but this cost will be recovered, and typically exceeded, when you spend much less time debugging your application. In addition, some of this cost is typically already hidden in the form of test console- or Windows-based applications. Unlike these informal testing applications, which are frequently discarded after initial verification, unit tests become a permanent part of the project, run each time a change is made to help ensure that the system still functions as expected. Tests are stored in source control very near to the code they verify and are maintained along with the code under test, making it easier to keep them synchronized.

Unit tests are an essential element of regression testing. Regression testing involves retesting a piece of software after new features have been added to make sure that new bugs are not introduced. Regression testing also provides an essential quality check when you introduce bug fixes in your product.

It is difficult to overstate the importance of comprehensive unit test suites. They enable a developer to hand off a system to other developers with confidence that any changes they make should not introduce undetected side effects. However, because unit testing only provides one view of a system's behavior, no amount of unit testing should ever replace integration, acceptance, and load testing.

In the same line but when refactoring code (changing code without changing the intend) to improve design in terms of maintainability, performance, etc. are more stimulated if you have a battery of unit tests. You’re more confident when you refactor some code. You can easily check if the code change is done correctly.

Automated unit testing should help reduce the amount of time you spend in the debugger. However, if testing results and code coverage do not help in providing the reason why your test is failing, don't be afraid to debug your unit tests. In Visual Studio 2005 Team System, developers can debug their unit testing assemblies using the Debug Selected tests option in Test Manager. You can also debug Nunit test.

Concerns

Unit testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors (except when you consider a component or subsystem as a unit) , performance problems or any other system-wide issues. In addition, it may not be easy to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.

It is unrealistic to test all possible input combinations for any non-trivial piece of software. Like all forms of software testing, unit tests can only show the presence of errors; it cannot show the absence of errors. Writing unit tests for every single class to test every aspect of every function is a tall order. Even with a large number of unit tests, is complete testing possible (or desirable)?

What if there are bugs in the unit tests? Where there is code, there will be bugs. You’ll be getting false positives.

Not all code can be unit tested (at least not easily). It could, of course, be argued that in this case, the code should be simplified until it can be unit tested.
Passing a Unit test only means that no the test it self did not catch any problem. All the assertions were correct. This doesn’t mean that there are no bugs. The success depends on the quality of the unit test as well.

Automation

Unit testing in spirit has existed since the beginning of programming. Every environment, development shop or even programmer had/has its way of conducting these kinds of tests. For example creating a form with 20 –odd buttons calling various part of your code base.
This “Form with buttons where each button performs a functionality of the code to be tested” –approach work s but you need a manual operation for each single test: clicking the button. The main problem is that you need to have the discipline to click on very button after very change. Secondly, it is very easy to drag a button on a form and hook and eventhandler leaving the name of the button to its default. If you revisit the form a after some time, chances are you don’t remember to meaning behind the button. It will be even more difficult for someone else interpreting the test behind all the buttons. Thirdly, knowing if the test succeeded or not , you had to make assertions in your code , or make the assertion visually but also you needed a way to give feedback to the developer ( eg infamous msgbox).

Third-party vendors of course offer solutions to the testing problems but they are usually very expensive and introduce their own, proprietary scripting language to conducts test.

The xUnit framework was introduced as a core concept of eXtreme Programming in 1998. It introduced an efficient mechanism to help developers add structured, efficient, automated unit testing into their normal development activities. This Toolkit lets you develop tests using the same language and IDE that they are using to develop the application. The xUnit framework defines concept of "test runner" as the application responsible for (a) executing unit tests, and (b) reporting on the test results. Automated unit tests are based on "assertions," which can be defined as "the truth, or what you believe to be the truth." From a logic standpoint, consider this statement, "when I do {x}, I expect {y} as a result."

The atomic parts of a unit test fixture are the test methods that make assertions about the behavior of some (or all) public members of the unit you're testing. A test suite is a library assembly that contains one or more such test fixtures, along with optional setup and teardown methods. You normally run this assembly from a GUI app, but these unit tests can run also from a command-line and be activated from other applications (build process).

With the help of this testing automation framework your unit tests are

• Test can be written in same language as the tested unit
• Structured.
• Self-documenting.
• Automatic and repeatable.
• Designed to test positive and negative actions.

Open questions

• Testing should become an integral part of the programming effort , not some things that comes afterwards. But how can you educate your team? How can you convince management
• How to ensure that having a lot of unit tests does in fact make tracking bugs easier?
• How to make sure your tests are easy to maintain?
• How to write readable unit tests

References and recommend reading

• Write Maintainable Unit Tests That Will Save You Time And Tears, Roy Osherove http://msdn.microsoft.com/msdnmag/issues/06/01/UnitTesting/
• Qualities of a Good Unit Test http://codebetter.com/blogs/jeremy.miller/archive/2005/07/20/129552.aspx
• Coder to Developer: Tools and Strategies for Delivering Your Software , Mike Gunderloy , ISBN:078214327X
• Pro Visual Studio 2005 Team System ,Jeff Levinson and David Nelson , ISBN:1590594606
• .NET 2.0 for Delphi Programmers ,Jon Shemitz , ISBN:1590593863
• Extreme Programming Refactored: The Case Against XP Matt Stephens and Doug Rosenberg ISBN:1590590961
• Professional Visual Studio 2005 Team System ,Jean-Luc David et al. ISBN:0764584367
• Code Complete: A Practical Handbook of Software Construction ,Steve McConnell , ISBN:1556154844
• Test-Driven Development in Microsoft .NET, James W. Newkirk and Alexei A. Vorontsov ,ISBN:0735619484
• Pragmatic Unit Testing in C# with NUnit: The Pragmatic Starter Kit, Volume II (Volume Set) , Andy Hunt and Dave Thomas ,ISBN:0974514020
• Why Agile Software Development Techniques Work: Improved Feedback: http://www.ambysoft.com/essays/whyAgileWorksFeedback.html
• http://en.wikipedia.org/wiki/Unit_testing
• The Humble Dialog Box (http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf )
• NUnit (http://www.nunit.org/ )
• http://martinfowler.com/eaaDev/uiArchs.html