Friday, 23 May 2008

Design for Testability


I have been reading up on "design for testability" because some of the code I deal with is not always easy to test. Some thoughts ....


“Business” wants software application to help them do their job. They use them because they bring “value” to them. In order to make that happen software application must do what they are required to do (fit for purpose) and must be “reliable”.

Through a mixture of process guidance, technology and human skills, to use the previous to elements efficiently and effectively, IT tries to deliver those application on time while adhering to the intrinsic quality attributes like for example reliability. Quality processes are put in place to ensure achievement. In order to assess the if the quality attributes are indeed incorporated into the software product, we need to check it. This is where testing comes in. Testing should give us insight into the quality of a system. This insight should help us decide to release the software for usage by the business because it can bring value to the business. Or it can tell us that there are still some discrepancies and we need to improve quality before handing it over to the business.

Testing is some something we must do although it does not immediately raises the quality of the system. It reports only. Many times “business” can not understand why we can’t deliver something right the first time. Testing costs money! Software creation is still an error-prone endeavour for many reasons despite all advancement in processes , technology and skills of people. In other words we start off with the assumption (certainty!) that software contains errors. Both functional and programming wise. Testing is about finding those. So it is important that we make the best possible test cases to reveal them. Use tooling to support our testing effort etc. Bust beside the test process it self , we can do more.

During the inception and construction of software we can introduce measures to make this testing effort even more efficient and effective. In other words we must make our system as “testable” as possible in order to and to facilitate performing the test and finding the errors.


Testability is about the ease of performing test against your system and ability of the tests to reveal any defects in a certain context. The context of the test , system testing (functional and non-functional) or developer testing, puts different meanings to testability. Also is it important that other software artefacts beside code, affect testability ; for example the requirement specification documents.

These are some aspects that influence the testability :

  • Understanding: The better you know what the thing is supposed to do , the better you can test for it.
  • Controlling ; You must be able to manipulate the input , in order to verify the correctness of the output.
  • Operational : We can only test it when it works !
  • Visibility : You can only verify the result , if you can see the output.
  • Simplicity : The less there is to test , the more quickly we can test it
  • Stability : The fewer the changes to less test if need to re-do and/or re-write

So manipulating the influencers, can change the testatbility !

It is important to acknowledge that testability does not come on its own. We must take testing into account from the beginning of our software development endeavour. Because we know our software will contain errors (this will only increase with the size and complexity of today’s systems), we must prepare for it to easily test our system and find them before we ship the code.

In other words we must design for testability. It is important because testing costs money. Also faulty software systems can cost the business money (in the best case or worse in case of threat to safety or life) . Therefore we must make the testing as effective and efficient as possible. But Business will benefit from this as well in the long run because we should be able to find defects thus raise the quality of the system, thus raise the value to the business.

Also future changes to the software to incorporate new requirements will benefit from this testability feature because we should be more confident to change working code with regression testing. Also external factors such as SOX compliance make it important for the business to assemble test evidence. Testable systems certainly can aid .

Design For testability

To increase both the ease and value of testing we can tack different areas of the software development process and the software product itself. For example

  • The software development methodology (“testable” requirement concept, etc)
  • The software technology (Procedural vs. Object-Oriented, Mainframe, etc )
  • The software architecture & design (Layered vs. monolithic, Coupling , Encapsulation , etc)
  • The code (Assertions , mocks/stubs , etc )
  • Tool support (Automation , Xunit framework , etx)
  • The developer (Training and education , code reviews , etc)

This broad spectrum calls for an holistic approach of course. The test context will put certain focus in that approach. For example

  • For functional testing you need good test reference in form requirements documents to establish your test cases
  • For System testing , you must be able to let system work under operational condition together with data and sometimes external systems.
  • For developer testing , you must sometimes be able test your unit in isolation and sometimes you would like to integrate a set of unit together.

Currently I'm interested in the developer testing context . So if you have experiences in raising the testability of your code and wish to share them , please do so. I would to hear from you how dependency injection help you ( or not) , how unit testing helped you find defects easily , etc.

Best regards,


Tuesday, 13 May 2008

Application taxonomies

A classification of software application can help you to think about an architecture and/or determine a reference architecture in your catalogue.
Multiple characteristics can be used to set up a taxonomy. These taxonomies can help you I think because it sets up a context for making a software architecture and the usage of reference architectures. Usually one classification criterion alone doesn’t fit the entire application . Multiple axes.

I think one of the main message would be : Don’t be fooled that one reference architecture can be used in all scenarios.

  • Front-end technology
    o Windows (classic / touch screen)
    o Web-based (classic / Rich Internet client )
  • Distribution
    o Desk-top
    o Mobile/PDA
    o Sever
    o Distributed
  • Connected
    o On-line
    o Disconnected with on-line capabilities (smart client)
  • Communication
    o Asynchronous
    o Synchronous
  • Target
    o Consumer ( E-commerce, PDA)
    o Business (Traditional administrative data keeping or/and decision support)
    o Industry (MES, CIM, MRP, SCADA, real-time systems, PLC, Robotics)
    o Government E-government
    o Entertainment (Games)
  • Connectivity
    o Internet
    o LAN
    o WAN
    o VPN
  • Function
    o OLTP applications
    o Reporting applications ,Data warehouse, BI
    o Batch application
    o Service–oriented apps
    o Mobile device applications
    o Integration Applications ( ERP, Mainframe , etc)
    o Office applications and Office integration
    o Portals
    o System software
  • Organisation boundaries
    o Departmental
    o organisation
  • Usage pattern
    o Collaboration
    o Transactional, data keeping
    o Decision support
  • Time pattern
    o Long running “transactions” (Workflow)
    o Short running business transactions
  • Expected life-time
    o Short-term
    o Long term
  • Business area
    o Medical
    o Financial
    o Government
    o Industry
    o Commerce
    o Etc
  • Saas (Software as a Service ) software.
  • Etc.

What do you think?

Best regards,


Software architecture decisions


I recently had to give a definition of what a software architecture is . Not always that easy because Software Architecture is a much loaded term. If you want some formal definitions what a software architecture is , I recommend reading the information on

General tone in these definitions is that you need to make high-level decisions about the system you' re going to build;
  • What style are you going to use? What is the structure?
  • How is it going to function? How do structural components of the architecture work together?
  • How does it meet the needs of all the stakeholders?

These high-level decisions have a high impact on the application. You can not change them easily if wanted or needed. Unless you’re prepared to put enough resources on the table.

In the bullets below I tried to sum up the different aspects of a software architecture. This list is not intended to be exhaustive.

  • Windows front-end or web front-end ( Rich Internet clients)
  • Layering, organization of code, separation of concerns,
  • Representing “business objects” persisted in a database
  • Exposed domain model or using Data transfer Objects / Presentation Model
  • DataSets versus classic classes
  • Organization of “back-end” logic
  • Organization of “Presentation” logic
  • Organization of “data-access” logic
  • Distribution or not
  • Remoting technology
  • Authentication, authorization, Audit
  • Data binding
  • Synchronous / asynchronous
  • ORM tools or raw ADO.NET
  • Multi-language
  • Exception handling versus status code
  • Instrumentation /Logging
  • Configuration
  • Caching
  • Database schema fixed or freely adaptable to need of design designs
  • Long-running “transactions” (conversations)
  • Reporting needs
  • Validation rules placement (Attribute based, Intra object based, Inter object based)
  • Etc.

It is important to take holistic approach towards the desicions. You will need to take some trade-offs and influencing criteria into account.

  • Kind of application built (taxonomy)
  • Tooling can constrain how you organize your application (code generators, frameworks, application building blocks, software factories ,etc)
  • Availability/knowledge of people can constrain the “feasibility” of your architecture
  • Process (top-down versus bottom up, behaviour-centric versus data-centric)
  • External / internal regulation can influence your architecture (SOX, internal Audit rules
  • Availability of reference architecture.
  • Non-functional requirements
  • Patterns
  • Time-to-Market
  • etc.

If I missing some points or you want to comment on this. Please feel free.

Best regards,