Application Development Chronicles

VISUG Event : Entity Framework & WCF

2008-12-22T22:24:00.005+01:00

Hello,

Kurt Claeys gave a presentation about using the Entity Framework (EF) in a distributed scenario (Service oriented with WCF, Tight coupled Client-server) for the Visug , the Belgian Visual Studio user group.

His presentation was based on his personal research into the subject so he didn't gave immediately all the answers. Instead he showed us via concrete examples which difficulties he encountered during his endeavor.

The reason for going N-tier and the way to achieve this were out of scope for the session but decoupling was the key concept he used to convey this requirement.

After a very quick introduction into WCF and the EF, Kurt showed us the challenges you are confronted with if going this route and how WCF, EF or something else could provide a solution: Serialization of object graphs, contract sharing , … But the biggest hurdle in the current implementation seems to be change tracking.

In EF, a special mechanism called the ObjectContext, keeps track of all the things you do with the entity instances you retrieve through the EF infrastructure. This means the entity identification , the relationships ….and also the changes you make to the entity instances or the relations between them. Without the objectContext the EF infrastructure cannot create the necessary insert/delete/update statements for your data store.

Now a typical Service-oriented WCF service (cfr. 4 tenets of SOA ) is stateless in nature. This means that an operation that enables to retrieve an object graph through EF registers this action with the EF objectContext , but due to the nature of the service operation, the objectContext doesn’t stay around.

Kurt showed in some examples what this means for us as a developer . He also tried to come up with solution candidates to the problems like for example re-fetching the entity again and using special functionality from the objectContext the apply the changes. It worked but only for a single entity instance without relations. If you would like to go further …you had to do it yourself. Another solution candidate was to re-attach the object and apply all the changes to the re-attached object . If you had relations….you get the picture: DIY was the common factor in the candidate solutions.

There are some efforts in the EF community to give support of object change tracking in a distributed scenario but some of them are not active anymore and some are more research like efforts to look for solution.

Kurt concluded his presentation with what is coming in Vnext of EF , as told on PDC 2008.

What I remember is that if you’re going to use EF in a distributed scenario you should clearly do your homework first and recognize the potential difficulties you about to face in your situation and come up with a strategy to solve them.

Feel free to check the Visug site if you want to browse through the presentation. Maybe Kurt will also post his examples so you can try them out?

What are your experiences with EF?

Thanks for reading,

Best regards,

Alexander

VISUG event : VSTS 2010

2008-11-28T09:32:00.005+01:00

Hello,

Yesterday (27/11/2008) I attended the VISUG presentation about Visual Studio team System 2010 ( previously known as Rosario) . Pieter Gheysens gave a focused presentation on some highlights based on the VSTS 2010 CTP (November 2008).
VSTS 2010 is the third incarnation of the Visual Studio team System product portfolio. Since the previous versions (VSTS 2005, VSTS 2008) it has improved on several levels most notably on Source control management, build management ,test management and architecture & design.

Testing

A new feature called Team System Lab Management. It is intended to ease up the configuration of testing environments using virtualization technology. (Not included in the CTP though)
A new standalone tool code-named “Camano” . This tool is intended to help you manage test plans; test configuration, functional test cases.
A functional (manual) test execution assistant (not sure if this is part of Camano or something separate) that helps the tester to perform the tests. Pieter gave demo and showed the potential advantage of this tool. It namely can capture everything the tester was doing while executing the test. Not only the visible step through video capturing the screen but also the system information and the event happing in the application and optionally also the called methods and parameters. When a issue arising during testing the tester can link up with TFS and submit a bug item together with the captured execution data. The developer can use this information while investigating the issue (No more “it works fine on my machine” replies. A question from the audience quickly also showed that you should be careful whet you capture because every thing is stored inside TFS SQLserver databases. It was also not immediately how the test runner gathers all program execution information. Do we need to build a special instrumented version of the application when we give it to the functional tester?
Beside Web UI testing that was already available in previous version, a new test template named “coded Ui test” is available. Like in web UI testing, you can record your actions on a Winforms or WPF application and convert it into code and run it from the VS integrated test environment
There are also a some new test window s; The Impacted Tests view provides a list of tests that need to be run and which code changes are covered by each of the tests. The Code Changes view provides a list of code changes and which tests must be run in order to validate each of them. Of course the best practice is to run all test again when you change something but I guess this can be a time-saver in some cases.

Architecture

For better understanding code the code (for example the code base you get on your plate you never seen before and where you need to add or fix something) the new Architecture Explorer enables you to create a visual representation of existing code assets to give better insight , to see how things fit and work together. This can help you also in Impact analysis for changes.
There are several UML model templates available. The difference with the previous Class diagram for example (beside not being UML ) is the fact that it is not linked with code. An exception is the sequence diagram that you can reverse engineer from existing code. I don’t know if it is two way though. So the most models are merely a communication medium. Being integrated in VSTS saves you the use of Visio or enterprise architect.
A example of architectural constraint enforcement was also given. The Architecture Layer Diagram enables validation of code against a visual representation of the architecture. For example the presentation layer may not talk to class from the data access layer, you must pass a service layer. With the diagram you can hook up the class to be part of certain layer.
I still wonder how this all fits in the OSLO vision of model-driven development .

Source control and management

The most visually appealing demo was the branch visualization. VSTS now can depict branch and merge operation in a viewer ( a picture tells a 1000 words sometimes)

There some other topics on project management and the integration of TFS with Office Excel and Microsoft project , gated check in, workflow and parallel build management , ….and the ones I forgot . It other words there was to little time. I think Pieter have could spend an entire day showing new features of VSTS 2010.

I guess download the CTP and start playing with it.

Also...a nice gesture from Microsoft , they made several books avaible for the attendees. You had to be quick to grab one. But anyway thank you Microsoft.

There are some interesting session coming up so feel free to check out the Visug web-site (Design principles, Entity Framework, WCF)

Thanks for reading.

Best regards,

Alexander

Isolating your Dependency Injection Container

2008-10-17T09:44:00.012+02:00

Hello,

While searching for some background information about Dependency Injection Frameworks (aka Ioc containers ; ex. Windsor Castle, Unity, Spring.NET , StructureMap, etc) I stumbled on this posting by Chris Tavares (Unity creator) regarding the creation of a isolation layer for DI containers.

Because there are several DI containers available , the creation of an extra shield between your code and the DI container of your choice seems not a bad idea if you want to keep a maximum of freedom to swith from container to container somewhere in time. It could be that in the lifetime of an application another DI container might be more suitable for your problem at hand because of certain feature it has.

This is made possible with a generic interface that makes up the operations you can call from a DI container. So your code, when it needs a service from a DI container, only works against this interface. At application startup you of course need to hook up a real DI Container implementation with its own configuration semantics. For several DI containers ther are already adapters written that convert the generic API semantics to the DI container specific API calls.

You call read about it and download the code from the codeplex site.

Maybe some code snippets to get your appetite perhaps started?

In the following code snippet you see the creation of the Castle Windsor Container and the act of resolving an instance of a particular type.


    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                IWindsorContainer windsorContainer = CreateWindsorContainer();

                ObjectCatalog.IMyObject obj = windsorContainer[typeof(ObjectCatalog.IMyObject)] as ObjectCatalog.IMyObject;

                System.Console.WriteLine("Straight Windsor");
                System.Console.WriteLine(obj.WhoAreYou());
                System.Console.ReadLine();
            }
            catch (Exception ex)
            {
                System.Console.WriteLine(ex.Message);
                System.Console.ReadLine();
            }
        }

        private static IWindsorContainer CreateWindsorContainer()
        {
            IWindsorContainer windsorContainer;
            windsorContainer = new WindsorContainer(new XmlInterpreter(new ConfigResource("castle")));
            return windsorContainer;
        }
    }

The Castle Windsor config section looks like this

ps: I'm using SyntaxHighLighter. The XML is not shown correctly. It always set to uppercase. I will need to look it to that. (Tips are welcome!)




 
              name="castle"
          type="Castle.Windsor.Configuration.AppDomain.CastleSectionHandler, Castle.Windsor" />
  
  
    
                  id="idIMyObject"
            service="ObjectCatalog.IMyObject, ObjectCatalog"
            type="ObjectCatalog.MyObject, ObjectCatalog" />

Consequently your code is pretty heavily "connected" to the DI container implementation of Castle Windsor in variuos ways (code + config). Which isn't necessarly all bad because in the beginning of the project you choose Windsor for its abilities. But suppose you want (or you need ; consolidation, standardization , features , ...) to change to another DI container implementation

Let's rewrite everything to the Microsoft P'n'P Unity DI container.



  class Program
    {
        static void Main(string[] args)
        {
            try
            {

                IUnityContainer unityContainer = CreateUnityContainer();

                ObjectCatalog.IMyObject obj = unityContainer.Resolve();

                System.Console.WriteLine("Straight Unity");
                System.Console.WriteLine(obj.WhoAreYou());
                System.Console.ReadLine();
            }
            catch (Exception ex)
            {
                
                System.Console.WriteLine(ex.Message);
                System.Console.ReadLine();
            }

        }

        private static IUnityContainer CreateUnityContainer()
        {
            IUnityContainer unityContainer;
            unityContainer = new UnityContainer();
            UnityConfigurationSection section = ConfigurationManager.GetSection("unity") as UnityConfigurationSection;
            section.Containers["containerUnity"].Configure(unityContainer);
            return unityContainer;
        }
    }

The config file for Unity for the example looks like this

So besides the config information and the creation of the container, you also must change every location where you use the services of the DI container (that's usually more than than one like in the example:-)

This is where the ServiceLocator comes in. It offers a generic way to talk to your DI container. The only implementation specifics are the config information and the creation of the container. So if you would need to change one day from one DI container to another, the changes kept to a minimum (although changing a DI config info from one format to another is also not to be taken light. Maybe some XSL magic !)

So how would the example look with the ServiceLocator?


   class Program
    {
        static void Main(string[] args)
        {
            string ioc ="" ;
            if (args.Length==0) 
                {ioc="Windsor";}
            else
                ioc=args[0] ;

            try
            {

                IServiceLocator myLocator = CreateLocator(ioc);

                ObjectCatalog.IMyObject obj = myLocator.GetInstance();

                System.Console.WriteLine("Service Locator");
                System.Console.WriteLine(" Implementation : " + myLocator.GetType().ToString());
                System.Console.WriteLine(obj.WhoAreYou());
                System.Console.ReadLine();
            }
            catch (Exception ex)
            {
                
               System.Console.WriteLine(ex.Message);
               System.Console.ReadLine();
            }
        }

        private static IServiceLocator CreateLocator(string ioc)
        {
           IServiceLocator myLocator=null;
           if (ioc=="Unity")
            myLocator =  new UnityServiceLocator( CreateUnityContainer());
           else if (ioc=="Windsor")
            myLocator = new WindsorServiceLocator(CreateWindsorContainer());
           else
               throw new System.ArgumentException("Only Unity or Windsor are valid arguments");

           return myLocator;
        }
        
        private static IWindsorContainer CreateWindsorContainer()
        {
            IWindsorContainer windsorContainer;
            windsorContainer = new WindsorContainer(new XmlInterpreter(new ConfigResource("castle")));
            return windsorContainer;
        }
           private static IUnityContainer CreateUnityContainer()
        {
            IUnityContainer unityContainer;
            unityContainer = new UnityContainer();
            UnityConfigurationSection section = ConfigurationManager.GetSection("unity") as UnityConfigurationSection;
            section.Containers["containerUnity"].Configure(unityContainer);
            return unityContainer;
        }
    }

So your code now makes use of a DI container generic API in order to hold on to a reference to a concrete DI Container implementation and also the calls to resolve dependencies or get instances from the DI container are written in a generic manner.
The DI Container specific calls are made in the adapters (one for each concrete DI Container). So "GetInstance" will be translated to "Resolve" for the Unity Container.

Thanks for reading.

Best regards,

Alexander

VISUG event : ASP.NET MVC

2008-10-15T10:18:00.003+02:00

Hello

Yesterday (14/oct/2008) , I attended a presentation on ASP.NET Model-View–Controller (MVC) framework by Maarten Balliauw that was organized by VISUG ( a Belgian .NET community). Last web development I did was in Netscape (Who?) LiveWire (What?) and I also some ASP (thus pre .NET era). So I somehow skipped ASP.Net webforms but still followed it from the side-line. So this was an excellent opportunity to keep up-to-date with web-development.

These are some points that I learned from the presentation:

ASP.NET MVC model is built on top of the ASP.NET framework.
It is something you have to install separately.
Fully integrated into Visual Studio
It is NOT a replacement for the current ASP.NET Webforms framework
The main goal is to allow programmers to clearly separate responsibilities according the MVC pattern. This clear separation of presentation logic, UI and the “rest” of the application is not only conceptually but also reflected in the code and project structure as created by the ASP.NET MVC project template
You can use ASP.NET infrastructure in an ASP.NET MVC application. Cohabitation of the webform way of working and the MVC way of working is possible. You can also use ASP.NET providers like membership and roles. You can also re-use ASP.NET session and so fort.
ASP.NET Server controls can be used as long as they don’t rely on the ViewState mechanism.
The filter-mechanism to create attributes to annotate parts of application tha can do reccuring activities for you (err handling, logging, etc)
A special routing mechanism for your requests is available.By default relies on a special convention of naming and structure to kick start the right controller. But you have the ability to intervene in this routing mechanism
Also the View parts must follow a convention (naming , folders)
You can more easily test the controller logic in “classic” unit test by controlling the dependencies with dependency injection and mocking frameworks. ASP.NET MVC plays along.
The “car and Motor-cycle” analogy : Webforms -> car , ASP.NET -> Motor-cycle. I guess it has to do with protection , speed, and so fort . Or with the fact that you can choose between them

Some questions I still have

Still in CTP. Maarten thinks at PDC 2008 a beta will made available and a V1 will soon follow. But how it will be packaged, I’m not sure.
What about a MVP style implementation of ASP.NET webforms. Don’t you also get clean(er) separation, testability of presentation logic, understandability , etc ?
Is the Web Client Software factory from Patterns& practices a viable alternative?
In the examples Maarten showed I show a lot of in-line code in the View pages. So while server controls in webforms spit out the markup. Apparently I have to do the most your-self though Maarten mentioned some tools that can help you to alter the view rendering (Nhaml)

Some questions from the audience

Can the controller be in separate assemblies? In the examples shown the relation between view and controller was very tight in the sense of the “actions” you can trigger from a view (so no reference dependencies). So in first line I would think this is a part of the presentation layer But according to Maarten there is someone who tampered with the routing mechanism to achieve this.
Hooking up Winforms in this MVC mechanisms: It is called the ASP.NET MVC framework. It uses the System.Web namespace. So in the lower parts of the framework I guess all Http-stuff (get, post, etc). Maybe the MVP pattern is better suited? Has anyone used the Smart Client Software Factory?
There was also a question about comparison , I believe, between Monorail , classic Web forms and ASP.NET MVC. I think this could be a valuable for many in order to have some “framework” to base a decision on what web development model to use (car-motorcycle analogy). One of the bullets is the presenation was : Not for everyone. So I guess there are some points to think about before embarking in a ASP.NET MVC implementation .

I’m sure I forgot some things from the presentation. If you have any comments or would like to add some points (benefits or concerns) , feel free to drop a line.

Visug has some more interesting sessions coming up ;

ASP.NET 3.5 sp1 novelties,
Rosario(VSTS 2010) ,
Entity framework & WCF and
a session by Juval Lowy (hardcore WCF).

Thanks for reading.

Best regards,

Alexander

Pex - Test Case 7

2008-08-20T10:08:00.014+02:00

Hello,

In this Pex test case I will investigate unit test on classes the have dependencies.( feel free to read the previous posts regarding this subject).
I'll first start with how you would unit test this scenario without the help of Pex and afterwards I'll investigate how Pex might help.

I have requirement to calculate the bonus a salesperson earns at the end of the year. The bonus depends on the sales he/she made this year and the year before.

The rules are

The bonus amounts to 1% of the sales for that person

If this year's sales is higher than 30000 € AND ALSO this year's sales are higher than the Sales of the previuos year for that salesperson, an extra bonus is added of 250 €

A quick test analysis should result in following test cases

currentYearSales greater or equal than 30000 €	Yes	Yes	No	No
currentYearSales greater than LastYearSales	Yes	No	Yes	No
Base bonus (1% currentYearSales )	X	X	X	X
Extra bonus (250€)	X	-	-	-

So three test cases would be sufficient for complete test coverage (last two test cases can be collapsed).

Suppose you implements the rule in a bonus class and to know the sales amounts you have a Sales class that has appropriate methods to give you them. You've also encapsulated the retrival of these sales amounts inside the bonus class method that calculates the bonus.


    public class Bonus
    {
         public double CalculateForEmployee(uint EmployeeId)
        {
         double baseBonusPercentage = 1;
         double extraBonusAmount = 250;
         uint minSalesForBonus = 30000;

         Sales sales = new Sales();
         uint currentYearSales = sales.CurrentYearSalesForEmployee(EmployeeId);
         uint lastYearSales = sales.LastYearSalesForEmployee(EmployeeId);

         double bonus = currentYearSales * (baseBonusPercentage / 100);
         
            if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
                bonus += extraBonusAmount;

            return System.Math.Floor(bonus);
        }
    }


    public class Sales 
    {
        public uint LastYearSalesForEmployee(uint EmployeeID)
        {
            //Could be a database call
        }

        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            //Could be a database call
        }
    }

Suppose you want to unit-test the method CalculateForEmployee in the Bonus class. The method calls into a Sales object to retrieve the sales for this year and the year before. Our code-under-test depends on another object to do its job. Nothing prevents you to write a standard unit test calling CalculateForEmployee.


     [TestMethod()]
        public void CalculateForEmployeeTest()
        {
            Bonus target = new Bonus(); 
            uint EmployeeId = 0; //  ?
            double expected = 0; // ?
            double actual;
            actual = target.CalculateForEmployee(EmployeeId);
            Assert.AreEqual(expected, actual);
        
        }

But how do I know which EmployeeId to use and how do I know what the expected value for the bonus should be? In fact the method-under-test has dependencies we don't control in the unit test. The bonus class doesn't control the value of the sales. it's the responsibility of the sales class. As a matter a fact this situation isn't a pure unit test anymore. It's a unit integration test. So how can we control this dependency on the sales object and know in advance the EmployeeId and Sales amounts necessary for our method-under-test. Suppose a database is used to store sales information. We could setup a test database with all the necassary records for retrieving the sales for a particular sales employee. But this would still not classify our test as a unit test (only testing the rules for caculating the bonus). Also you must account for setting up three different test situation in the database.

Rewritting our classes a little bit could helps us. Interfaces, dependency injection and test doubles are techniques I will us to make the CalculateForEmployeeTest easier to test : knowing it advance what the sales amounts will be in order to assert the bonus.

First we extract an interface from our Sales class


    public interface ISales
    {
        uint LastYearSalesForEmployee(uint EmployeeID);
        uint CurrentYearSalesForEmployee(uint EmployeeID);
    }


  public class Sales : ISales
    {
        public uint LastYearSalesForEmployee(uint EmployeeID)
        {
            //Could be a database call
        }
        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            //Could be a database call
        }
    }

In this example I'll use "manual" dependency injection (constructor injection) to demostrate the dependency injection principle.


    public class Bonus
    {
        ISales mSalesdao;
   
     
        public Bonus(ISales aSalesDao)
        {
            mSalesdao = aSalesDao;
        }

        public double CalculateForEmployee(uint EmployeeID)
        {
            double baseBonusPercentage = 1;
            double extraBonusAmount = 250;
            uint minSalesForBonus = 30000;
            double bonus;

            uint currentYearSales = mSalesdao.CurrentYearSalesForEmployee(EmployeeID);
            uint lastYearSales = mSalesdao.LastYearSalesForEmployee(EmployeeID);
            
            bonus = currentYearSales * (baseBonusPercentage / 100);
            if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
                bonus += extraBonusAmount;
            
            return System.Math.Floor(bonus);
        }
    }

So how does this help me in my unit test you might wonder? A test-double is a test-specific class that we will use in our unit tests so the code-under-test will not notice it that much. There are several variations of test-doubles. For example a test stub is an equivalent of the dependend class but gives "fixed answers" if we ask for something or does something in a pre-determined way.

So instead of passing a real instance of the type Sales to the constructor of Bonus, I slip in a test stub when I write the unit tests. Because it gives fixed answers , I'll need a test stub for each test case.


    public class SalesStub_CYSsmaller30000 : ISales
    {
       
        public uint LastYearSalesForEmployee(uint EmployeeID)
        {
            return 29999;
        }
        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            return 29998;
        }
    }

    public class SalesStub_CYSgreater30000_CYSgreaterLYS : ISales
    {
        public uint LastYearSalesForEmployee(uint EmployeeID)
        {
            return 30001;
        }
        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            return 30002;
        }
      
    }

    public class SalesStub_CYSgreater30000_CYSsmallerLYS : ISales
    {

        public uint LastYearSalesForEmployee(uint EmployeeID)
        {
            return 30003;
        }

        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            return 30002;
        }

    }

The unit tests use these stubs to do their thing. I create a teststub that is appropriate for the test case and inject it into the constructor of bonus.


        [TestMethod()]
        public void CalculateForEmployee_Lys29999_cs29998_withStub()
        {
            ISales aSalesDao = new SalesStub_CYSsmaller30000();
            Bonus target = new Bonus(aSalesDao); 
            uint EmployeeID = 0; // Dummy EmployeeID not important for this example
            double expected = 299; //
            double actual;
            actual = target.CalculateForEmployee(EmployeeID);
            Assert.AreEqual(expected, actual);     
        }


        [TestMethod()]
        public void CalculateForEmployee_Lys30001_cs30002_WithStub()
        {
            ISales aSalesDao = new SalesStub_CYSgreater30000_CYSgreaterLYS();
            Bonus target = new Bonus(aSalesDao); 
            uint EmployeeID = 0; // Dummy EmployeeID not important for this example
            double expected = 550; //
            double actual;
            actual = target.CalculateForEmployee(EmployeeID);
            Assert.AreEqual(expected, actual);
        }

       //and so

Another implementation for test doubles is the use of a Mock framework. This utility dynamically create test-equivalent objects that are setup (expectations) through the framework facilities . I'll illustrate this with the help of Rhino.Mock.


     [TestMethod()]
       public void CalculateForEmployee_Lys29999_cs29998_withMock()
        {
             uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose

            Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
            ISales salesDao = mr.CreateMock();
            Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(29999);
            Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(29998);
            mr.ReplayAll();

            Bonus target = new Bonus(salesDao); 
          
         
            double expected = 299; 
            double actual;
            actual = target.CalculateForEmployee(EmployeeID);
            Assert.AreEqual(expected, actual);

            mr.VerifyAll();

           
        }


       [TestMethod()]
       public void CalculateForEmployee_Lys30001_cs30002_WithMock()
            {
                uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose

                Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
                ISales salesDao = mr.CreateMock();
                Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(30001);
                Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(30002);
                mr.ReplayAll();

                Bonus target = new Bonus(salesDao); 
               
               
                double expected = 550; //
                double actual;
                actual = target.CalculateForEmployee(EmployeeID);
                Assert.AreEqual(expected, actual);

                mr.VerifyAll();
            }
      //and so on

The use of a Mock framework doesn't force me to explicitly write numerous of test stubs.

So where does Pex come in? For Pex to do its job I'll need a parametrized unit test. This is a unit test that will take arguments. Consider it as a test refactoring thing. Things that are common in the unit test are extracted and put into separate method/function. Pex needs this setup because it will be in charge with generating "traditional" unit tests with test data values that it will determine during its exploration phase. These generated unit test will call the parametrized unit test , passing the necessary arguments.

In our "mock" example we explicitly setup the mock for each unit test. In analogy with the parametrized unit test we need a parametrized Mock object (see Pex Tutorial).


    [TestClass]
    [PexClass]
    public partial class BonusTest
    {
        [PexMethod]
        [PexUseType(typeof(SalesPexMock))]
        public void Calculate(ISales aSalesPexMock)
        {
            PexAssume.IsNotNull(aSalesPexMock);
            
            uint EmployeeID = 9999; //dummy , not important for our test
            Bonus target = new Bonus(aSalesPexMock);
            double result = target.CalculateForEmployee(EmployeeID);
            PexValue.AddForValidation("result", result);
        }
        
    }

    [PexMock]
    public class SalesPexMock : ISales
    {
        public uint LastYearSalesForEmployee(uint aEmployeeID)
        {
            var call = PexOracle.Call(this);
            uint lys = (uint)call.ChooseFrom("lastYearSales", new uint[] { 29999, 30000, 30001 });
            PexValue.Add("lastYearSales", lys);
            return lys;
        }

        public uint CurrentYearSalesForEmployee(uint EmployeeID)
        {
            var call = PexOracle.Call(this);
            uint lys = (uint)call.ChooseFrom("CurrentYearSales", new uint[] {29999, 30000, 30001});
            PexValue.Add("CurrentYearSales", lys);
            return lys;
        }
    }

Running the PexExplorartion on the PexMethod will result in three unit test being generated.


   [TestMethod]
        [PexGeneratedBy(typeof(BonusTest))]
        public void Calculate01()
        {
            PexValue.Generated.Clear();
            IPexOracleRecorder oracle = PexOracle.NewTest();
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "CurrentYearSales", (object)30000u);
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "lastYearSales", (object)30000u);
            SalesPexMock salesPexMock = new SalesPexMock();
            this.Calculate((ISales)salesPexMock);
            global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "300");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(BonusTest))]
        public void Calculate02()
        {
            PexValue.Generated.Clear();
            IPexOracleRecorder oracle = PexOracle.NewTest();
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "CurrentYearSales", (object)29999u);
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "lastYearSales", (object)30000u);
            SalesPexMock salesPexMock = new SalesPexMock();
            this.Calculate((ISales)salesPexMock);
            global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "299");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(BonusTest))]
        public void Calculate03()
        {
            PexValue.Generated.Clear();
            IPexOracleRecorder oracle = PexOracle.NewTest();
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "CurrentYearSales", (object)30001u);
            ((IPexOracleSessionBuilder)
              (oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
                .ChooseAt(0, "lastYearSales", (object)29999u);
            SalesPexMock salesPexMock = new SalesPexMock();
            this.Calculate((ISales)salesPexMock);
            global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "550");
        }

Of course meanwhile you verified the test coverage results :-) in the Pex report or via the Visual Studio Test Coverage facility.

So with the PexMock framework expectations for the indivudual unit test are setup , just like we did in our Rhino.Mock example. Of we had to help the Pex mock framework a little bit just like we did with the parametrized unit test.

Thanks for reading. Of course your comments are most welcome.

Best regards,

Alexander

ps: We already had complaints of Salespersons that find this bonus rule not fair. for example sales-persons that are above the minimum sales amount eligable for extra bonus but are not progressing , don't get it. Like wise someone who is progressing doesn't receive extra incentives.... :-)

Pex - test case 6 (revisited...again)

2008-08-11T11:57:00.008+02:00

Hello,

I did some further test on test case 6 . I removed all pex assumptions regarding the dates. This time Pex had no problem finding the test cases necessary for a complete block coverage.

So the PUT states no assumptions regarding the dates;


 [TestClass]
    [PexClass(typeof(EmployeeServices))]
    public partial class EmployeeServicesTest
    {
        [PexMethod()]
        public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
        {
           PexAssume.IsNotNull(aEmployee);
           PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
           PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);


            PensionContribution result = target.DeterminePensionContribution(aEmployee);

            PexValue.AddForValidation("BaseContribution", result.BaseContribution);
            PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

            if (aEmployee.Salary <= 3000)
                Assert.IsTrue(result.BaseContribution == 80);
            if (aEmployee.Salary > 3000)
                Assert.IsTrue(result.BaseContribution == 135);
            if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aEmployee.Salary > 3000 && aEmployee.ServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 30);
        }
        
    }

When you run Pex this time it generated none less than 11 unit tests. The generated DateTime value are constructed with the Ticks argument, so it no easy to see the actual dateTime. Actually the first time I run this Pex generated a tick value with a value larger than 32 bit. When running the generated unit tests, an exception was thrown specifying thant the tick value should be between the DateTime.Minvalue.Ticks and DateTimeValue.Maxvalue.Ticks. It seemed to me that i generated a 64-bit value.
I haven't been able to reproduce this error though. May be it is one of the remarks in the release of this version 0.6x. I'm running on Windows XP SP2!

Nevertheless here are the generated unit tests.


    public partial class EmployeeServicesTest
    {
        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution01()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(1L);
            employee = EmployeeFactory.Create
                (2, "", "", default(DateTime), s0, 8192u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution02()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(1L);
            employee = EmployeeFactory.Create
                (2, "", "", default(DateTime), s0, 1983u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "25");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution03()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(1L);
            employee = EmployeeFactory.Create
                (2, "", "", default(DateTime), s0, 8192u, Contract.PartTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution04()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(647341024241516544L);
            DateTime s1 = new DateTime(1L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "25");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution05()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(647341024241516544L);
            DateTime s1 = new DateTime(1L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "15");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution06()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(647341024241516544L);
            DateTime s1 = new DateTime(1L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution07()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(1262076461318144L);
            DateTime s1 = new DateTime(634715138118123520L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "30");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution08()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(56623104001835008L);
            DateTime s1 = new DateTime(634715138118123520L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "15");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution09()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(647341024241516544L);
            DateTime s1 = new DateTime(634715138118123520L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "30");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution10()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(647341024241516544L);
            DateTime s1 = new DateTime(1L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.PartTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "0");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContribution11()
        {
            PexValue.Generated.Clear();
            Employee employee;
            DateTime s0 = new DateTime(633532870975488000L);
            DateTime s1 = new DateTime(126226978751643648L);
            employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices employeeServices = new EmployeeServices();
            this.DeterminePensionContribution(employeeServices, employee);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "15");
        }

    }

When you investigate the coverage results for Visual Studio and the output from Pex, you will see that both confirm the 100% coverage result.

Part of the Pex report , coverage section (click on picture for better view):

What about my "business" assumptions? Maybe it would be easier to incorporate them as pre-conditions in the code under test. For example Start date of the contract should be greater than 25/May/1968 in the setter method for Startdatecontract of our Employee class. Non-compliance would than result in an exception being thrown. how will pex cope with them?

Thanks for reading. As always, feel free to comment.

Best regards,

Alexander

Pex - test case 6 (revisited)

2008-08-11T08:56:00.005+02:00

Hello,

In the previous case I tried to come up with a scenario where a method takes a "complex" type instead of primitive types like integers of booleans. The exercise didn't go as expected. By the way , the latest downloadable version of Pex is 0.6.30728.0

To see whether pex still works on a method with primitive types, I re-wrote the method as follows.


  public PensionContribution DeterminePensionContribution2(
             uint aSalary, 
             uint aAge, 
             uint aServiceYears, 
             Contract aTypeContract)
        {
            //no pre-codition checks ....

            PensionContribution contribution = new PensionContribution();

            if (aSalary <= 3000)
            {
                if (aAge < 45)
                {
                    if (aTypeContract == Contract.FullTime)
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 15;
                    }
                    else
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 0;
                    }
                }
                else
                {
                    if (aServiceYears < 5)
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 15;
                    }
                    else
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 25;
                    }
                }
            }
            else
            {
                if (aAge < 45)
                {
                    if (aServiceYears < 5)
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 30;
                    }
                    else
                    {
                        if (aTypeContract == Contract.FullTime)
                        {
                            contribution.BaseContribution = 135;
                            contribution.SupplementaryContribution = 25;
                        }
                        else
                        {
                            contribution.BaseContribution = 135;
                            contribution.SupplementaryContribution = 20;
                        }
                    }
                }
                else
                {
                    if (aServiceYears < 5)
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 30;
                    }
                    else
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 20;

                    }
                }

            }
            return contribution;
        }

The PUT method looks like this.


      [PexMethod]
        public void PUTDeterminePensionContribution(
            [PexAssumeUnderTest]EmployeeServices target,
            uint aSalary,
            uint aAge,
            uint aServiceYears,
            Contract aTypeContract
        )
        {
            PexAssume.IsTrue(aAge > 18 && aAge < 75);
            PexAssume.IsTrue(aServiceYears > 0 && aServiceYears < 50);
            PexAssume.IsTrue(aSalary > 1400 && aSalary < 10000);
            PexAssume.IsTrue(aTypeContract == Contract.FullTime || aTypeContract == Contract.PartTime);

            PensionContribution result = target.DeterminePensionContribution
                (aSalary, aAge, aServiceYears, aTypeContract);
            PexValue.AddForValidation("BaseContribution", result.BaseContribution);
            PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

            if (aSalary <= 3000)
                Assert.IsTrue(result.BaseContribution == 80);
            if (aSalary > 3000)
                Assert.IsTrue(result.BaseContribution == 135);
            if (aSalary <= 3000 && aAge >= 45 && aServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aSalary <= 3000 && aAge < 45 && aTypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aSalary <= 3000 && aAge >= 45 && aServiceYears >= 5)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.PartTime)
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aSalary > 3000 && aAge >= 45 && aServiceYears >= 5)
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aSalary > 3000 && aServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 30);
        }

In no time Pex finds the test cases for full block coverage

target	aSalary	aAge	aServiceYears	aTypeContract	BaseContribution	SupplementaryContribution	Summary/Exception	Error Message
1	...	8192	64	2	FullTime	135	30
2	...	8192	64	30	FullTime	135	20
3	...	8192	31	2	FullTime	135	30
4	...	8192	31	30	FullTime	135	25
5	...	1532	64	2	FullTime	80	15
6	...	1532	64	30	FullTime	80	25
7	...	1532	31	2	FullTime	80	15
8	...	8192	64	2	PartTime	135	30
9	...	1532	31	2	PartTime	80	0
10	...	8192	31	30	PartTime	135	20

So there is nothing "wrong" with the method under test. The problem when you investigate the output window, lies maybe in the calculation of the "age" and "serviceYears" properties.


00:00:00.0> starting execution
  00:00:00.0> reflecting tests
  00:00:00.3> TestProject1
    00:00:00.3> EmployeeTest
      00:00:00.3> DeterminePensionContributionIntern(Employee)
         maxconstraintsolvertime - 1s (constraint solver time out after 1 seconds)
        [execution]     4 runs
        [execution]    15 runs
        [test] (run 20) DeterminePensionContribution
        [test] (run 21) DeterminePensionContribution
        [execution]    21 runs, 49/81 blocks covered
        [test] (run 25) DeterminePensionContribution
        [execution]    29 runs, 49/81 blocks covered
        [execution]    41 runs, 49/81 blocks covered
        [modelsearch] could not flip 2 branches in a row
        [execution]    50 runs, 49/81 blocks covered
        [execution] imprecision at testingPEX.Employee.differenceInYears, offset 0x26
        [modelsearch] could not flip 4 branches in a row
        [execution]    77 runs (97,40% unique paths), 49/81 blocks covered
         timeout - 120s
         Timeout, 1 times
         MaxConstraintSolverTime, 60 times
        [coverage] 49/81 block (60,49%)

00:02:01.4> [finished] 3 generated tests (0 failures), 00:02:01.3757768.
 -- 0 critical errors, 0 errors, 0 warnings.

[reports] skipping report generation

  EXPLORATION SUCCESS


monitored process exited with Success (0)
finished

Pex needs to find dateTimes in order to test the conditions specified in the method. I used a helper method to calculate the years to date for the age and serviceyears:


   private uint differenceInYears(System.DateTime aDate1, System.DateTime aDate2)
        {
            //missing checks ....
            //approximative
            uint years = (uint)(aDate1.Year - aDate2.Year);
            if (aDate1.Month == aDate2.Month)
            {
                if (aDate2.Day < aDate1.Day) years = years - 1;
            }
            return years;
        }

Could it be the unsigned integer type declarations? Or are the assumptions wrongly formulated? Or both? What does "could not flip 2 branches in a row" mean?

I'll try to do some more test around this. Feel free to comment!

Thanks for reading.

Best regards,

Alexander

Pex - Test Case 6

2008-08-06T09:11:00.010+02:00

Hello,

In the previous Pex posts I used examples that used primitive types and/or strings as parameters for methods under test. In this post I will "test" Pex how it wil cope with a method that accepts a "complex type". This test scenario is similar to the case study in Pex - Test case 3 except we don't work directly with primitive types a method parameters or return types.

Suppose the HR department asked you to include a new module in their Salary-application that will determine the contribution each employee has to make in a private pension fund. The salary administration uses the following desicion table.

While analysing the table you conclude you can simplify the table to help you code this rule into c#.

You already have some classes in the application :


    public class Employee
    {
        //missing elementary checks in order to keep code concise....
        public int EmployeeID { get; set; }
        public string FirstName { get; set; }
        public string LastName { get; set; }
        private DateTime _BirthDate;
        public DateTime BirthDate 
        { 
            get { return this._BirthDate; }
            set { 
                this._BirthDate = value;
                this.Age = differenceInYears(_BirthDate, DateTime.Now);
                }
        }
        private DateTime _StartDateContract;
        public DateTime StartDateContract { 
            get { return _StartDateContract; } 
            set { 
                _StartDateContract=value;
                this.ServiceYears = differenceInYears(_StartDateContract, DateTime.Now);
                 } }
        public uint Salary { get; set; }
        public Contract TypeContract { get; set; }
        public uint Age { get; set; }
        public uint ServiceYears { get; set; }
        private uint differenceInYears(DateTime aDate1, DateTime aDate2)
        {
            //missing checks ....
            //approximative
            uint years = (uint)(aDate1.Year - aDate2.Year);
            if (aDate1.Month == aDate2.Month)
            {
                if (aDate2.Day < aDate1.Day) years = years - 1;
            }
            return years;
        } 
    }

    public enum Contract
    {
        FullTime=0,
        PartTime=1
   }
    
    public struct PensionContribution
    {
        public uint BaseContribution{get;set;}
        public uint SupplementaryContribution {get;set;}
    }

The rule has also been translated into c#. The method DeterminePensionContribution has a parameter of the type Employee as described in the above code snippet. The return type is the structure PensionContribution.


    public class EmployeeServices
    {
        public PensionContribution DeterminePensionContribution(Employee aEmployee)
        {
            //no pre-condition checks .....

            PensionContribution contribution = new PensionContribution ();

            if (aEmployee.Salary <= 3000)
            {
                if (aEmployee.Age < 45)
                {
                    if (aEmployee.TypeContract == Contract.FullTime)
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 15;
                    }
                    else
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 0;
                    }
                }
                else
                {
                    if (aEmployee.ServiceYears < 5)
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 15;
                    }
                    else
                    {
                        contribution.BaseContribution = 80;
                        contribution.SupplementaryContribution = 25;
                    }
                }
            }
            else
            {
                if (aEmployee.Age < 45)
                {
                    if (aEmployee.ServiceYears < 5)
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 30;
                    }
                    else
                    {
                        if (aEmployee.TypeContract == Contract.FullTime)
                        {
                            contribution.BaseContribution = 135;
                            contribution.SupplementaryContribution = 25;
                        }
                        else
                        {
                            contribution.BaseContribution = 135;
                            contribution.SupplementaryContribution = 20;
                        }   
                    }
                }
                else
                {
                    if (aEmployee.ServiceYears < 5)
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 30;
                    }
                    else
                    {
                        contribution.BaseContribution = 135;
                        contribution.SupplementaryContribution = 20;

                    }
                }

            }
            return contribution;
        }


        

    
       
    }

In a "test-after-you-code" scenario, you could use the desicion table to specify the values for actual for the different properties of a employee object that is being passed to the method under test. So 9 unit tests would be sufficient to touch all corners of the code.

A data-driven in test in Visual Studio is somewhat difficult because we are not dealing with simple datatype parameters. We could serialize the different instances and save it in string form into an external data source.

Can Pex help generate the necessary unit test to fully test our code?

Before heading to the PUT (Parameterized Unit Test) I would like to refer you to the Pex tutorial for more background information on setting up PexMethods.

A new element in this case study is the use of a factory method to create our Employee object that will be used in the generated unit test. Pex will call this method with appropriate values that it has discovered during its exploration of the code in the generated unit tests. The cool part is that Pex will first complain about not being able to create a Employee object but then suggest a soulution for it : the Factoty method and generates it for us.


  [PexFactoryClass]
    public partial class EmployeeFactory
    {
        [PexFactoryMethod(typeof(Employee))]
        public static Employee Create(
            int i0,
            string s0,
            string s1,
            DateTime dt0,
            DateTime dt1,
            uint ui0,
            Contract c0
        )
        {
          
            Employee e0 = new Employee();
            e0.EmployeeID = i0;
            e0.FirstName = s0;
            e0.LastName = s1;
            e0.BirthDate = dt0;
            e0.StartDateContract = dt1;
            e0.Salary = ui0;
            e0.TypeContract = c0;
            return e0;
        }

    }

In the generated unit test , this factory method is used as follows:


  [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_095656_001()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(626406282140467200L);
            DateTime s1 = new DateTime(631149005309067264L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

Let's return to the PUT. The following PUT was based on a PexMethod that you can let the Visual Studio Pex integration build for you.
During its explorartion Pex will try out different value for the properties of our Employee object. The assumptions should help Pex create reasonable test cases. The code in this example did not enforce them in order to keep the code somewhat limited in size.

Because this method will be used for all generated unit test, the asserts have to be more general than you would normally do for a single unit test.


    [TestClass]
    [PexClass(typeof(EmployeeServices))]
    public partial class EmployeeServicesTest
    {
        [PexMethod(MaxConstraintSolverTime = int.MaxValue, Timeout = int.MaxValue)]
        public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
        {
           PexAssume.IsNotNull(aEmployee);
          PexAssume.IsTrue(aEmployee.BirthDate < new DateTime(1992, 1, 1) && aEmployee.BirthDate > new DateTime(1955, 1, 1));
           PexAssume.IsTrue(aEmployee.StartDateContract < DateTime.Now && aEmployee.StartDateContract > new DateTime(1968, 1, 1));
           PexAssume.IsTrue(aEmployee.BirthDate < aEmployee.StartDateContract);
           PexAssume.IsTrue(aEmployee.Age > 18);
           PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
           PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);
            

            PensionContribution result = target.DeterminePensionContribution(aEmployee);

            PexValue.AddForValidation("BaseContribution", result.BaseContribution);
            PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

            if (aEmployee.Salary <= 3000)
                Assert.IsTrue(result.BaseContribution == 80);
            if (aEmployee.Salary > 3000)
                Assert.IsTrue(result.BaseContribution == 135);
            if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 15);
            if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
                Assert.IsTrue(result.SupplementaryContribution == 25);
            if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5 )
                Assert.IsTrue(result.SupplementaryContribution == 20);
            if (aEmployee.Salary > 3000 &&  aEmployee.ServiceYears < 5)
                Assert.IsTrue(result.SupplementaryContribution == 30);
        }
        
    }

Running the Pex exploration several times (and after quite a while) I got results but no full coverage. Maybe the assumptions and/or asserts are not "pex-friendly"?

The generated unit tests only cover half of the code blocks.


   public partial class EmployeeServicesTest
    {

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_000()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(619794039740301312L);
            DateTime s1 = new DateTime(631148677322244096L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_001()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(619794039740301312L);
            DateTime s1 = new DateTime(631148677322244096L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "25");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110121_002()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(619794039740301312L);
            DateTime s1 = new DateTime(631148677322244096L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110200_003()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(626094836655304192L);
            DateTime s1 = new DateTime(633479291125776385L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "30");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110201_004()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(626094836655304192L);
            DateTime s1 = new DateTime(633479291125776385L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "15");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_000()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(609681269903603218L);
            DateTime s1 = new DateTime(621044308151107616L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_001()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(609681269903603218L);
            DateTime s1 = new DateTime(621044308151107616L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "25");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112422_002()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(609681269903603218L);
            DateTime s1 = new DateTime(621044308151107616L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "20");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115647_003()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(627392712227689792L);
            DateTime s1 = new DateTime(633479779357556736L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "135");
            PexValue.Generated.Validate("SupplementaryContribution", "30");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(EmployeeServicesTest))]
        public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115718_004()
        {
            PexValue.Generated.Clear();
            Employee e0;
            DateTime s0 = new DateTime(627392712227689792L);
            DateTime s1 = new DateTime(633479779357556736L);
            e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
            EmployeeServices es0 = new EmployeeServices();
            this.DeterminePensionContribution(es0, e0);
            PexValue.Generated.Validate("BaseContribution", "80");
            PexValue.Generated.Validate("SupplementaryContribution", "15");
        }

    }

If you have any suggestions to improve (re-factor) the test code or the code-under- test (test-friendliness), don't hesitate to drop a line.

Thanks for reading.

Best regards,

Alexander

Pex - test Case 5

2008-07-31T10:30:00.011+02:00

Hello,

This post is a continuation in a series of post about Pex, a test case generation tool from Microsoft research. This test scenario is a variation of test case 4. (sorry in advance for the table formatting , I still can not get it right :-(

In the Pex tutorial on p37 you'll see an example of testing a Regular Expression. I didn't find one in the samples though. Too simple maybe?

I'm no hero in regular Expressions. Thankfully there's a library loaded with regular expressions for every kind of application. The following regular expressing is for ensuring strong passwords.

Expression	^(?=.\d)(?=.[a-z])(?=.*[A-Z]).{4,8}$
Description	Password matching expression. Password must be at least 4 characters, no more than 8 characters, and must include at least one upper case letter, one lower case letter, and one numeric digit.
Matches	asD1 \| asDF1234 \| ASPgo123
Non-Matches	asdf \| 1234 \| ASDF12345

The code example where the regular expression is implemented in


    public class UserAccount
    {

        private string mPassword;

        public string Password
        {
            get { return mPassword; }
            set {
                if (value == null)
                    throw new System.ArgumentNullException("Please specify a valid password. (non-null)");
                else
                   {
                    bool valid = RegularExpressions.Regex.IsMatch(value, @"^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{4,8}$");
                    if (valid)
                        mPassword = value; 
                    else
                        throw new System.ArgumentException("Please specify a Strong Password. ");
                    }          
                }
        }
    }

The pex method (keeping in mind the remarks in the comments from the previous test case: the invariant remark)


        [PexMethod()]
        public void PasswordTest( string aPassword)
        {
            UserAccount target = new UserAccount();
            target.Password = aPassword;

            //no problem calling target.password several times, stays the same
            Assert.IsTrue(target.Password != null);
            Assert.IsTrue(target.Password.Length != 0);
            Assert.IsTrue(target.Password.Length >= 4);
            Assert.IsTrue(target.Password.Length <= 8);
            Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[A-Z]"));
            Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[a-z]"));
            Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[0-9]"));
         

            Assert.AreEqual(aPassword, target.Password);
        }

We could do this without Pex of course. We have have to come up with several input values to trigger an exception (at least that is what we intended to program and now are verifying) and some test values for some "positive" tests. The equivalence class technique can surely help us out.

Valid equivalence classes

bigger or equal than 4 characters

smaller or equal than 8 characters

include at least one upper case letter

include at least one lower case letter

include at least one numeric digit

Invalid equivalence classes

not null, non-existing

smaller than 4 characters

bigger than 8 characters

no upper case letter

no lower case letter

no numeric digit

You normally start with searching a physical test case (password value) that covers all the valid equivalence classes and for each invalid equivalence class you should also make a test case (i.e. unit test as well). Try not to overlap any invalid equivalance classes so you can be sure that the forced invalidity throwed the exception.

asDF1234 (covers all valid classes)

null (non existing)

Az1 (smaller than 4 chars)

ASDf12345 (etc)

abcd1

ABCD1

aBcDefGh

So 7 test cases should be enough ?

What would Pex do? It should be as good as this (if not better , at least the tutorial almost implies so).
If you would let the PexMethod work on the password setter method, you would get following results.

Run	aPassword	Summary/Exception	Error Messag
1	null	ArgumentNullException	Value cannot be null. Parameter name: Please specify a valid password. (non-null)
2	""""	ArgumentException	Please specify a Strong Password.
3	\0	ArgumentException	Please specify a Strong Password.
4	\u8000	ArgumentException	Please specify a Strong Password.
6	\0\0	ArgumentException	Please specify a Strong Password.
7	8	ArgumentException	Please specify a Strong Password.
8	\0\n	ArgumentException	Please specify a Strong Password.
10	\0\0\0	ArgumentException	Please specify a Strong Password.
11	\n	ArgumentException	Please specify a Strong Password.
12	\n\0	ArgumentException	Please specify a Strong Password.
16	\0\u8000\u8000	ArgumentException	Please specify a Strong Password.
21	\n\n	ArgumentException	Please specify a Strong Password.
25	\u00062\u00f1\u0016\n\n	ArgumentException	Please specify a Strong Password.
31	\09\u00ef\u0088f\r	ArgumentException	Please specify a Strong Password.
75	\u8000\0\0\0\n\0	ArgumentException	Please specify a Strong Password.
83	0\u0088o]O<
112	\u00062Oo\n\n	ArgumentException	Please specify a Strong Password.
113	\u00062Oo\n
114	\u00062Oo\n\n\n\n\n\n\n\n\n\n	ArgumentException	Please specify a Strong Password.

Pex established 19 test cases. A little bit more if you would go for the manual approach. But are the test input values better? In terms of "readability" no. But Pex generated some input values that I wouldn't have thought of in the first place.

The empty string was added in the Pex generated approach. In our manual exercise with didn't explicitly took that test case because this was implied in the "smaller than 4 chars" equivalence class. But again testing a little bit too much is maybe better (or saver) than testing too little.

Also the Pex generated unit tests include non alfanumeric signs like * or /. Did I intended to do that with my expression? If not I have to re-write it. With my manual technique I "forget" them. So Pex went a little deeper than my test analysis.
What about Escape characters such as "\n" (new line). pex included them and some tests passed. Was this the way I intended it? Were these the specs from the customer?

The examples I used to get acquinted with Pex may seem academic but I believ show potential for a tool like Pex. How to use it in your day-to-day applications will take some time and study. And like the Pex FAQ already states : it is not Pex versus unit testing (or other forms of testing or QA assurance measures like code analysis for example). I hope the Pex team will (continue to) find sponsers within Microsoft to integrate it in the next releases of Visual studio. At least the pex team already took a patent!.

Thanks for reading.

Best regards,

Alexander

Pex - test case 4

2008-07-28T14:10:00.008+02:00

Hello,

I've got one more test scenario for Pex. From comments on my previous post
Case study 3 I understood that the way I specified the PUT was not done correctly (though Pex generated the unit test methods I expected).

A PUT should specify what the intended behaviour of the code under test should do. It should not do this like in a tradiotional unit test for a particular combination of test input but in a more general way.

Sounds reasonable but I find some difficulity in writing them. I'll try to come with a "better" PUT this time.

Our code under test is as follows (Sorry in advance for some bad formatting HTML tables later in the post.):


      private string mCode;

        public string Code
        {
            get { return mCode; }
            set {
                if (value == null ) 
                    {
                    throw new System.ArgumentNullException("Please specify a valid code. (non-null)");    
                    }
               else if (value.Length < 4)
                {
                    throw new System.ArgumentException("Please specify a valid code. (bigger than 3 characters.");
                }
                else if (value.Length > 10)
                {
                    throw new System.ArgumentException("Please specify a valid code. (max 10 characters. including XYZ prefix) ");  
                }
                else if (value.StartsWith("XYZ") == false)
                {
                    throw new System.ArgumentException("Please specify a valid code (starting with XYZ - uppercase !) ");
                }
                else
                {
                    this.mCode = value;
                }
              
              }

With the comments for my PUT in mind , I created the following PUT:


       [PexMethod()]
         public void CodeTest(string aCode)
         {
             measurement target = new measurement();
             string actual;
             target.Code = aCode;
             actual = target.Code;
             PexValue.AddForValidation("actual", actual);
             if (( aCode != null) && 
                 (aCode.Length > 3) &&  
                 (aCode.Length <= 10) && 
                 (aCode.StartsWith("XYZ")) ) 
                 PexAssert.IsTrue(actual == aCode);
            
         }

Running Pex on this PUT will result in several generated test cases.
(Pex reporting option : Enable via Options/Pex/Reports)

Parameter Values
Run	aCode	actual	Summary/Exception
1	null		ArgumentNullException: Value cannot be null. Parameter name: Please specify a valid code. (non-null) (testingPEX)
2	""		ArgumentException: Please specify a valid code. (bigger than 3 characters. (testingPEX)
3	\0\0\0\0\0\0		ArgumentException: Please specify a valid code (starting with XYZ - uppercase !) (testingPEX)
4	new string('\0', 14)		ArgumentException: Please specify a valid code. (max 10 characters. including XYZ prefix) (testingPEX)
5	XYZZZZZ	XYZZZZZ

Of course we first need to specify the expected exceptions (Pex will suggest these for you !) :

Add attribute [PexAllowedExceptionFromAssembly(typeof(ArgumentNullException), "testingPEX")]

Add attribute [PexAllowedExceptionFromAssembly(typeof(ArgumentException), "testingPEX")]

The generated unit tests look like this :


  public partial class measurementTest
    {

        [TestMethod]
        [PexGeneratedBy(typeof(measurementTest))]
        public void CodeTestString_20080728_141020_005()
        {
            PexValue.Generated.Clear();
            this.CodeTest("XYZZZZZ");
            PexValue.Generated.Validate("actual", "XYZZZZZ");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void CodeTestString_20080728_143541_000()
        {
            this.CodeTest((string)null);
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void CodeTestString_20080728_143620_001()
        {
            this.CodeTest("");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void CodeTestString_20080728_143620_002()
        {
            this.CodeTest("\0\0\0\0\0\0");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void CodeTestString_20080728_143620_003()
        {
            this.CodeTest(new string('\0', 14));
        }

    }

If you would verify the code coverage results in Visual Studio or via the Pex reporting option, you would see 100% Block coverage.

Pex generated for us the strings to serve as input values. I don't know about you but I think this is pretty clever.

Still I'm a little confused about the PUT I created. If I would delete all the pexAssert statements the exercise would still be the same for the generated unit tests.


  [PexMethod()]
         public void CodeTest(string aCode)
         {
             measurement target = new measurement();
             string actual;
             target.Code = aCode;
             actual = target.Code;
             PexValue.AddForValidation("actual", actual);  
         }

Thanks for reading.

Best regards,

Alexander

Pex - Test case 3

2008-07-25T11:45:00.008+02:00

Hello,

This post is a continuation of previous posts. Feel free to check them out.

Introduction

Case study 1

Case study 2

In this example I base my example on the following rule:
In our Order system we need to implement a validation rule to see whether we can accept an order or not. The criteria for acceptance where given to us (luckily) in the form of a decision table. There are three criteria ; an internal customer qualification scheme, the fact if the previous payments of the customer were usually on time and correct and the fact if the customer has sufficient credit possibilities.

Cust-code

paymentsOK

CredietLimitOK

AcceptOrder


    public enum CustomerType 
        { 
            U_Unknown = 0, 
            A_Status = 1, 
            B_Status = 2, 
            C_Status = 3 
        } 

    public class OrderManager
    {
        public bool IsOrderToBeAccepted(CustomerType aClientType,  bool aGoodpaymentHistory,  bool aCreditSufficient) 
            { 
                bool orderAccept = false; 
                switch (aClientType) { 
                    case CustomerType.A_Status: 
                        if (aGoodpaymentHistory == true)  { 
                            orderAccept = true; 
                        }
                       else
                        {
                            orderAccept = false;
                        }
                        break; 
                    case CustomerType.B_Status: 
                        if (aGoodpaymentHistory || aCreditSufficient) { 
                            orderAccept = true; 
                        } 
                        else { 
                            orderAccept = false; 
                        } 
                        break; 
                    case CustomerType.C_Status: 
                        orderAccept = true; 
                        break; 
                    default: 
                        orderAccept = false;  
                        break; 
                } 
            return orderAccept; 
            } 
    }

With the help to the decsion table (thanks to the analyst :-), we can easily determine the test cases.
There 2 test sitaution (different behaviours) we must account for : the order is accepted or is not accepted. So we need test cases that will result in a non-accepted order and we will need a test case that will result in an accepted order. Those this mean two test cases (i.e. unit tests) are enough. There is always this "test completeness criterium". If this were a black-box test (we don't "see" the code) we could use a tool like Pict33 to propsose the combination of input values.

In this "lab" test we both have design information (desicion table) and our code. The table in this case can help us enough. Each column will represent the values we will supply to the method under test. This means we’ll end up with 16 unit tests to achieve full coverage of our table (and hopefully also of our code). The unit test are there to verify if our “intentions” (i.e. implementing this method with the above rules) were correctly transformed into code. Both test situations (order accepted / order not accepted) should normally be taken into account with these 16 test cases.

We can write out our units tests :


  [TestMethod()]
        public void isOrderToBeAcceptedCase1()
        {
            OrderManager target = new OrderManager();
            CustomerType ClientType = CustomerType.A_Status;
            bool GoodpaymentHistory = false; 
            bool CreditSufficient = false; 
            bool expected = false; 
            bool actual;
            actual = target.isOrderToBeAccepted(ClientType, GoodpaymentHistory, CreditSufficient);
            Assert.AreEqual(expected, actual);
           
        }

       [TestMethod()]
       public void isOrderToBeAcceptedCase2()
       {
           OrderManager target = new OrderManager();
           CustomerType ClientType = CustomerType.A_Status;
           bool GoodpaymentHistory = true;
           bool CreditSufficient = false;
           bool expected = false;
           bool actual;
           actual = target.isOrderToBeAccepted(ClientType, GoodpaymentHistory, CreditSufficient);
           Assert.AreEqual(expected, actual);

       }

        //and so one

Or we make a data-driven unit test.


      [DeploymentItem("TestProject1\\OrderAcceptanceCombinations.csv")]
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", 
            "|DataDirectory|\\OrderAcceptanceCombinations.csv",
            "OrderAcceptanceCombinations#csv", DataAccessMethod.Sequential)]
        [TestMethod]
        public void DataDrivenIsOrderAccepted()
        {
            OrderManager target = new OrderManager();
            CustomerType ClientType =  (CustomerType)(System.Convert.ToInt32(this.TestContext.DataRow["CustomerType"])) ;
            bool GoodpaymentHistory = System.Convert.ToBoolean(this.TestContext.DataRow["PaymentsOK"]);
            bool CreditSufficient= System.Convert.ToBoolean(this.TestContext.DataRow["CreditOK"]) ;
            bool actual;
            bool expected = System.Convert.ToBoolean(this.TestContext.DataRow["AcceptOrder"]) ;
            actual = target.isOrderToBeAccepted(ClientType, GoodpaymentHistory, CreditSufficient);
            Assert.IsTrue(actual == expected);
        }

Bottom line is we achieve acceptable code coverage. We can verify this with the Visual Studio Code Coverage facility of the test framework.

In that respect we can actually reduce the number of test cases and still achieve full coverage. A technique we can use, is "collapsing" the desicion table. If we closely look at our rules, we can detect that several combinations should result in the same result. This means that some conditions are not taken into account (or don’t make sense) in some test cases. For example whether or not a customer with customer-type UnKnown has good creditRecords or even goodpayments records, it doesn’t matter. We don’t trust “Unknown” customers. We don’t allow an order for them! In order words we can reduce the number of test cases for 4 to 1

If we do this exercise for the entire table , we should come up with the following decision table.

Cust-code	a	a	b	b	b	c	u
paymentsOK	y	n	y	n	n	-	-
CredietLimitOK	-	-	-	y	n	-	-

AcceptOrder	y	n	y	y	n	y	n

This would mean we only need to 7 test cases to fully test our rule. Collapsing tables is powerful in terms of test case reduction but can also be “dangerous” while making an error in our reasoning. We could miss out on something. Better some more than too little, I think.

How does Pex help me with testing this class method?

Our pex trigger could look like this :


     [PexMethod()]
        public void isOrderToBeAcceptedTest( CustomerType ClientType,
                                            bool GoodpaymentHistory,
                                           bool CreditSufficient)
        {
            OrderManager target = new OrderManager(); 
            bool actual;
            actual = target.isOrderToBeAccepted(ClientType, 
                                                GoodpaymentHistory, 
                                                CreditSufficient);
            PexValue.AddForValidation("actual",actual);
        }

This parameterized test should reach all our code if certain combination of input values is used. So Pex should be able to work with this test method.

You can launch Pex from within Visual Studio 2008 C# by rightclicking the mouse (or pressing CTRL+F8). You will see a menu option ” Run Pex Exploration”. This will launch Pex in doing it’s job, running your code while generating the right combination of input values for each possible code path and then generating the code for all the test cases.

It is the same number of test cases as with our "manual" approach. So this looks pretty good.


  public partial class OrderManagerTest
    {
        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143741_000()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.U_Unknown, false, true);
            PexValue.Generated.Validate("actual", "False");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143742_001()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.A_Status, false, true);
            PexValue.Generated.Validate("actual", "False");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143742_002()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.A_Status, true, true);
            PexValue.Generated.Validate("actual", "True");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143742_003()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.B_Status, false, true);
            PexValue.Generated.Validate("actual", "True");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143742_004()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.B_Status, true, true);
            PexValue.Generated.Validate("actual", "True");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143743_005()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.B_Status, false, false);
            PexValue.Generated.Validate("actual", "False");
        }

        [TestMethod]
        [PexGeneratedBy(typeof(OrderManagerTest))]
        public void isOrderToBeAcceptedTestCustomerTypeBooleanBoolean_20080725_143743_006()
        {
            PexValue.Generated.Clear();
            this.isOrderToBeAcceptedTest(CustomerType.C_Status, false, true);
            PexValue.Generated.Validate("actual", "True");
        }

    }

We can check the coverage criterium within Visual Studio.

So Pex did an excellent work here. Of course the examples are somewhat academic and only use simple (primitive) types for arguments and/or return values. What about strings or complex types?

My next case study will check out how Pex works on a method accepting a string type. Complex types ....?

Thanks for reading.

Best regards,

Alexander

Pex - test case 2

2008-07-24T16:46:00.022+02:00

Hello,

I'm currently trying out Pex , an automated unit test generator. I'm trying out several scenarios. If you like you can read about a previous test case I tried out. In the same spirit , I'll start of with what you could do without Pex.

The code snippet is as follows. There is a "measurement class" that implements the following method (property).


     private int mTemperature;

        public int Temperature
        {
            get { return mTemperature; }
            set { 
                if (value > -274 & value < 1000)
                {
                     mTemperature = value; 
                }
                else
                {
                    throw new System.ArgumentOutOfRangeException("Please .....");    
                }
               
                }
        }

There are two test situations (different behaviours) in this unit to be tested.

an integer value is assigned to a field

an exception is thrown

If you would use the Equivalence class technique to establish a good test suite, you could come up with following findings.
Valid classes

integer value

integer value between -274 and 1000

Invalid classes

NOT an integer value

integer value smaller or equal than -274

integer value bigger or equal than 1000

Because the .NET type system gives us a hand (int) , our method only can accept integers. So we only have 1 valid class and two invalid classes or one if you would combine then in one sentence (Not between -274 and 1000). Both approaches should cover our two test situations (assignation and exception).

Now we have to come up with actual values for these classes which shouldn't be too difficult. This would result in two physical test cases (i.e. unit tests). We could pick -30 and 1500 . Or -650 and 0. And so one... All these values should give equal test results while assuring complete coverage.

In case of two invalid classes you will have three test cases :

valid : 25

invalid 1 : - 300

invalid 2 : 1250

A common place for errors are the comparison signs. The Border Value Analysis "technique" stipulates you should pick values around the "borders". In our example these are -274 and 1000. In addition you could also pick values aboove (and/or under) these border values. Any mistake (current or future) should be detected by our set of test cases (i.e. unit tests).

valid 1: 273

valid 2: 999

invalid 1 : - 274

invalid 2 : 1000

So which unit tests will Pex generate for us?


         [PexMethod()]
        public void TemperatureTest(int aTemperature)
        {
            measurement target = new measurement();
            int actual;
            target.Temperature = aTemperature;
            actual = target.Temperature;
            PexValue.AddForValidation("actual",actual);
        }

Pex, using the specified PexMethod, explored our code. It took him two runs to come up with the necessary values for temperature. On the last value (int.MinValue : the smalles possible 32-bit integer value in .NET ) it discovered an exception. Just like in "traditional" unit testing you can specify an "expected exception" . meaning when this exception is thrown, the test is succesfull because that is what you ....expected.

If you re-run the PexMethod again you will see two classic unit test.


   public partial class measurementTest
    {
        [TestMethod]
        [PexGeneratedBy(typeof(measurementTest))]
        public void TemperatureTestInt32_20080725_080720_000()
        {
            PexValue.Generated.Clear();
            this.TemperatureTest(0);
            PexValue.Generated.Validate("actual", "0");
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentOutOfRangeException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void TemperatureTestInt32_20080725_082344_001()
        {
            this.TemperatureTest(int.MinValue);
        }

    }

If you run these two unit tests and enable Code Coverage for the target assembly in VS Test framework, you will notice full coverage as understood by VS test framework.

If you compare the Pex analysis with our "manual" analysis you can see that Pex took the minimal set of test cases to achieve full coverage (as understood by Visual Studio). For example someone would change the comparision operator in the method, the unit tests generated by Pex will not detect it in this particular example.

But if you would slightly change the pexMethod (tip from Peli de Halleux) you would see one extra unit test. You need to re-write the assert to verify the border values.


      [PexMethod()]
        public void TemperatureTest(int aTemperature)
        {
            measurement target = new measurement(); 
            int actual;
            target.Temperature = aTemperature;
            actual = target.Temperature;
            PexAssert.IsTrue((actual > -274 & actual < 1000));
        }

The unit tests generated by Pex would now include the 1000 border value. I don't understand yet why -274 wasn't picked up. I guess it has to do with the fact 0 was the value chosen in the first run. Maybe there are better ways to write this PexMethod? I'll will need to look into the "assumptions" part of Pex (PexAssume class).


    public partial class measurementTest
    {
        [TestMethod]
        [PexGeneratedBy(typeof(measurementTest))]
        public void TemperatureTestInt32_20080725_111609_000()
        {
            this.TemperatureTest(0);
        }

        [TestMethod]
        [ExpectedException(typeof(ArgumentOutOfRangeException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void TemperatureTestInt32_20080725_111609_001()
        {
            this.TemperatureTest(int.MinValue);
        }

        [TestMethod]
        [ExpectedException(typeof(PexAssertionViolationException))]
        [PexGeneratedBy(typeof(measurementTest))]
        public void TemperatureTestInt32_20080725_111953_002()
        {
            this.TemperatureTest(1000);
        }

    }

Is it worthwile to look for test cases but beyond absolute minimal? Well, it dependends. One of the advantages of of having (automated) unit tests is the ability to use them over and over again. In the context of regression testing these test suitses can be of great help to assure that all code that is using your unit (i.e. method) will still function after you made modifications. or least you will know in which situation something will break so you can take appropriate measures (defaults, overloads, release notes ,...).


         private int mTemperature;

        public int Temperature
        {
            get { return mTemperature; }
            set { 
                if (value > -274 & value <= 1000)
                {
                     mTemperature = value; 
                }
                else
                {
                    throw new System.ArgumentOutOfRangeException("Please ...");    
                }
               
                }
        }

So if regression testing is of high priority , it think some methods need those extra test cases. They can happily live beside the ones generated by Pex. It is sometimes (value of software; money, safety) better to have "too much" than "too little" test cases I think (even if it means extra (test) code to maintain.

As shown in the example, the way you setup your Pexmethod will influence how Pex "explores" your code. Although the the call to the (setter) method was the same. The Assertion part of the Pexmethod made Pex do an extra effort.

If you have other thoughts or remarks about pex or devloper testing in general, don't hesitate to drop a comment.

Thanks for reading.

Best regards,

Alexander

Pex - test case 1

2008-07-23T14:21:00.028+02:00

Hello,

I just started investigating Pex . In order to learn the tool I'm using several (academic) test cases to see how Pex can help there.

This case case is called Commercial Cut percentage for an order. In our Order system we have implemented a rule that will calculate the commercial cut based on the total OrderAmount. The following table depicts the rules for applying the commercial cut.

Total (rounded) order-amount (€)	<=1500	[1501-5000]	[5001-10000]	[10001-25000]	>25000
Commercial cut	0	2	3	4	5

I have implemented this rule in a c# method. For the sake of this exercise I will not argue about whether or not this should be a private method (Which by the way can perfectly be unit tested in the VS Test framework).


       public uint DetermineCommercialCut(uint aTotalOrderAmount)
       {

           if (aTotalOrderAmount <= 1500)
           {
               return 0;
           }
           else if (aTotalOrderAmount > 1500 & aTotalOrderAmount <= 5000)
           {
               return 2;
           }
           else if (aTotalOrderAmount > 5000 & aTotalOrderAmount <= 10000)
           {
               return 3;
           }
           else if (aTotalOrderAmount > 10000 & aTotalOrderAmount <= 25000)
           {
               return 4;
           }
           else
           {
               return 5;
           }
       }

Of course we will unit test this class method. This method should always return some positive integer value back depending on total order amount that is supplied via unsigned integer parameter. So I give a positive integer (or zero) to the method and I receive a positive integer (or zero) back. Of course the purpose of the method is determine the commercial cut percentage based on the order amount. Our table showed on the rules we need to apply and our code is (hopefully) constructed in the same way. So we will construct a unit test that will call this method.


   [TestMethod()]
   public void DetermineCommercialCut_ZeroOrderAmount()
      {
            Order target = new Order();
            uint aTotalOrderAmount =0;
            uint expected = 0;
            uint actual;
            actual = target.DetermineCommercialCut(aTotalOrderAmount);
            Assert.IsTrue(expected == actual);
   }

We could continue with testing all positive integers up to 4.294.967.295. Of course this is not worthwhile. We can pick certain values for this entire collection in such way that any value within certain ranges of this big collection , will (should) behave in the same way. This is referred to as equivalence classes in testing literature. So whether I pick 0 or 1500 as total (rounded) order amount or any value in between, it should always resolve to a return value of zero. So based on this principle I can detect several equivalence classes.

Must be positive integer

Integers between 0 and 1500 included

Integers between 1501 and 5000 included

Integers between 5001 and 10000 included

Integers between 10001 and 25000 included

Integers higher then 25000

I’m spotting 5 valid classes. Because the.NET type system helps me out here I don’t need explicit checks on the invalid class namely negative integers.

now I need to pick a value I will actually use in my unit tests. I will at least need to pick 5 values, one from each equivalence class. In my test I will verify if the reduction cut is appropriate for that value.
So instead of making 4.294.967.295 unit tests I only need 5 to “cover” all possible test situation ( different commercial cut percentages).

Another technique that is usually combined with the former is border value analysis. In the example I can with a minimal additional effort also verify if the comparison signs are as they were intended to be. Instead of picking any value in the equivalence class , I will look up the values that are used in the conditions for example. Instead of 2531 , I will pick 5000 just at the border. So 5000 should give me a commercial cut of 2. Just like 2531 but I have an extra check on the comparison sign with no extra effort. With a little more effort I could also include the values just under (or above) the border.

So basic techniques told me I need 5 test case to fully test my rule (i.e. Method). So this means 5 Visual Studio test methods. Actually , Visual Studio test Framework can, in this case , help us even further with reducing the number of unit test to …..one. That is one data-driven unit test. What we do, is externalize the test data values into for example an Excel file. This data-source will then “feed” our data-driven unit test with actual values. So although we have only one written unit test, we would like this test code to be executed 5 times. We now use this table (with some modification) as data-source for our data-driven unit test in Visual Studio 2008. For more info see MSDN.


        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML",
                     "DataDirectory\\testData.xml",
                     "CommercialCut", DataAccessMethod.Sequential),
        DeploymentItem("TestProject1\\testData.xml")]
        [TestMethod()]
        public void DetermineCommercialCut_DataDriven()
        {
            Order target = new Order();
            uint aTotalOrderAmount = System.Convert.ToUInt32(this.TestContext.DataRow["input"]);
            uint expected = System.Convert.ToUInt32(this.TestContext.DataRow["expected"]);
            uint actual;
            actual = target.DetermineCommercialCut(aTotalOrderAmount);
            Assert.IsTrue(expected== actual);
        }

We determined on basis of our analysis that 5 test-cases should cover the testing of the rule. Here Visual Studio Test framework also can help us by instrumenting the code before executing the unit tests (Code coverage). A nice feature of the Visual Studio Test framework is the colorization of the code under test to show which parts have been “touched” by the unit tests and which parts didn’t.

When looking at the Visual Studio Code coverage results of one test executed :

You can see the statements in different coloring; touched versus not touched. This should tell you that you must add some test cases to see code paths in your method also work as you intended and consequently coded.

If you would run the data-driven test with the 5 test cases you would get the following result :

Everything is highlighted in blue, meaning that every “block” in our code has been executed during the run of our unit test. That should come as no surprise because we rigorously followed our table for our rule.

So far our “manual” effort . Lets see how Pex can help you. So the documentation states that Pex can create a test suite with a high coverage by exploring your code triggered from a parameterized test you will have to write.

I highly recommend reading the tutorial for more background information on Pex. Meanwhile I will continue with the example to show you how Pex can create a test suite for us.

First we must create a “parameterized” unit test to give Pex an entry point into our code under test. “Parameterized Unit Tests (PUTs) are... tests that take parameters. Unlike traditional unit tests, which are usually closed methods, PUTs take any set of parameters. Is it that simple? Yes! From there, Pex will try to generate the (minimal) set of inputs that fully coverthe code reachable from the test. “

So in our case this could look something like:


[PexMethod()]
public void CaluclateReductionPercentageTest(uint aSales)
{
 Order target = new Order();
 uint actual;
 actual = target.DetermineCommercialCut(aSales);
 PexValue.AddForValidation("result", actual);
}

You can launch Pex from within Visual Studio 2008 C# by right clicking the mouse
(or pressing CTRL+F8). You will see a menu option ” Run Pex Exploration”.
This will launch Pex in doing it’s job, running your code (several times) while generating the
right combination of input values for each possible code path and then generating the code
for all the test cases.

Pex generated 5 tests for us. It clearly also detected (in another way) that there are 5 test cases needed to cover the code. If we take our 5 equivalence classes we can see that this is correct. If our “conduct-of-test” specifies the use of border-value analysis where appropriate , then these values are not so well chosen in my opinion.

Lets say that we want to add an extra level of commercial cut for sales between 1000 and 1500 Euros. And we want to let our client enjoy a commercial cut of 1 percent.

In the manual test analysis we would come up with an additional equivalence class. Pex can also detect this.

But first lets re-run our initial set of unit tests generated by pex.

Our Code Coverage run should warn us that we missed out on some parts of the code. We need another unit test. To do so , we will re-run Pex on our pexMethod. The will force pex to examine our code again and it should find an extra path in our code.

Doing so resulted in pex finding an additional test (nr 3 - see screenshot). If we run the generated test methods and also look at the coverage findings we find a positive result.

So in a nutshell. In this (although academic) situation, Pex found the same test cases if we would use a manual technique. Border value Analysis would be advantageous in this test case but Pex doesn't propose these values while exploring this method based on the PexMethod we proposed.

So the most important lesson I think is , as also stated in the Pex FAQ : Use Pex as one of your tools not as the (only) tool.

Maybe you also have tried out Pex? Don't hesitate to share....

Thanks for reading.

Best regards,

Alexander

ps: I'm still experimenting with SyntaxHighLighter. If you have any tips how to improve showing code snippets in Blogger...let me know.

Starting with Pex (Program Exploration)

2008-07-22T15:06:00.004+02:00

Hello,

I recently downloaded Pex from Microsoft Research. Pex is a tool for finding test cases for you on basis of “looking” and “executing” your code. It is fully integrated in Visual Studio 2008 (c#). Pex explores your code by executing special Pex test method you have written. It basically comes down by making a “parameterized” version of a unit test you would normally make. This Pex test method will then call you unit under test (i.e. class method). Pex supplies values for the parameterized unit test . Meanwhile it will instruments your code do see which conditions it needs to check and which code paths it took during the exploration phase. After completing it adventure, it will generated unit test code. This unit test can then be executed by the Visual Studio Test Framework. For more information..... Before taking Pex for a spin , let's look at why a Pex was created.

Automated Unit testing is considered a good practice in software development. Nevertheless it is usually perceived as a burden by the developer. Besides the writing the actual code the programmer has to create and maintain this extra test code and of course exercising these unit tests. Depending on the code the amount of test cases , hence the unit test, can quickly grow. Also if you exercise the practice of “good” code coverage by your unit tests, you’ll discover that writing unit tests is hard work.

Sometimes the amount of test code supersedes the code under test. Much effort goes into creating the right amount of test cases so they cover every “corner” of you code. This code coverage criterion should result in a higher probability to find errors in your code. There several techniques that can help you establish these test cases. But these still leave writing the unit tests yourself.

Here's where pex steps in. Pex from Microsoft research can help achieve this coverage criterion by dynamically introspecting your code and based on that analysis can generate different test cases in the form “traditional” unit tests for the Visual Studio test framework.

I currently trying out some scenario's. Keep an eye for future posting...

Thanks for reading.

Best regards,

Alexander

Design for Testability - Dependencies

2008-07-17T09:16:00.005+02:00

Hello,

This post is a continuation on previous posts about design-for-testability (here and here).

Control and observation are important characteristics in test-automation. If our code under test makes an effort to facilitate this , we improve the testability of our code. Improving the testability should lead to find any errors in our code more easily. The more we errors we find early in the development cycle, the cheaper it is to correct them. We early testing and defect correction, we get a good feel on the quality of the software. We can make educated decisions on when to promote the code to QA and/or production. When our customer can use our software to do their job , they create value for the company they work for ….(I will stop rambling here) . But you see , testability (and in a broader sense quality assurance) should be baked into the software. The entire organization can benefit from it.

Let’s start on the “control” aspect of our code under test (CUT) and specifically on the controlling the dependencies of our CUT with other parts of the program or the environment it runs in.

It is in the nature of an object-oriented language to build up a program from lot of different “units” where some “units” or better classes work closely together to make something happen. Object-oriented principles like single-responsibility, information-hiding , inheritance , etc are common in this world. So it is quite natural that classes have dependencies on other classes. The term coupling is often used to state the existence of relationships between class (inheritance, composition , etc). coupling is evitable. If there were no coupling we would end up with monolithically part of codes, actually they would be separate programs. The large classes would be too complex and the cohesion of the operations and member data would very low.

In the realm of testability we sometimes want to be able to test a class in a controlled way, maybe even in total isolation. In order to do that we must be able to control the dependencies. The fewer the dependencies the easier it will be. Hence “managed” coupling can increase testability.

Programs run on machines and make also use of their environment. For example the file system, Message Queue system, database systems , web services, etc. These resource dependencies are expressed in the programs usually via string tokens. For example a folder name, database connection string, queue name, web service URL etc. A good practice of course is to externalize the values for these resources for example in a configuration file. These practice is also beneficial for testability. Avoiding “hard-coded” resource strings in your code is of course not a good programming practice. Because many times the resource string are depended on where the program actually will run. For example development , production or …testing. So controlling the resource tokens outside the program can give us flexibility will setting up tests.
.NET has excellent support to easily consume these externalized resource identifiers in your programs. In the Visual Studio test framework you can easily let the test framework pick up these configuration files and let your code under test use the configured element just like if the code under test was running in their proper host (windows app, web service, etc). In other words , you will copy the configuration info that you need for the code under test from the base app.config or web.config in your test project and modify it as needed.

Let’s focus again on class dependencies. So dependencies are inevitable but we can make them more manageable for testing purposes ( or other reasons). In order to continue our discussion lets introduce some terms. If a class A depends on another class B to fulfil its functionality A is called a client class and B is called a server class. During unit test we want to test a class in isolation. Therefore we want to substitute its server classes by stubs or mock objects . Hard-wired dependencies to server classes hinder this. A dependency on a server class is hard-wired if the server class can not be substituted by some other class (e.g. a subclass of the server class) without changing the source code of the client class. Similar problems arise during integration testing if a server class should be stubbed because it is not test-ready. So how can we manage class dependencies that will benefit testability.
One of the areas lies in the construction of the dependencies. We don’t control them. So our solution strategy should revolve around getting control about the creation of the dependencies. Several techniques can help us to come up with a solution:

Interface-based programming : Separating the interface from the implementation is core principle. Our client class uses an abstraction of a server class (interface) , not a particular implementation of it (object). So during testing we could create another implementation of the server class that implements the correct interface. This “test-double” (Gerard Meszaros , xUnit patterns) version of our server class can for example always give a fixed set of values back or just ignores the fact that we ask it to do something. Interfaces are first-class citizen in .NET so you should have no problems with this. But this technique is just a first step toward the solution. We still need to hook up an implementation of the server class into the client class at test time.
Service locator: We would delegate the creation of objects to a specialized object that for example would fabricate objects based on a string identifier.
Provider model : Variation of Interface-based programming :. Abstract class , concrete implementation hooked up at runtime (configuration file). You can a test-specific implementation of the provider in order to control the tests
Manual Dependency injection : We will make another class responsible to establish the link between the client and the server. We will add a parameter of the server type to a method of the client class (a constructor, or a method that requires access to a server instance, or a dedicated setup method) which allows other objects to set the link. During testing the parameter(s) can be used to establish a link to a “test-double” instead.
Automatic dependency injection : a DI container framework takes care of the details of which objects are interconnected, so you can build each one independently. No need for passing the dependencies along with the constructor or methods or assigning properties with dependent objects

Still there some concerns (or trade-offs) you should take into account;

Information hiding :Why can’t leave it up to A to know its dependencies : Exposing the dependencies in a constructor function can be viewed a violation of encapsulation because now a client of Class A would have to create instances of Class B first before calling the constructor function of class A
The ability to substitute a server class is not always important for the “production” implementation. So is testability a good enough reason for implementing this possibility?
We could make a special constructor that will accept the dependencies. The default constructor would use that hard-wired implementations.
If there are dependencies on many server classes, this approach would result in too many parameters.
DI Framework to the rescue ...but Someone has to know the dependencies
Assembler , DI Framework via declarations or via config files

There are still other things besides managed coupling you can work on to improve testability , hence the quality of our software : cohesion, encapsulation, ... But that's for another post.

Maybe you have others techiques to improve testability? If you would like to share them, don't hesitate to drop a commont.

Thanks in advance.

Best regards,

Alexander

Winforms (VS2005) DataGridViewComboBoxColumn, CTRL+0 and Cannot set Column 'myColumnName' to be null. Please use DBNull instead.

2008-07-15T14:19:00.017+02:00

Hello,

I recently had some trouble with setting a dataGridViewComboboxColumn to a dbNull value with the CTRL+0 (zero) keystroke. As you may know, the datagrid supports the CTRL+0 keycode to force a dbNull value into cell and subsequentialy into the bound datasource (a datatable in my case). In other words you can use that to clear out a combo box's value and set it back to "nothing". At least when the bound datatable column allows DbNull to be true.

This is where I experienced some strange behaviour. I used a datagridView to display some customer data. One of the fields specifies an optional characteristic , namely the qualification of a customer. At database level this is enforced with a referential integrity constraint in the form of foreign key specification with a look-up table. The primary key of this look-up table is an integer. Of course in the datagrid , on the comboxbox showing the customer qualification , you will see a description (cfr. valuemember versus DisplayMember).

This is where it goes wrong. When I try to clear an existing value in the combobox , I get the following error message. Cannot set Column '' to be null. Please use DBNull instead.

What is even more strange is that when you have a combobox in the datagridview where the valueMember's type is System.String, You will not have this exception. It just works. It is also only apparent with a datagridViewComboxBoxColumn. Because if I would use a
DataGridViewTextBoxColumn for the "integer" case, I would neither experince any problem with CTRL+0.

I looked for a solution on Google and I stumbled on this post . Somewhere on the middle you will see an entry by Mark Rideout , DataGridView Program Manager Microsoft (27 Jan 2006, 3:34 PM ) . ...And he talks about a bug. But more interssenting for me he also hints a possible work-around.

He talks about overriding the DataGridView.ProcessDataGridViewKey Method . This method calls the key-processing method appropriate to the key pressed. In this method you could check for the CTRL+0 keystroke (D0) . That is exactly what I immediately tried out. I made a custom DataGridView (i.e. I created a class inheriting from the System.Windows.Forms.DataGridView and overrided the ProcessDataGridViewKey(see MSDN) . In this method I check for the CTRL+0 keystroke and some other characteristics . And if everything is true, I programmatically set the value for the currentcell to system.DbNull.Value.


Public Class CustomDataGridView : Inherits System.Windows.Forms.DataGridView


    Protected Overrides Function ProcessDataGridViewKey(ByVal e As System.Windows.Forms.KeyEventArgs) As Boolean

        If (e.KeyCode = Keys.D0 or  e.KeyCode = keys.NumPad0) _
            AndAlso TypeOf MyBase.Columns(MyBase.CurrentCell.ColumnIndex) Is _
              System.Windows.Forms.DataGridViewComboBoxColumn _
            AndAlso MyBase.Columns(MyBase.CurrentCell.ColumnIndex).ValueType IsNot _
              GetType(System.String) Then
            MyBase.CurrentCell.Value = System.DBNull.Value
            Return True
        Else
            Return MyBase.ProcessDataGridViewKey(e)
        End If


    End Function
End Class

And guess what? It works (at least for me)! I also tried it in VS2008. There the same error is thrown ( I gues the bug report got lost somewhere ). But the same trick with ProcessDataGridViewKey works in VS2008.

I haven't tried it yet on other dataTypes like guid's and so. That why i check for datatypes in the code. But for my situation, this solution is OK .

Maybe there is another solution? If you know of one, don't hesitate to let me know...

Meanwhile if you like, you can download my test-program.

Thanks for reading.

Best regards,

Alexander

Design for testability - developer testing context

2008-07-09T15:12:00.001+02:00

Hello,

Some more thoughts on design for testatbility in the developer testing context (see previous post ).

Test automation plays an important role in the developer testing context. The advent of the xUnit testing concept , nUnit and the integration of xUnit concept in Visual Studio, has put developer testing more in focus.
The test case is a description that tells us how we can see whether to system has done something as we expected it do that something.
Before executing a test , we must put the put the system in state so our test can actually operate. For example before testing the deletion of an order in our order system, the system must contain an order that we know of. For example a order record in a database.
The act of deleting an order necessitates that we can say to the system that we want to delete that explicit order . So not only to we need control of the initial situation, but we also need control of specifying the input and controlling the place where the order is physically kept (i.e. our test database with Order record with ID ORD1)
After executing the functionality, we must be able to verify if the system has actually done its piece of work. We must “see” the result of its processing. We need to compare this outcome with our initial expectations. A discrepancy can mean either the behaviour of the system was not correct or test case was not correct in terms of initial situation , the actions we took or the expected results.

An automated test (case) in xUnit terms can be viewed as follows (freely adapted on drawings found on http://xunitpatterns.com/ (Gerard Meszaros)).

Automated means that important steps in the execution of a test are preformed “automatically” . In other words exercising a test case on a unit (a method ) involves programmatically

supplying the initial situation (setup)
doing the actions (execute)
comparing the actual results with expected values (assert)
Cleaning up (teardown) so the next test case can proceed in a clean situation.

So how can we improve the testability in a developer context. On which aspects of the code can we influence the ease of performing developer test and facilitate the localisation of defects.

Control : In order make automation possible we need to be control of the initial situation as of the all the input that a method needs, so we can steer the processing of the unit in such a way it will produce the outcome we expect.
Observation : In order to compare actual and expected results , we must be able to observe what the outcome of the processing of a unit is.
Complexity: Network integration, database integration, Message queuing, Security , Registry, Windows services, COM+ , …. Makes it difficult to set up a test environment.
Large methods with complex conditional flow necessitates many test case in order to get desired test coverage.
Inheritance : abstract base class can not be instantiated.
Large inheritance tree : may test cases to see if combination of base and specialized code works as expected.
Isolation: Ability to isolate certain dependencies and replacing them with test-doubles improves testability because we control what the test-doubles will do. That way we can concentrate on the logic in the CUT . The DUT will replaced by a test-double. Any calls from the CUT to the DUT (replaced by the test-double) will result in “controlled answers or behaviour”. Hence our test case for the CUT will be easier to setup
Separation of concerns : The class with a clear responsibility should improve testability in terms of knowing what to test in the first place.
Smaller things to test
Heterogenity : Single language or language framework improves testability

If you have remarks or other thoughts, don't hesitate to drop a comment.

Best regards,

Alexander

Design for Testability

2008-05-23T11:18:00.000+02:00

Hello,

I have been reading up on "design for testability" because some of the code I deal with is not always easy to test. Some thoughts ....

Context

“Business” wants software application to help them do their job. They use them because they bring “value” to them. In order to make that happen software application must do what they are required to do (fit for purpose) and must be “reliable”.

Through a mixture of process guidance, technology and human skills, to use the previous to elements efficiently and effectively, IT tries to deliver those application on time while adhering to the intrinsic quality attributes like for example reliability. Quality processes are put in place to ensure achievement. In order to assess the if the quality attributes are indeed incorporated into the software product, we need to check it. This is where testing comes in. Testing should give us insight into the quality of a system. This insight should help us decide to release the software for usage by the business because it can bring value to the business. Or it can tell us that there are still some discrepancies and we need to improve quality before handing it over to the business.

Testing is some something we must do although it does not immediately raises the quality of the system. It reports only. Many times “business” can not understand why we can’t deliver something right the first time. Testing costs money! Software creation is still an error-prone endeavour for many reasons despite all advancement in processes , technology and skills of people. In other words we start off with the assumption (certainty!) that software contains errors. Both functional and programming wise. Testing is about finding those. So it is important that we make the best possible test cases to reveal them. Use tooling to support our testing effort etc. Bust beside the test process it self , we can do more.

During the inception and construction of software we can introduce measures to make this testing effort even more efficient and effective. In other words we must make our system as “testable” as possible in order to and to facilitate performing the test and finding the errors.

Testability

Testability is about the ease of performing test against your system and ability of the tests to reveal any defects in a certain context. The context of the test , system testing (functional and non-functional) or developer testing, puts different meanings to testability. Also is it important that other software artefacts beside code, affect testability ; for example the requirement specification documents.

These are some aspects that influence the testability :

Understanding: The better you know what the thing is supposed to do , the better you can test for it.
Controlling ; You must be able to manipulate the input , in order to verify the correctness of the output.
Operational : We can only test it when it works !
Visibility : You can only verify the result , if you can see the output.
Simplicity : The less there is to test , the more quickly we can test it
Stability : The fewer the changes to less test if need to re-do and/or re-write

So manipulating the influencers, can change the testatbility !

It is important to acknowledge that testability does not come on its own. We must take testing into account from the beginning of our software development endeavour. Because we know our software will contain errors (this will only increase with the size and complexity of today’s systems), we must prepare for it to easily test our system and find them before we ship the code.

In other words we must design for testability. It is important because testing costs money. Also faulty software systems can cost the business money (in the best case or worse in case of threat to safety or life) . Therefore we must make the testing as effective and efficient as possible. But Business will benefit from this as well in the long run because we should be able to find defects thus raise the quality of the system, thus raise the value to the business.

Also future changes to the software to incorporate new requirements will benefit from this testability feature because we should be more confident to change working code with regression testing. Also external factors such as SOX compliance make it important for the business to assemble test evidence. Testable systems certainly can aid .

Design For testability

To increase both the ease and value of testing we can tack different areas of the software development process and the software product itself. For example

The software development methodology (“testable” requirement concept, etc)
The software technology (Procedural vs. Object-Oriented, Mainframe, etc )
The software architecture & design (Layered vs. monolithic, Coupling , Encapsulation , etc)
The code (Assertions , mocks/stubs , etc )
Tool support (Automation , Xunit framework , etx)
The developer (Training and education , code reviews , etc)

This broad spectrum calls for an holistic approach of course. The test context will put certain focus in that approach. For example

For functional testing you need good test reference in form requirements documents to establish your test cases
For System testing , you must be able to let system work under operational condition together with data and sometimes external systems.
For developer testing , you must sometimes be able test your unit in isolation and sometimes you would like to integrate a set of unit together.

Currently I'm interested in the developer testing context . So if you have experiences in raising the testability of your code and wish to share them , please do so. I would to hear from you how dependency injection help you ( or not) , how unit testing helped you find defects easily , etc.

Best regards,

Alexander

Application taxonomies

2008-05-13T11:58:00.000+02:00

A classification of software application can help you to think about an architecture and/or determine a reference architecture in your catalogue.
Multiple characteristics can be used to set up a taxonomy. These taxonomies can help you I think because it sets up a context for making a software architecture and the usage of reference architectures. Usually one classification criterion alone doesn’t fit the entire application . Multiple axes.

I think one of the main message would be : Don’t be fooled that one reference architecture can be used in all scenarios.

Front-end technology
o Windows (classic / touch screen)
o Web-based (classic / Rich Internet client )
Distribution
o Desk-top
o Mobile/PDA
o Sever
o Distributed
Connected
o On-line
o Disconnected with on-line capabilities (smart client)
Communication
o Asynchronous
o Synchronous
Target
o Consumer ( E-commerce, PDA)
o Business (Traditional administrative data keeping or/and decision support)
o Industry (MES, CIM, MRP, SCADA, real-time systems, PLC, Robotics)
o Government E-government
o Entertainment (Games)
Connectivity
o Internet
o LAN
o WAN
o VPN
Function
o OLTP applications
o Reporting applications ,Data warehouse, BI
o Batch application
o Service–oriented apps
o Mobile device applications
o Integration Applications ( ERP, Mainframe , etc)
o Office applications and Office integration
o Portals
o System software
Organisation boundaries
o Departmental
o organisation
Usage pattern
o Collaboration
o Transactional, data keeping
o Decision support
Time pattern
o Long running “transactions” (Workflow)
o Short running business transactions
Expected life-time
o Short-term
o Long term
Business area
o Medical
o Financial
o Government
o Industry
o Commerce
o Etc
Saas (Software as a Service ) software.
Etc.

What do you think?

Best regards,

Alexander

Software architecture decisions

2008-05-13T11:42:00.001+02:00

Hello,

I recently had to give a definition of what a software architecture is . Not always that easy because Software Architecture is a much loaded term. If you want some formal definitions what a software architecture is , I recommend reading the information on http://www.sei.cmu.edu/architecture/published_definitions.html#Modern

General tone in these definitions is that you need to make high-level decisions about the system you' re going to build;

What style are you going to use? What is the structure?
How is it going to function? How do structural components of the architecture work together?
How does it meet the needs of all the stakeholders?

These high-level decisions have a high impact on the application. You can not change them easily if wanted or needed. Unless you’re prepared to put enough resources on the table.

In the bullets below I tried to sum up the different aspects of a software architecture. This list is not intended to be exhaustive.

Windows front-end or web front-end ( Rich Internet clients)
Layering, organization of code, separation of concerns,
Representing “business objects” persisted in a database
Exposed domain model or using Data transfer Objects / Presentation Model
DataSets versus classic classes
Organization of “back-end” logic
Organization of “Presentation” logic
Organization of “data-access” logic
Distribution or not
Remoting technology
Authentication, authorization, Audit
Data binding
Synchronous / asynchronous
ORM tools or raw ADO.NET
Multi-language
Exception handling versus status code
Instrumentation /Logging
Configuration
Caching
Database schema fixed or freely adaptable to need of design designs
Long-running “transactions” (conversations)
Reporting needs
Validation rules placement (Attribute based, Intra object based, Inter object based)
Etc.

It is important to take holistic approach towards the desicions. You will need to take some trade-offs and influencing criteria into account.

Kind of application built (taxonomy)
Tooling can constrain how you organize your application (code generators, frameworks, application building blocks, software factories ,etc)
Availability/knowledge of people can constrain the “feasibility” of your architecture
Process (top-down versus bottom up, behaviour-centric versus data-centric)
External / internal regulation can influence your architecture (SOX, internal Audit rules
Availability of reference architecture.
Non-functional requirements
Patterns
Time-to-Market
etc.

If I missing some points or you want to comment on this. Please feel free.

Best regards,

Alexander

Comparing to Linq to SQL

2008-04-23T14:34:00.000+02:00

Hello,

Recently I took some time to investigate Entity Framework (CTP3) and Linq to SQL (RTM) further. While going through the examples I compiled following list with benefits & concerns. Maybe you have made a list for your own? Maybe you would care to comment. Please do so!

Linq to SQL

Benefits

Already available in VS2008 RTM
Generation of partial classes based on the database schema
Relationship are also detected and modelled as special properties in the entity classes
You work with these entity classes in you application.
Generated classes can be augmented with custom methods
Generated classes can be augmented with partial methods
LINQ support to specify queries
Dynamic generation of SQL (you do not have to write SQL your self)
Databinding support
Lazy loading (default) and immediate loading flexibility
Updating only changed columns
Optimistic Concurrency support through mapping directives.

Concerns

Basis mapping capabilities ( 1 table -> 1 Entity)
Only SQLServer
Distributed application requires special attention (dataContext)
Designer lets you start from a blank model but I haven't found something to create a database schema from the entity model.
Serialization of entity classes only set if generated with SQLMetal command-line tool option.

Entity Framework

Benefits

Generation of partial classes based on the database schema
Designer lets you start from a blank model but I haven't found something to create a database schema from the entity model.
You work with these entity classes in you application.
Generated classes can be augmented with custom methods
LINQ support to specify queries
Dynamic generation of SQL (you do not have te write SQL your self)
Databinding support
Normally foreseen to accommodate multiple database systems
More flexible mapping capabilities than Linq To SQL (fine-grained entity model, Inheritance mapping flexibility, etc)
Generated classes by default serializable
Data driven Object-modelling point of view
Lazy loading (default) and immediate loading flexibility

Concerns

Promised to be shipped with SP1 VS2008
Distributed application requires special attention (ObjectContext)
Designer lets you start from a blank model but I haven't found something to create a database schema from the entity model.
In the designer you can not add behaviour in the model( somewhat normal because it is a DSL for data persistence but still....). Entity are now more geared towards a data-container role.
Two flavours to query the databases system (linq to entities and entity SQL). At this moment I'm not very sure when to use which.

So if you have other points of interest or remarks , I would like to hear them.

Thanks in advance.

Best regards,

Alexander

VISUG Event (03 April 2008): WCF - Beyond endpoint - Juval Löwy

2008-04-05T16:18:00.000+02:00

WCF : A brave new world

The Belgian Visual Studio User group (VISUG) had invited no one less than Juval Löwy to present a session on WCF Juval is an authority on the subject and is/was? a member of the advisory comitee of WCF. From the summary on the VISUG website (http://www.visug.be/) you could tell that this wasn't going to be your typical WCF 200 type of presentation (or 300 or 400 for that matter).

Juval immediately set the tone of his presentation (or should I say show) by stating that WCF is a better .NET . WCF will deprecate .NET. During the next hour and a half it took the audience the numerous of demo apps to make his point.

WCF is the next development platform according the Juval. He pushed it a little bit further by saying Microsoft is masquerading this "Truth" from us. He mentions the whole marketing message Microsoft put behind the start of the first version of .NET : .Net is the platform to build web services. We all learned in time that there is much more than web services to .NET. This was primarily done to soothe the organisations and companies that just spent tons of money in making their developers COM- savvy and converted their applications to VB6 / C++ ATL/ COM + . Don't worry, people. We have got this new thing here but it is for web services. Don't be alarmed. By the way we have created VB.NET . You already know half. And yes there were migration wizards and .NET - COM interoperability features. In the end .NET was not so easy ...

Comes along WCF: ....It is a new breed of communications infrastructure built around the Web services architecture ... ( MSDN site) Again they stress onto web services (although the term connected systems is more present in the WCF literature on MSDN) . Again to soothe the companies that have invested in their developers to learn ASP.NET web services but also other technologies like Messaging (MSMQ) , .NET remoting, Enterprise services (COM+). Again ...don't be alarm folks, it is only about web services.

Juval has more visionary message to tell regarding WCF. We are standing again on the foothill of a big mountain. We at the brink of a new paradigm shift. A service-orientation, declarative program model we you , if you like, also can create web services. But the granularity of what a service might be is not solely at level of web services. According to Juval the interception mechanism of the WCF run time can be used at every class. WCF everywhere is the nirvana. At least to Juval. I don't know if he really meant that or just wanted to keep us on the edge. I think the message was as your class need certain services (security, reliability, transactions, etc) And WCF can offer them , why need turn your class into a "service" and leave to plumbing to WCF.
To me that was the thing I learned from the presentation. Don't ever write plumbing code yourself, leave it to the experts. Concentrate on the domain logic.

With a few smart examples he showed us the essence of what WCF can give to you . And the sheer power you sometimes get by just changing the config settings or changing an attribute on a class ( or should I say service). Some of things you of course could do with regular .NET but the amount of effort and knowledge you need to have to do so are so big ,that it can be difficult to defend not using the out-of-the-box features of WCF. And don't talk about the testing effort and maintenance effort you have to put into all this plumbing code (your customer hasn't asked you for in the first place. After the session we al received the WCF resource CD from IDesign with lots of demo's.

But the again developers love to know how things work ( so they can share this "wisdom" with the world in for example VISUG event ...or blogs :-) They love make their own stuff. But like Juval said "By giving a VB programmer the ability the use over the threading library is like giving a child a hand full of blades. They will bleed to dead. " But he immediately added "C++ don't always do it better. They are only more used to seeing blood."

Sometimes we act more as "Google-programmers" . We (me included) look something up . Bend it and shape it a little so it works for us; Put the name of the original author on it and carry on. Sometimes not really really understanding what is going on under the hood. Disasters lurk around the corner.

WCF takes responsibility for all that hard plumbing code . You never should create it on your own (except if you are the plumber). In WCF you actually don't see any plumbing code. All its powers get unleashed by configuration and attributes. Will not writing the infrastructure code yourself doesn't mean you don't need to understand conceptually what is going own or what the annotations are for. In fact Juval mentions that we need to think about the design more than ever but we do not necessarly write it our selves (as application programmer) . The paradigm shift maybe needs a somewhat different approach to the design process and the way we need to structure our applications.

This is the foothill Juval is talking about. Of course he can offer that guidance :-) He wrote a book on the subject (Programming WCF Services) and gives master classes on the WCF and Service Orientation in his own company IDesign .

So enter this brave new world of service orientation. It may like a class but offers so much more . A whole application of services , all running in their own "process" giving you a lot of infrastruture functionality right out of the box.

Still I have some concerns.

Focus on your own domain code. Leave plumbing to someone else. We heard that one several occasions and other (non-Microsoft) technologies before. In fact in 1996 when Microsoft transaction Server was released as part of option pack for NT4, you already had that notion of interception. In COM+ additional "services" were made available to "objects" hosted in the COM+ space. What is different now? Yes of cours WCF can do lots more. But still COM has been deprecated. (Has it? Office is still COM isn't it ?) and .NET came along. Now WCF is there.
Companies have invested lots of money in applications running on previous . Programmers have invested time in learning these technologies. Now we have to learn something new again ! Of course in each branch there is a progressing insight how things work and how problems can be solved. Our business the pace is somewhat quicker. If you come to think about it .NET started somewhere in 2001-2002. Time for something new thus . Maybe this is another reason not (as a application developer) get too deep into the nitty-gritty details of something although I must admit... I too like to know how things work). We should concentrate on acquiring domain knowledge or the skill to translate requirements to the platform . So actually the abstraction gap between the real world needs we translate into an information system and the actual code bits gets pushed from both side of the gap. On the one hand coding it becomes more clear, no nuisance of infrastructure code. On the other the whole model-driven movement is trying to find a way to abstract the requirements in such a way "machinery" can kick in to produce the (part) of the code. I think both efforts will find its way to each other. ( Entity Framework, Web Service Factory Modelling edition, Volta, Oslo) . I wonder how this all stand against "agile development"? Are these opposites?
Technology is one thing. Process (design , methodology) and people are other big variables in the equation. Or maybe synergy is a better word. How will WCF affect our design process. What skills do I need to learn understand WCF and correctly apply WCF. How to do I architect my apps?
Besides development there is also the operations side. How will this be affected?
Is WCF everywhere actually feasible with the current WCF. Juval made a special in proc factory to ease some of the config burden. Will this be enough to ease configuration (XML hell ) Will it not be too slow?
Why is an outside expert promoting this vision? Where are the Microsoft Evangelists? Where is Don Box actually?

Best regards,

Alexander

Microsoft Software Factory Portfolio

2008-03-19T21:41:00.000+01:00

Introduction

The Software Factories provide you with a set of proven practices for building distributed applications. These practices are exposed through architectural overviews, patterns, how-to topics, reference implementations, automated guidance packages, and application blocks. There are several SW factories you can download today ( Web Service (WSF), Smart Client(SCSF), web client (WCSF))

Microsoft Software Factory

Software factories mean, different things to people: some are more academic, some are more pragmatic and more tool-focused. Like the one Microsoft is using for example “A software factory is a structured collection of related software assets.
When a software factory is installed in a development environment, it helps architects and developers predictably and efficiently create high-quality instances of specific types of applications.”
Although the differences, they all share some common motivator : helping the developer (or designer/architect) to get the job done in a more efficient way (time-to-market) and more with more quality.
MS SF is also about guidance. It should feel having an experienced person next to you and advising you what you should , when you should do it and how you should do it . In the current incarnation of SF this will translate into some menus, wizards and templates to the standard Visual Studio environment (Solution Explorer) to help developers build whatever the SF was intended for.
MS SF is also about automation. Capturing this expertise/best practices/guidance into integrated, automated features of Visual Studio and letting Visual Studio do some tasks in a consistent (so less error-prone) way , takes away some chores from the developer. You could say SF are about re-use but not only code but also expertise.
MS SF is also geared towards abstraction . The patterns and best practices to build a web service are not directly translated into code but there’s is an intermediary step. This gives you an opportunity to delay certain implementation decisions. It enforces you to think about it first. So the model is not the code but an abstraction (visual or not) of what you’re trying to establish and this data is kept and not lost.
The technologies that makes this possible are
- The Guidance Automation Toolkit and its "run-time" companion Guidance Automation Extensions (GAX/GAT)
- Domain Specific Language toolkit
- Advancements in Visual Studio itself (Designer Integration Service)
The main point is that Microsoft Software factories help Software Factories help you build a specific kind of application.

Domain Specific language and DSL Toolkit

A domain-specific language (DSL) is a programming language or executable specification language that offers, through appropriate notations and abstractions, expressive power focused on, and usually restricted to, a particular problem domain.
The DSL Toolkit (in the SDK) allows you to build visual interaction components that can be used from with in VS just like you would for example use a Winform Form designer. You drag & drop the VS-integrated graphical notations for a given model on the canvas and model the part of the application the DSL was conceived for. Later the actual code is generated for you. So the toolkits allows you to construct a custom-tailored designer for a model for a particular problem domain.
The main reason for this specialisation is that the more focused you are on a domain, the more specific you can make the abstractions and the better you can generate code from this models. Note that domain is not necessarily technical but also can be created for a particular sector like healthcare or finance.

Guidance Automation extensions and toolkit

The Guidance Automation Toolkit and its "run-time" companion Guidance Automation Extensions (GAX/GAT) enable you to extent the Visual Studio 2005/2008 experience.
The Guidance Automation Extensions (GAX) is the runtime part that needs to be installed on the machines of the developers that want to use the Guidance Packages.
The Guidance Automation Toolkit (GAT) is the authoring part that helps you create the Guidance Packages and the setup projects to install them. Once you've installed GAT, you'll notice a new project type in Visual Studio called "Guidance Packages". This will contain all the packages that were installed on this machine.
Designed to simplify integrating reusable code into applications.
Allows architects to automate development activities that developers would usually have to perform manually; often by following a series of instructions.
By using the toolkit, architects can also ensure that repetitive and often error-prone activities are performed in a consistent manner, streamlining and accelerating the development process.

Portfolio

Gives answer to various scenarios

Smart Client Software factory
Web Client Software factory
Web service software factory
Mobile Client Software Factory
Repository factory : The Repository Factory generates business entities, stored procedures, and repository classes based on an existing database. Can be enabled in the other factories.
Application Block Software Factory : Included in the Enterprise Library - guides developers through the process of creating new application blocks and extending existing Enterprise Library application blocks.

We can see a line in the advancement of Guidance and guidance automation in particular

Zero generation : guidelines in paper form, Application block as accelerators , VS templates
First generation : wizard-like , modelling data is translated into code, modelling data is not kept (Smart Client Software Factory , Web Client Software factory )
Second generation : Integration of DSL technology, modelling data is kept (Web Service Software factory Modelling Edition )
Next generation : Integration multiple models

Smart Client Software factory

addresses the challenges around building complex enterprise Winforms applications. As the complexity increases and the teams grow, the application becomes increasingly difficult to maintain.
enables designing a composite application that is composed of many discrete, loosely coupled modules. These modules can be developed, tested by separate teams.
Build upon the Composite Application Block (CAB)
Not all part are compiled together in a giant .exe file,
parts are more or less separately and at runtime they are compose together
CAB provides the “infrastructure” to support the loos-coupling
- Microsoft.Practices.CompositeUI.dll
- Microsoft.Practices.CompositeUI.WinForms.dll
- Microsoft.Practices.ObjectBuilder.dll
Make use of Enterprise Library modules (Exceptions, Logging, Caching, Security)
Aim SCSF: let application developers concentrate on the “application” logic not the infrastructure part (CAB)
Generates consistent application infrastructure, style and logic
Intrinsic separation of form from function / application layers
Uses Model-View-Presenter pattern (Passive View Variant)
Integrated front-end to diverse data sources
Sync & async web services, online & offline capable, caching
Based on concept of Dependency Injection (ObjectBuilder component from Enterprise Library )

Web Client Factory

Similarities with SCSF in concept
Creates Initial Visual Studio Solution
Web Application Projects or Websites
VB or C#
Add View-Presenter, Business Modules, etc.

Mobile Client Software Factory

for developing applications where the client is running on mobile devices such as PDAs and smart phones.

Web Service Software Factory Modelling Edition

This is the third version.
C# only
ASMX and WCF
For developing distributed applications communicating via Web services.
The factory uses three types of models to generate service code ; Service Contract models , Data Contract models, and Hosting models.
The models are visualized with special “shapes” and their corresponding properties. The look & feel is like the Class diagram tool in Visual Studio. So it is not UML (too generic) .
The designers that enable the modelling features are actually build with the DSL toolkit. For each of the three model (service, data and hosting) you get different designers. From the demo it looked very slick.
Extending the WSFm is possible . P&P identified some 11 scenarios. Some of them , like changing something in the exiting code Text templates are very easy , while other require you the download the source code for the WSFm and modify (and maintain ) it yourself.

Repository Factory

Used to be a part of Web Service factory as the Data Access Guidance Package.
Now separate package that can be enabled in other packages
The Repository Factory generates business entities from tables in a existing database, CRUD stored procedures for each table in the database, and repository classes based on an existing database.
Data Mapper (Fowler) with pure ADO.NET using SPROCS
Multi-Database Support via Data Access Application Block
No Relationship Support- 1:1, 1:m, m:m
No Object State Tracking
Currency Support Only via Timestamp
Community Led Development on codeplex.com .

References

Patterns and Practices Guidance ( http://www.pnpguidance.net/)
Microsoft patterns & practices SF ( http://msdn2.microsoft.com/en-us/teamsystem/aa718951.aspx)
Software Factories: Assembling Applications with Patterns, Models, Frameworks and Tools (Jack Greenfield & Keith Short)
Compuware Belgium forum site, http://intofactories.net/
Code generation Conference 2008 ( http://www.codegeneration.net/conference/index.php)
Domain-Specific Development with Visual Studio DSL Tools (Microsoft .NET Development Series) by Steve Cook, Gareth Jones, Stuart Kent, Alan Cameron Wills.
The EFx Architectural-Guidance Software Factory,Jezz Santos,http://msdn2.microsoft.com/en-us/library/aa905331.aspx
HL7 Software Factory , Mauro Regio , http://softwarefactories.com/workshops/OOPSLA-2005/Papers/Regio.pdf
Software Factories: report from the trenches, and thoughts from p&p ,Edward Jezierski , http://blogs.msdn.com/edjez/

VISUG event : Web service Software factory

2008-02-22T10:15:00.000+01:00

Hello ,

The Belgian Visual Studio User group (VISUG) invited yesterday (21/02/2008) Olaf Conijn to give a presentation about Software Factories and the Web service Software factory modelling edition.

Olaf has worked together with Microsoft patterns & Practices group on the Web service Software factory modelling edition (WSFm) where he was responsible for the extensibility part of the WSFm. Later he implemented Software Factories in the Netherlands (Capgemini BV). So you could not image a better person to give this presentation. Currently Olaf is trying to make the world a safer place as he's implementing software to alert the world in case of outbreaks of pandemics.( http://instedd.org/).

Olaf’s presentation was divided in several logical parts

The “What” of Software factories
The “Why” of Software factories
The “How” of software factories in the form of demo’s of the WSFm

Some key point I got from the presentation and the talk with Olaf afterwards:

There are many definitions of Software factories : some are more academic, some are more pragmatic and more tool-focused. Like the one Microsoft is using for example (MS SF). (http://msdn2.microsoft.com/en-us/library/bb871630.aspx) Although the differences, they all share some common motivator : helping the developer (or designer/architect) to get the job done in a more efficient way (time-to-market) and more with more quality. (http://appdevchronicles.blogspot.com/2007/10/software-factory.html)
MS SF is about guidance. It should feel having an experienced person next to you and advising you what you should , when you should do it and how you should do it .
MS SF is also about automation. Capturing this expertise/best practices/guidance into integrated, automated features of Visual Studio and letting Visual Studio do some tasks in a consistent (so less error-prone) way , takes away some chores from the developer. You could say SF are about re-use but not only code but also expertise.
MS SF is now also geared towards abstraction: This was very apparent in the WSFm (hence the name of course). The patterns and best practices to build a web service are not directly translated into code but there’s is an intermediary step: You model all the different parts of a web service like the MS engineers envision them. This gives you an opportunity to delay certain implementation decisions. It enforces you to think about it first. So the model is not the code but a very visual abstraction of what you’re trying to establish.
The models are visualized with special “shapes” and their corresponding properties. The look&feel is like the Class diagram tool in Visual Studio. So it is not UML (too generic) . The designers that enable the modelling features are actually build with the DSL toolkit. For each of the three model (service, data and hosting) you get different designers. From the demo it looked very slick.
There are several key technologies that make MS SF possible : GAX/GAT is about leveraging the automation possibilities of Visual Studio in the field guidance. DSL are geared toward modelling and code generation.
MS Software factories come in different forms and shapes (Web client SF, Smart Client SF , Web Service SF, etc). So there isn’t a general SF that can encompass all types of applications. The main reason for this specialisation is that the more focused you are on a domain, the more specific you can make the abstractions and the better you can generate code from this models. Note that domain is not necessarily technical but also can be created for a particular sector like healthcare or finance.
Extending the WSFm is possible . P&P identified some 11 scenarios. Some of them , like changing something in the exiting code Text templates are very easy , while other require you the download the source code for the WSFm and modify (and maintain ) it yourself.
You can even build your own SF. Olaf figures , based on own experiences in the Netherlands, that make a SF is 2 to 3 times more expensive than building the product that the SF has to build (if I understood him correctly).
So building a SF only makes sense if you build multiple applications based on the same architectural principles that the SF enforces. Olaf estimates that you earn the SF build back if you “sell” 5 project build with SF. So you only should invest in SF you expect “economies of scale”
If you embark on a “journey/ordeal” to make your own SF , be sure to have “domain” experts at hand, be technical or functional in nature.
But building a SF doesn’t have to automate everything . You can also make more “pragmatic” SF that only focus on very explicit stuff your trying to implement. For example creating a skeleton for a solution with the GAX/GAT

Some concerns ( expressed in the form : “I wish I knew how to …”)

H2 assess the adoption grade of MS SF in organization…. And this is not the same as announcing the number of downloads :-)
H2 assess the importance in MS SF in the big scheme of development tools at MS. MS SF came out of the P&P group. One of the key technologies GAX/GAT) has been a CTP for a long time (sine 2005) . It just recently got the label release. As far I can tell the DSL (another key technology) is maintained by another group. Why are these technologies not in the Visual Studio release cycle?
H2 integrate VB.NET into the WSFm. In the version (I presume the latest at the time) we show you could not select VB.NET as your language. When Olaf added a WCF implementation in created a c# project. According to him , you only need to translate the text templates. If it is so easy , why didn’t P&P do it for us?
H2 find proper “guidance” to build “guidance tools” with GAT and DSL besides books and blogs.
H2 cope with generated code, your own code and the model (keeping it synchronized).
H2 position agile techniques with model-driven development. Are they opposites or do the complemented each other ?
H2 interpret the disappearance of the data access wizard in the previous versions of the WSF (at least I didn’t see it in the demos). These wizards enabled you to generate data access code and business entities. Maybe the advent of Linq to SQL an the Entity data framework have something to do with it.

Some references mentioned (or I heard from other participants)

Microsoft patterns & practices SF ( http://msdn2.microsoft.com/en-us/teamsystem/aa718951.aspx)
Software Factories: Assembling Applications with Patterns, Models, Frameworks and Tools (Jack Greenfield & Keith Short)
Compuware Belgium SF focus group: http://intofactories.net/
Code generation Conference 2008 ( http://www.codegeneration.net/conference/index.php)
Domain-Specific Development with Visual Studio DSL Tools (Microsoft .NET Development Series) by Steve Cook, Gareth Jones, Stuart Kent, Alan Cameron Wills.

Maybe you some own thoughts or opions? Be sure to drop a line ....

Thanks to VISUG for inviting people like Olaf to Belgium. Be sure to keep an eye on the VISUG site (http://www.visug.be/) because the one of the next speaker will be none less than Juval Lowy. Keep up the good work.

Best regards,

Alexander

Stay in the comfort zone with OBA

2008-02-14T08:25:00.000+01:00

Hello,

Yesterday (13/02/2008) I attended a MSDN evening organised by Microsoft Belgium about Office Business Applications (OBA). Serge Luca from U2U and MVP for Connected Systems gave an overview about OBA from a development perspective . He had also plenty of demo’s available. Too bad his time was limited.

This is what I remembered from it.

OBA are applications that leverage the Office Suite applications beyond their out-of-the box capabilities and integrate them with other systems in your organisation (Line-Of-Business Apps).

The key driver for leveraging Office is to let users stay in their “comfort zone”. While organisations have invested large amounts in the user experience in packaged and custom applications , users still like to work in their familiar Office tools. Even the advent of web-enabling all the LOBs didn’t stop users from for example pumping SAP data into Excel and use it from there on. The user-experience the people get from Office tools is unbeatable.

That’s why Microsoft made Office 2007 System a truly development platform to enable developers to create OBA. The technology enablers are not only situated on the client side (i.e. Office Apps themselves) but also on the server side. For example you can extend the Office user interface (ribbon extension, task panes, Outlook from region) . Thanks to the Office open XML format you can intervene in the documents at file-level . This open possibility to programmatic creation of for example word documents. You even don’t need a Word instance on the server. Infopath enable the creation of electronic forms. Sharepoint and related technologies are the enablers to create workflow oriented applications and extend the content document capabilities of Sharepoint. The Business data Catalog makes it easier to retrieve and search information that is already available in ERP systems or databases.
And all this technology can be unleashed through one development environment ; Visual Studio 2008.

In final part of his presentation Serge mentioned 6 basic OBA patterns or scenario’s if you will. Unfortunately he didn't have the time to go through them but the demo’s he showed , were actually implementation of these 6 patterns. A book he highly recommends in that respect is 6 Microsoft Office Business Applications for Office SharePoint Server 2007 from Microsoft Press .

So if you think Office Development is only about VBA (which is still possible by the way) , think again and check out http://www.obacentral.com/. Or like Serge said : "Let's Rock 'n' Roll (with OBA)"

Best regards,

Alexander

Don’t bother System testers with programming errors.

2008-02-13T14:47:00.000+01:00

Hello ,

Suppose you're a team lead on a project and you take a look in your bug tracking tool one fine morning. You see that the tester on the team has filled some bugs.

Here are some details of it:

Bug ID: 6
Short desc: Conversion from string "r" to type 'Double' is not valid.

Last changed by	FuncTest
Reported By	FuncTest
Reported On	2008-02-13 2:32
Project	Petshop Redux
Organization	Omega
Category	bug
Priority	high
Assigned	developer
Status	new

comment 6 posted by FuncTest on 2008-02-13 2:32

test case # 5487 Enter new Order

Entering the non-number quantity for a certain product resulted in an error after filling out the order form. The error message was : Conversion from string "r" to type 'Double' is not valid.

There is no check for numbers ?

Bug ID: 5
Short desc: Could not find a part of the path 'c:\PetShopRedux\orders.xml'

Last changed by	FuncTest
Reported By	FuncTest
Reported On	2008-01-31 11:09
Project	Petshop Redux
Organization	Omega
Category	bug
Priority	high
Assigned	developer
Status	new

comment 5 posted by FuncTest on 2008-01-31 11:09

test case reference (#4531) : Export Open orders to xml file.

I wanted to export the open oders. Unfortunately I received the follwing error message Could not find a part of the path 'c:\PetShopRedux\orders.xml'.

When I look at my c drive I can not find a folder named PetShopRedux.

Can you fix this?

Bug ID: 3
Short desc: Index was outside the bounds of the array.

Last changed by	FuncTest
Reported By	FuncTest
Reported On	2008-01-31 10:38
Project	Petshop Redux
Organization	Omega
Category	bug
Priority	high
Assigned	developer
Status	new

comment 2 posted by FuncTest on 2008-01-31 10:38

test case reference (#5781) : Show overview Orders to be saved.

When i press the nenu to show me the orders that i have entered until a certain point , I get a message saying Index was outside the bounds of the array.

Can you fix this? I need to test this feauture before Wednesday.

Bug ID: 2
Short desc: Object reference not set to an instance of an object.

Last changed by	FuncTest
Reported By	FuncTest
Reported On	2008-01-31 10:25
Project	Petshop Redux
Organization	Omega
Category	bug
Priority	high
Assigned	developer
Status	new

comment 1 posted by FuncTest on 2008-01-31 10:25

Test case reference (#4578) ; Create Order with no products.

I get this strange message saying "Object reference not set to an instance of an object." But I have not made a reference to an order yet ??

Can you check this? Urgent, is blocking me.

Ok , you say :

"The tester is doing a great job ! I'll quickly instruct the developers to fix
these errors. "

Or .... you could take a step back and analyze the type of bug the tester has found. You shoudl may be take a look at the test cases result of the tester . You might find out that the tester wasn't able to do a lot of testing after all because he was blocked by the bugs...

The Bug list (artifical) shows a high digree of programming errors . There are no real findings of missing functionality, wrongly interpreted specifiaction. In other words the function tester has been able to do his/her thing. The tester got hold up by types of errors that are more technical in nature. Now the functional tester is somehow blocked until the corrected version is released again in QA. Your project manager will not be happy to see his/her people being idle.

It's time to revise your development process , educate your developers and give them the means to conduct testing themselves : enter development testing and test automation.

Maybe your developers will answer :

What are you talking about, we already do testing! : Maybe that is true . For example making a Form with 23 buttons and text boxes that test some piece of code they have written. The trouble with this approach is that it a custom implementation. Each developer can do what he likes. It can not be (easily) automated : you must click on the buttons and verify the result in a file or through the textboxes.
We don't have time to do testing! : Yes testing is hard and does take time. But finding a bug early on is "cheaper" than when some poor tester finds it after a candidate release . Or even worse when the customer/end-user finds it. It will avoid long cycles of develop-test-correct-retest. When a developer finds a bug , he/she knows the context and can more easily hunt down the bug and eliminate it.
It is not our job , we have testers on the team. Why do it twice?: The developer testing is is somewhat different from functional testing (or non-functional testing like performance). In developer testing you primarly want to eliminate the any programming errors. Sure you have some clue how and end-user will use the application, but you're master of the code and therefore have more insight then, anyone where problems lure. So your entry point is the code itself . A functional tester looks at application from the outside (white-box vs. black-box). He doesn't see the code in motion. He only sees the interface of the system. The functional tester will concentrate more end-to-end scenario's that are possible (or that should not be possible) . Also testing throughout the process avoid big bang testing . When you test the smallest parts and continue testing by putting the tested parts together, the chance of (programming) errors coming up later in the process are reduced.

Introducing developer testing and preferably also test automation (for example Visual Stuid Test framwork or NUnit) is not soley a technical matter. You should get the developers to test that their code meets the "their" intentions . There will always be a loss infomation before a letter of code is typed. So system testing remains necessary . Developers should think about what can go wrong with what they're coding, as they're coding it or even before they start coding (TDD). Early detection of errors cost less in the construction phase. System / acceptance test never can reach same coverage of testing. Developers should control quality before shipping next phase (minor for example at check-in or major : release to QA) .

Of course testing alone won't cut it. Testing does not put quality into the code. It just gives insight in the quality of the code (application). You will still need tons of other measures to assure quality code...But that's for another post.

Best regards,

Alexander

GAT : Skeleton Pattern

2008-02-08T10:30:00.000+01:00

Hello,

This entry describes another typical scenario where you can use GAT to automate certain VS Solution creation activities.

Driving forces
When developers in your organization want to create a new VS solution for an application , you want to help them in creating a typical structure of a multiple-project solution. You don’t want a describe everything in detail in a document but you want to automate as much as possible and make it available within the VS environment. You want also to add some possibility for customization of namespace and project naming.

Solution
Use the GAT to make a Guidance package that unfolds the “Skeleton” structure. Add some recipe wizards to collect initial information from the user. Install the Guidance Package on the developers machine to make the Guidance Package available in Visual Studio 2008. The developer can choose the “skeleton” from the Visual Studio 2008 new project–dialog to unfold the initial structure of the solution after entering initial values to create the skeleton.

Structure

Attention

Guidance package ,GAT template for making Guidance Packages should be stripped to bare minimum before proceeding with own package creation
References to assemblies, must exist on target machine (GAC or same folder : Hintpath !!!)
Take care of parameter replacements ; In project files (references) and import statements in any code.

Alternatives

Copy-paste a skeleton available in a source control system
“Pure” VS2005 templates solution with parameter replacement and Wizard extension mechanism

Best regards,

Alexander

GAT : StarterKit Pattern

2008-02-07T08:23:00.000+01:00

Hello,

Making guidance packages GAT (Guidance Automation toolkit) is also making knowledge available for novice users regarding certain processes or technology. This entry I will discuss a typical scenario of how you can make an example application available through GAT in Visual Studio 2008.

Driving forces
You want to make a working example of how a typical Winforms application is made in your organization available for your developers. The developers must be able to run it , learn from it and make changes to it if they want to.

Solution
Use the GAT to make a Guidance package that unfolds the “Starterkit” application. Install the Guidance Package on the developers machine to make the Guidance Package available in Visual Studio 2008. The developer can choose the Starterkit from the Visual Studio 2008 new project–dialog to create the example application.

Structure

Attention

GAT template for making Guidance Packages should be stripped to bare minimum before proceeding with own package creation
References to assemblies, must exist on target machine (GAC or same folder : Hintpath !!!)

Alternatives

Installed example application a demo box
Example application in SourceSafe
Setup package (msi) of the application with all the sources
“Pure” VS2005 templates solution

Best regards,

Alexander

First steps with the Guidance Automation Toolkit (GAT)

2008-02-06T08:38:00.000+01:00

Hello,

I recently tried out the Guidance Automation Toolkit and its "run-time" companion Guidance Automation Extensions (GAX/GAT - July 2007 CTP) on VS Rosario November CTP image.

The GAT enables you to extent the Visual Studio 2005/2008 experience. Extensibility comes in different forms and shapes but the GAT is geared helping the developer when creating a VS solution. It gives "guidance" in the form VS project or item templates combined with automation hooks (recipes) that can collect information from the developer (wizards) while the solution structure is being created (un-folded).

GAT find it's origin in the Microsoft Software Factory Initiative. The Software Factories provide you with a set of proven practices for building distributed applications. These practices are exposed through architectural overviews, patterns, how-to topics, reference implementations, automated guidance packages, and application blocks. There are several SW factories you can download today ( Web Service (WSF), Smart Client(SCSF), web client (WCSF))

The idea behind GAT is not completely new. A previuos technology in VS2003, the Enterprise Templates was also an effort for rapidly defining the initial structure of applications and to minimize the need to read through a bunch of of white papers, standards, and policies documents.

In VS2005 we've got the new template mechanism (Visual Studio Templates) . The Export template makes is fairly easy to make shareable templates. Actually GAT uses these exported template. What makes GAT different is the way you can hook automation to these templates.

Wojtek Kozaczynski (http://blogs.msdn.com/wojtek/archive/2005/05/02/414129.aspx) has a nice blog-entry describing the origins of GAT.

Yet another technology, called Domain Specific Languages (DSL), is available to extent VS. The DSL Toolkist (in the SDK) allows you to build visual interaction component that can be used from with in VS just like you would for example use a Winform Form designer. You drag & drop the visual elements on the form and code is generated for you. An example in the SDk is a DSL that enables you to visually build a Wizard-like application. So you get a special toolbar and drag & drop the visual elements on your "canvas" and "model" your wizard application. The model then is used by a code generator to produce executable code.

So GAT and DSL Tools both build on top of existing VS extensibility featueres but with more possibility in guidance experience or modelling experience.

Daniel Cazzulino has some thought about the status of developer guidance in realm of GAT, DSL and Software Factories (http://www.clariusconsulting.net/blogs/kzu/archive/2007/01/08/BuildingSoftwareFactoriesToday.aspx)

So I see certainly advantages

Productivity can be increased : It includes automation for Visual Studio: with this automation, de-velopers can easily apply guidance in consistent and repeatable ways.
Adaptable : It is open and customizable.development leads can customize the factory to meet specific needs.
Accelerated start. It provides an effective way for architects and developers to create a high-quality starting point for their application. This means that projects begin with a greater level of maturity than applications developed from scratch.
Enforce standards : to enforce standards independent of the solution domain

But I have also some concerns

Support : I don’t think so. Is not an official product of MS. It’s from the Patterens & practices group. It is still CTP ... since July 2005. Forum exists but I haven't seen any MS employee replying to questions...(http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=78&SiteID=1)
Some (steep) learning curveExamples only on blogs (http://jelle.druyts.net/CategoryView.aspx?category=Blog%7CProgramming%7C.NET%7CGuidanceAutomation) or you have to dissect the WSF/SMSF/SCSF or you have to study the default Guidance package that is produced when you create a new guidance package project.
Support for next versions of GAT/GAX once an application has been built with a version of GAT/GAX
Mistakes are severely punished :-). You can easily scr*w up your registry (like I did) without a warning. Re-installing GAX/GAT can result in something like this

In that respect Victor Garcia Aprea from Clarius Consulting made a GaxTroubleshooter (http://weblogs.asp.net/vga/archive/2007/12/28/troubleshooting-gax-gat-installation-issues.aspx) but I wasn't able to repair my mistakes with it though. Clarius has also a Software Factory Toolkit (http://www.softwarefactoriestoolkit.net/) but I dare not to re-install it because it is rather invasive . It seems to intervene in the default GAX/GAT installation.

Does someone actualy used al these specific VS features to create a custom Guidance package ? What were your experiences? Like to hear from you.

Best regards,

Alexander

Unit testing ADO Entity Framework

2008-01-22T10:16:00.000+01:00

Hello,

I was playing with the ADO.NET entity framework beta 3 that I installed on the Rosario November 2007 CTP and I had some problems making unit tests. I kept getting the follwing error.

System.Data.MetadataException: The specified metadata path is not valid. A valid
path must be either an existing directory, an existing file with extension
'.csdl', '.ssdl', or '.msl', or a URI that identifies an embedded resource

When you use the wizard to create a model several "hidden" files are generated for you; .csdl, .ssdl, and .msl files. The information contained in these files is encapsulated in the .edmx file but the files .csdl, .ssdl, and .msl files reside in the bin\Debug or bin\Release directories of the solution. Apperently these files are still necessary at run-time. There not embedded in the assembly.

In a connectionstring you specify these files in the metadata part :

connectionString="metadata=.\Northwind.csdl.\Northwind.ssdl.\Northwind.msl

When you're using for example the VS test framework, the test infrastructure copies the assemblies to a certain test folder (isolation). If you have an app.config or use custom files in your code, you must assure that the test framework put these file in that particular test folder.

Now the .csdl, .ssdl, and .msl are also not copied to that test folder automatically. I used the deployment feature of the VS test environment. I specified the three files and now there copied to the test folder.

Maybe there are other ways? Let me know!

Best regards,

Alexander

Unit Testing Checklist

2008-01-13T21:46:00.000+01:00

Hello,

Over time I compiled a list of things regarding unit and unit integration tests; books, articles, forums and other blogs. I did not always kept track where I found some piece of information. So I certainly don't take any credit in this list (except compiling it). Recently I discovered the XUnit Test Patterns website by Gerad Meszaros (http://xunitpatterns.com/index.html). I wished I found that one earlier.

Still I wanted to share with you the following list.

Do you know how to incorporate the unit testing into the development process?
Unit tests are written by the developer who write the code and executed by the developer.
Feedback period between coding and finding error is smaller
Coder and tester is same person
Accounted in the development budget
Continuous testing versus phase-end testing
Avoid arguments like: I don’t have the time to conduct unit test. I must deliver ASAP.
Don’t rely on phase-end tests to find your errors.
Do you have a strategy to incorporate unit test code in production code organization?
Be prepared to see as much production code as test code so good organization primordial.
Integration in source control system is needed
Are you aware of the type of errors unit testing tries to find?
Some errors are easier to find with unit (integration) test than in formal phase-end tests. Or at least the effort is bigger to find those errors.
Do you use an automated tool such as NUnit or VS2005 Integrated testing Framework to make the unit-testing process easy and repeatable?
Same Framework available for every developer. Same framework used by every developer.
Automation in two ways: invoking the tests and checking the results.
Can be integrated into automatic build/continuous integration scheme.
Do you know which type (layer) of code benefits the most of unit testing?
Some type code is more appropriate for unit testing than other for example business logic versus User interface.
Whenever you are tempted to write something into a debug.writeLine type of thing, write a test for it
Do you know how to deal with code that is to hard to unit test?
Consider other ways of testing
GUI skin code,Main methods ,Asynchronous method ,mult-threading
Do you know how to deal with developers that don’t write unit code?
Education
Take away resistance by showing advantages
Do you how to avoid that unit-testing becomes de-prioritized when deadlines come near?
Planning
Do you have the right mindset to conduct unit test?
Showing that it works versus finding errors versus
verifying expectations (specification) compared to the realisation (code)
Are you prepared to scrutinize your work?
Do you know when to write unit test?
Write the tests as you go along. Don’t leave them for a “later” that may never come.
Do you know when to execute unit test?
Run unit test as often as possible to decrease feedback period between coding and finding error is smaller

Do you know how to integrate unit testing into the build process /continuous integration scheme?
Unattended execution of unit tests to let the test run as much as possible.
Do you have a policy about how to deal with failed unit test?
Make sure code passes unit tests and integration tests before you check it into the source control system.
Do you make a distinction between pure unit test and unit integration test and when to use the them?
Make test for collaborating units in function in stead of big bang approach in system testing. Unit test are good for regression testing
Do you have a policy to distribute knowledge about conduction unit testing and unit (integration) testing and automated testing in general?
Optimal use of resources
Do you have a policy to use unit test for regression testing?
Changes to an existing product may not result in breaking unit tests.
Do you have a policy to keep the quality of unit tests high?
Avoid that the first unit test are good and the remaining unit test are of lesser quality (Coverage) because of time pressure.
Do you consider the unit test code on the same level as production code?
Be prepared to see as much production code as test code.
Test must be written and maintained to the same professional standards as your production code
Did you code to Interfaces rather than Classes?
To plug in different implementations at runtime or test time.
Makes it easy to provide test stubs that don’t need a sophisticated implementation of collaborators’ functionality and hence are freed from needing to depend on supporting infrastructure.
Do you use Inversion of Control/dependency injection pattern?
Makes is easier to feed the class under test with alternative implementations of collaborating classes.
Does each class have a well-defined set of responsibilities?
So it will be more easy to come up with unit test.
High cohesion , low coupling
Did you expose too much of a class for the sake of testability?
Don’t mark a method public just for the sake of testability. Keep encapsulation in mind.
VS2005 test framework can deal with class and methods marked friend or private.
Did you separate unit test code from the code under test?
Unnecessary code to deploy.
Is there one test class per class under test?
Code generation of VS2005 test framework encourages it and uses it when you generate additional test method skeletons

Is there a separate assembly per assembly under test?
Code generation of VS2005 test framework encourages it and uses it when you generate additional test class skeletons
Is there a separation between tests that need configuration files and the one that don’t need configuration files?
You can more easily identify which test should be able to run on any machine without setup (eg. Build server)
Pay attention on place of test data /configuration files because of the way xUnit framework works.
Is there a separation pure unit test and unit integration test?
With unit integration test you must always verify if the collaborating units didn’t cause an error when a test should fail.
Did you separate database access code test?
This kind of tests requires much more attention in preparation, configuration and execution speed.
Is the unit testing code under source control
Is part of the code base.
Any developer must be able to use it afterwards.
An automatic system (Build/CI) must be able to use it.
Can all test methods run autonomous so they don’t rely on other test or require that they are run in a particular order?
Test method can be selected/de-selected via the Test UI so test must run independent of each other.
VS2005 test framework allows ordered test runs (more functional like testing, use case scripts)

Is every test able to run over and over again, in any order, and produce the same results?
Otherwise automation of the assertions is difficult.
Did you use mocks or stubs to accomplish test repeatability?
Use mock objects /stubs help you to isolate the item under test and keep it independent from the environment because they give always the expected values.
Did you consider positive unit tests?
To exercise the code as intended and verify the right result.
Did you consider negative unit tests?
To intentionally misuse the code and verify for robustness and appropriate error handling.
Did you consider "Stress" tests?
To check the code for its limits (overflow , etc).
Did you consider data conformance test?
To check whether an result value conforms to an expected format
To check how the code deals with input value that conforms with expected data format and vice versa
Did you consider Ordering tests?
To check whether the set of result values are ordered or unordered as appropriate.
To check how the code deals with input values that are ordered or unordered.
Did you consider range tests?
To check whether the result falls within reasonable minimum and maximum values
To check how the code deals with input values that fall within reasonable minimum and maximum values.
To check how the code deals with input values that fall outside the boundaries of reasonable minimum and maximum values.
Did you consider reference tests?
At the end of the method, post conditions are those things that you guarantee your method will make happen. Direct results returned by the method are one obvious thing to check, but if the method has any side-effects then you need to check those as well.
Did you consider existence tests?
To test whether the result exist (e.g., is non-null, non- zero, present in a set, etc.)?
To test how the code deals with empty or missing input values (such as 0, 0.0, “”, or null).
Did you consider cardinality tests?
To test whether there are there exactly enough values in the result.
To see how the code deals with input value list that is of the correct size or contains duplicates.
Did you consider process logic test?
To check if all control structure constructs (if, else, case, etc.) function correctly.
Did you consider which part of the code is reasonable for testing invalid parameters?
Check input at the boundaries of the system, and you won't have to duplicate those tests inside the system. Internal components can trust that if the data has made it this far into the system, then it must be okay. You still can take the pessimistic way but then be prepared to see a lot of extra unit tests.
Do you use code coverage tools?
In order to verify the thoroughness of the unit tests
Do you write tests for bug-fixes?
To demonstrate the bug isn’t present anymore.
Do you test for specific number/datetime problems like rounding’s , regional settings?
Decimal sign, DD/MM/YYY versus MM/DD/YYYY

Does the unit test method only test one specific thing?
When a unit test breaks you can more easily pinpoint the problem. Maybe even without debugging.
Did you consider the use of a Mock library or a stub while unit integration testing?
A stub is a piece of code used to stand in for some other programming functionality. A stub may simulate the behaviour of existing code (such as a procedure on a remote machine) or be a temporary substitute for yet-to-be-developed code. Stubs are therefore useful in unit testing.
Mock objects are simulated objects that mimic the behaviour of real objects in controlled ways. In a unit test, mock objects can simulate the behaviour of complex, real (non-mock) objects and are therefore useful when a real object is difficult or impossible to incorporate into a unit test.
Are the test suites enough self-documenting?
Documentation in the form of meaningful test case names and assertion messages is usually more important than comments on test cases itself. When a test fails, the output will show the failed assertion (or an error) and the name of the test in question
Is the name of the test method intention-revealing?
Name unit test shows up in interactive screen reports and console output
Did you avoid test method names with an appended numbers to distinct between them?
That doesn’t do a very good job of communicating your intentions
Is there a consistent naming convention for the unit test code?
So you know what are the test methods and the help methods.
Did you consider putting setup code that is used in every test method put into separate initialization XUnit framework methods that are triggered on test class or test method level?
Excessive data setup can obfuscate a unit test beyond comprehension so hide it if needed.
All state is known in advance and will always be the same no matter where or when your test is run.
To avoid one unit test leaves some kind of dirty data hanging around
Preconditions that must be true when you want to run the method under test.
Is your unit test fast?
Many unit tests are launched often so fast execution avoids friction and encourages you to run them often.
Did you consider factory methods that create the object instance under test for you?
You can then reuse that method to get fresh instances of your class under test in other test methods. This helps to keep the tests maintainable across time and guards your tests from unforeseen changes to the code under test.
Is the actual act of executing the method under test separated from the act of asserting on the result by creating a result variable on a different line?
The invocation against the object under test may be very long and might make your Assert line stretch all the way beyond the edge of the screen, forcing the test reader to scroll to the right.
Does the variable that contains the result have a readable name?
Makes your Assert line very understandable and easy to read
Did you consider using separate verification methods?
This is a reusable method in your test class that contains an Assert statement but that can take different inputs and verify something on them.
You can use these verification methods when you are asserting the same thing over and over again with varying inputs.
Even though the Assert is located in a different method, if the Assert fails you'll still get an assert exception and the original calling test will be shown in the test failure output window.
Can improve readability.
Did you avoid multiple Asserts in a Single Unit Test?
After a failure, subsequent Asserts aren't executed.
These unused Asserts could provide valuable data (or symptoms) that would help you quickly narrow your focus and discover the underlying problem.
Put the additional Asserts should be run in separate, self-contained unit tests so that you have a good opportunity to see what fails.
Did you use an informative assert message?
A good Assert message should always explain either what should have happened or what happened and why it's wrong. shows up in interactive screen reports and console output
Test code maintenance
Did you avoid code duplication?
Refactor the Test Suite as necessary and identify areas in which you can reuse code to speed up writing individual test cases

References & recommend reading

xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros, 2007 by Addison Wesley Professional
The Art of Unit Testing, Roy Osherove, Manning 2008 (MEAP 2007)
Next Generation Java Testing: TestNG and Advanced Concepts ,Cédric Beust and Hani Suleiman, Addison-Wesley 2007
Pragmatic Unit Testing in Java with JUnit ,Andy Hunt and Dave Thomas ,Pragmatic Programmers LLC 2004
Pragmatic Unit Testing in C# with NUnit: The Pragmatic Starter Kit, Volume II ,Andy Hunt and Dave Thomas , Pragmatic Programmers LLC © 2004
Coder to Developer: Tools and Strategies for Delivering Your Software ,Mike Gunderloy ,Sybex 2004
EJB Design Patterns ,Floyd Marinescu John Wiley & Sons 2002
Code Craft: The Practice of Writing Excellent Code ,Pete Goodliffe No Starch Press 2007
Code Complete, Second Edition ,Steve McConnell ,Microsoft Press 2004

Best regards,

Alexander

MSDN session Application Lifecycle Management with Visual Studio Team System (19/12/2007)

2007-12-20T09:05:00.000+01:00

Hello,

Yesterday I attended the MSDN evening session about Application Life cycle Management (ALM) with Visual Studio Team System (VSTS).

It was very interesting and very well presented by Yves Goeleven (Compuware Belgium). He was very knowledgeable about the subject and VSTS in particular. His presentation was a good mix between slides and demo's so you had a good feel what actually was meant.

Still I have some "concerns" about the subject. Concerns are formulated as "I wish I knew how to ...."

Position VSTS/TFS as an ALM tool or more as an SDLC tool? Yes there are hooks (extensibility)....
Capture the requirements from the customer in VSTS? And manage them in VSTS? Yes a wokitem will be the "materialization" of a requirement. But is this still the language of the customer?
To model use case diagrams, UML class diagrams , User stories
To integrate with operations (deployment, monitoring, incidents , capacity management)

I looked up the the terms ALM and SDLC. Application Lifecycle Management (ALM) is the administration and control of an application from inception to its demise. It embraces requirements management, system design, software development , testing , configuration management and operating the information system. The following picture tries to capture that idea.

Key is that the life of application does not start at the beginning of development. It start somewhere in the “business”. The end of application is not when it is delivered or even installed. It must be kept up and running to serve the business. It is also called life cycle because in effect an “application” is never finished. It is not something like a tangible product that rolls out a factory. It’s an ongoing process where new insights and requirements shape the life of an application, hence application life cycle management. As the business changes, information requirements change, and the cycle continues. Sometimes resulting in the demise of an existing system and the birth of a new application.

The software development Lifecycle (SDLC) on the other hand has everything to do with developing the information system. But it gets kicked of by the business and hands over application to operations. It is also a lifecycle because within one ALM cycle because it is the rare exception that everything is said from the beginning. So there can be several iterations before an applications gets to be delivered to the business. On the other hand of course a change request or new requirements can trigger of a new ALM cycle.

In order to achieve business objectives within an organization via information systems, user functional and non functional requirements must be defined in a consistent manner, prioritized and monitored. The administration and control of the information needs of users (Requirement Management) is therefore primary motor in the ALM and de facto also in the SDLC. All the “knowledge” that’s is necessary to transform these requirements into an information system is produced during the SDLC and put into a repository (or a collection of repositories). This will for the test basis for the test cases that will be used to verify it the information does what needs to do.
The SDLC will take that the requirement will be turned into architecture and design specifications which will eventually lead into code doing what the business expects it to do. Herein lies the objective of testing. Testing actually is conducted throughout the SDLC. Testing preparation can even start as soon as the requirements are written down (testable requirements concept). During development in the form of unit and unit integration testing or system testing (functional, security , performance ,etc). Before the application get rolled-out, the business itself can conduct test in the form of user (functional) acceptance test. Also operations can conducts acceptance tests that verify the operational requirements.

The input for establishing these test case are the expectations of the business. Ideally written down in the form of requirement, detailed functional specification and design documents. So there is a strong link between all the knowledge of what the system must do and all the information of how system is going to be tested.

A integrated system (or an integrated set of tools) should bring some added value to manage this link. Because the objective of testing is to report if the system does what it needs to do. So traceability is a key feature of this tool. Other benefits are accountability (who did what when), visibility (what did we do so far).

So if we would define testing as a kind of “fit-for-purpose” activity, testing is another gatekeeper in the life cycle. With the results of testing the stakeholders (customer/project lead) can take a decision based on facts if there will be release or not (ALM cycle) or within the SDLC cycle if something has to be corrected before release.

Monitoring the information system in production and the actual usage of the application will further keep the ALM cycle running by incident reporting (functional and non-functional).

So were does VSTS/TFS fit in this story?

I tried use the following picture to get some (visual) answers.

I can see several aspects being covered by VSTS while others (at this time) seem to not be (immediately) clear to me. Maybe this will change over time as learn more about ALM and VSTS/TFS in particular? I would be glad to hear your insights on the subject.

Best regards,

Alexander

Trigger TimedSubscription programmatically

2007-12-05T12:00:00.000+01:00

Hello,

Subscriptions are a way to kickstart Reporting Service in delivering a report to somewhere instead of you asking for the report through a URL access request (portal or custom app). This is also callled push versus pull.

You can configere a timed event to trigger off those kind of subscriptions. But in some occasions you want some "custom" event to trigger of a subscription. You can not configure something like that through the configuration tools (reportmanger or SqlServer Management Studio).

Lukasz Pawlowski (http://blogs.msdn.com/lukaszp/archive/2005/10/07/478391.aspx) explained a way to tell Reporting Services to kick of a subscription with the SOAP API of Reporting Services. There is a method called FireEvent that can trigger the subscription. Lukasz suggests creating a dummy schedule that is will never run because it's configured in the passed. The report subscription you want to launch uses this dummy schedule. In a program you call the Firevent method on the reporting WebService .... and your report is delivered as you specifed it in your subscription.

I will show some screenshots how I did it, following Lukasz's tips.

So this is the report I want to be delivered on a file share in PDF.

First I make a dummy shared schedule. But one that will never fire a time event.

Now I create a new subscription on the report I showed before. It will use the shared schedule as Time Event generator.

In order to launch this subscription I must simulate a time event. The SOAP API of Reporting Services just gives me that.

This is a code snippet in VB.NET 2005 of how you could just do that.

It is important that you use the ID (ie. GUID) to specify the subscription. You can also look it up in the Subscription table of the reportserver database.

But before you go off and launch this code one thing is left to be configured (besides the folder share of course) . The account under which this program must execute will need "generate event" privilege on Reporting Service level. Because this is a single machine demo setup (november 07 CTP of Rosario). The user tfssetup is also as system admin on RS. So i just checked the checkbox for generating events on the RS syst. admin role .

The code snippet you just so will actually write an entry in the reportserver.Event table. The RS service will pick it up and act upon it. In our example it will write a pdf file into the file share folder (on the local disk ).

Have fun.

Best regards,

Alexander

Navigation button Visual Studio 2005

2007-12-04T08:28:00.000+01:00

Hello,

When you open a freshly installed Visual Studio 2005 , you might miss certain things. For example the navigate backwards and forward button. Very handy when you navigate throughout your code.

You can easily add those buttons but nevertheless it is a bit strange that are not included by default (at least not on my machine) In VS 2008 (at least in the Rosario November edition) they are present and even a litlle bit smarter too. You can choose where you want to go back.

Open the "tools" menu and go to the "customize" menu-item.In the "view" category you will find the two missing buttons. When you select them , you can drag-and-drop them on your toolbar.

In Visual Studio 2008 you get the navigation buttons by default at a little bit more!

Best regards,

Alexander

Refactor!™ for Visual Basic® .NET 2008 and 2005

2007-11-16T12:41:00.000+01:00

Hello,

In VS 2005 you have a refactor possibility in C# but there was none out-of-the-box in VB.NET. In VS 2008 it is still the same (at least in the Beta edition) for VB.NET. You still have to reside to third party products. Something odd to me because I would think that they would have had plenty of time to see how their C# colleagues did it . I guess they give the Visual Studio Eco system some crumbles from the table ...

DevExpress offers a FREE refactor product for Visual Studio. There was already a free version for VS2005. Now they have updated their refactor utility for VS2008. But the good news is that you also can use it in VS 2005. Moreover by registering, you are entitled to download and use a set of bonus refactorings for Visual Basic .NET including:

Create Method Contract
String Composition to String.Format
Rename Local
Safe Rename

Check out their web-site (http://www.devexpress.com/Products/NET/IDETools/VBRefactor/) for a complete overview of the refactors the add-on offers.

The latest incarnation is called Refactor! Free VB.NET with Bonus Refactorings v2.5.8 release 11 october 2007

Standard Version for Visual Studio 2005, 2008 (size 20,197,328 bytes)
Bonus Refactorings for Visual Studio 2005, 2008 (size 222,550 bytes) Only after registration.

An example of a refactor form the bonus pack : String.Format. Very handy!

Here some screenshots while executing the String.Format refactor

Regards,

Alexander

RDB Transactions [HYC00]

2007-10-22T10:35:00.000+02:00

Today I got an error message when I tried to connect to an RDB data server through the ADO.NET ODBC brigde surrounded in a System.transaction. I'm using the ODBC driver for Oracle 3.02

So the following piece of code gives an error

Using tx As New TransactionScope
Using con As New Odbc.OdbcConnection
con.ConnectionString = "Driver={Oracle RDB Driver};server= ......"
con.Open()
' do your work here
tx.Complete()
End Using
End Using

The following error is thrown : System.Data.Odbc.OdbcException: ERROR [HYC00] [Oracle][ODBC]Optional feature not implemented. Meaning that the driver does not support the version of ODBC behavior that the application requested.

but when i slightly change the code, it works fine

Using con As New Odbc.OdbcConnection
con.ConnectionString = "Driver={Oracle RDB Driver};server= ......"
con.Open()
Using tx As New TransactionScope
' do your work here
tx.Complete()
End Using
End Using

Also using ODBCTransaction Object works fine

Dim tx As Odbc.OdbcTransaction = Nothing
Dim con As System.Data.Odbc.OdbcConnection = Nothing
Try
con = New System.Data.Odbc.OdbcConnection
con.ConnectionString = "Driver={Oracle RDB Driver};server= ......"
con.Open()
tx = con.BeginTransaction
'do your work here
tx.Commit()
Catch ex As Exception
If tx IsNot Nothing Then tx.Rollback()
Throw
Finally
If tx IsNot Nothing Then tx.Dispose()
If con IsNot Nothing Then con.Dispose()
End Try

So morale of the story ... somewhere System.transactions uses ODBCtransactions ?

Any info is welcome!

Regards,

Alexander

VISUG-devmatch 18/10/2007 - Datasets vs. OO

2007-10-19T08:07:00.000+02:00

During this evening event , the Belgian Visual Studio user group (http://www.visug.be/) invited two speaker to present their views regarding datasets-oriented development and a more object-oriented way of working respectively.

Kurt Claeys (Ordina) was the advocate for the dataset approach while Yves Goeleven (Compuware) "preached" the Domain-Driven Design(DDD) way. Both did a good job !

Benefits (What did I learn)

Use dataset-approach in application that needs to be build fast.
Use dataset-approach If you are more at ease with the relational model than OO-models.
Use a dataset to persist information on a PDA . You can use the dataset xml features for that.
Check out the new dataset features in VS2008 . You can separate the generated TableAdapters from the dataset type in different projects. So you make a assembly with dataset types that are shared on several layers and/or tiers. There is also a "manager" that handles multi-table updates.
Microsoft continues to invest in Datasets (VS2008 , LINQ to Datasets).
But Microsoft also tries to keep up with the DDD camp (Entity Framework)
There is also stuff in between LINQ to SQL
Domain-driven design(DDD) is object-oriented design with emphasis on the problem domain. The domain model is the basis. I should catch up on that (http://www.domaindrivendesign.com/).
DDD is difficult. Expect big learning curve.
Don't expose domain layer immediately to UserInterface layer. You should consider other objects to transport the information of an entity (DDD for principal domain object representing something valuable for the business (customer order, etc).
Use an ORM to help you bridge the Object-Relational mismatch
You need involvement from all stakeholders to do DDD to practice the Ubiquitous Language principle.

Concerns (What did I miss)

Example of DDD applications (or parts of ) to see side-by-side difference. Kurt prepared a lot of example applications to support his viewpoints.

Best regards,

Alexander

Datasets versus custom objects

2007-10-17T09:10:00.000+02:00

Here is a list of benefits and concerns for each approach I compiled from various Internet sources and books. If you see other points or disagree with (some) things , don't hestitate to drop a line.

Datasets (.NET2.0)

Benefits

Disconnnected data-container out-of-the box
Typed datasets enhance ease-of-programming compared to "plain" datasets
Table dataAdpaters automatically generated
Visual designer in VS.NET to create typed datasets manually or via server explorer
Add new data source wizard creates Typed datasets from database schema.
Full two-way data-binding capabilities
Built-in support to receive notifications when something changes inside.
Search/sorting capabilities out-of-the-box (some help of DataView class)
Work together with data adapters to retrieve and persist data.
You can define relationships between tables (referential constraints)
You can define constraint on columns (Uniqueness)
Can hold many kinds of data types (.NET framework)
The DataSet is serializable out-of-the-box (also binary )
Has integrated XML capabilities
Has built-in support for optimistic concurrency
You can add custom logic in typed Dataset partial classes
Using annotations, you can change the names of contained objects to more meaningful names without changing the underlying schema, making code easier for clients to use.
handle NULL values out-of-the box
Lot of third party controls support datasets
Multiple version of a column value can exist and is available out-of-the-box (row state , GetChanges, Merge)
With SQLdataAdapters you can load muliple tables in a dataset at once

Concerns

Everything is represented in .NET Object type. There is a lot of casting/boxing going on
Datasets used in webservices are not ideal from interoperability standpoint
relational API maybe too restrictive (Tables and Relations properties). Need for special methods to represent/manipulate business entities.
DataSet is tied directly to the database model. Abstractions are more diffeiclut . you must adhere to thinking in tables and related concepts.
Inheritance. Your typed dataset must inherit from DataSet, which precludes the use of any other base classes.

Custom Collections

Benefits

provide the means to expose data in easy-to-access APIs without forcing every data model to fit in the relational model. You can still make a one-to-one mapping with the database but you can more easily use OO technisues to model the your problem domain.
Advanced relationships like inheritance are possible.
You can add any behaviour that is needed. Custom entities can contain methods to encapsulate (simple) business rules. Custom entity classes can perform simple validation tests in their property accessors to detect invalid business entity data.
custom class can be marked as serializable
Code can be easier to understand/maintain
LINQ To SQL, Entity Framework are special features in future version of .NET. So MS recognizes benefits of this way of working (or is it under market pressure :)
using custom classes makes for easier unit testing

concerns

Programming effort can be bigger than all-dataset scenario
you must implement a interfaces in order to provide for effective containment and data-binding capabilities
Mapping custom collections and entities to a database can also be a complicated process that can require a significant amount of code (need for code generation/ORM tool)
To support optimistic concurrency time stamp columns must be defined in the database and included as part of the instance data.
Support for multiple versions of data state within an entity must be coded.
No Searching and sorting of data out of the box. You must define your own mechanism to support searching and sorting of entities.

Sources & recommended reading

DataSets vs. Collections, Dino Esposito, http://msdn.microsoft.com/msdnmag/issues/05/08/CuttingEdge/default.aspx
Designing Data Tier Components and Passing Data Through Tiers , MSDN Practices&patterns, http://msdn2.microsoft.com/en-us/library/ms978496.aspx
Dataset versus objects, Paul Stovell, http://www.paulstovell.net/2006/10/18/DataSets+versus+Objects.aspx
Data Binding with Windows Forms 2.0: Programming Smart Client Data Applications with .NET , Brian Noyes, Microsoft .NET Development Series

SSIS and RDB

2007-10-15T09:08:00.000+02:00

Hi,

Facing some problems with the connectivity towards RDB, we were looking towards solutions

Problems

Schema information did not appear in SSIS wizards/Tools. For example character columns were represented as nvarchar (0).
Precision with numeric columns

Alternatives

After scanning the Internet for solutions, we came up with following alternatives

Buy a OLE-DB driver for RDB (Attunity)
Emulate RDB as an Orcale Database and use the OLE-Db driver for Oracle
New ODBC driver for RDB (3.0) from Oracle.

Solution

Fortunately a third option worked for us (free !)

Check out
http://www.oracle.com/technology/software/products/rdbodbc/index.html

Regards,

Alexander

SAI conference : KBC .NET factory

2007-10-12T15:47:00.000+02:00

KBC .NET software factory

Yesterday (11/10/2007) I attended a conference organized by SAI (www.sai.be) about Agile development and a .NET software factory as used at the KBC group (www.kbc.be) , a banking and insurance group.

The presenters (Peter Bauwens, Jan Laureys, Kim Verlot) did a very fine job in explaining and sharing their experiences with setting up a Software delivery & Maintenance center for .NET application with the KBC .NET software factory .

I'll try to sum up some point in a benefits & concerns style, meaning the things I 've learned and the things I wish I understood better.

Benefits

Software factories (in the technology sense of the word) alone don't cut it. There is also a process part and an organization(people) part that are equally, if not more , important than the technology at hand.
Pragmatism is key. They were not afraid to mix more "classical" engineering approaches in their agile methodology: The key phrase of Jan was : being agile in an agile way. for example scrum was expanded with an envisioning and a kind of requirements gathering period. After the scrum sprints, a phase for stabilization and transition was included.
Visual Studio Team foundation was a big enabler in this way of working
Only allow a fixed application architecture with some variability (winforms, Asp.net, batch)
Always question your factory: it is never finished. Incorporate feedback of the developers.
Only work with people who believe in this way of working because not the technology is the biggest hurdle (you have a lot of training possibilities) but the process asks a particular kind of mentality.

Concerns

Having authority to impose certain things (process, technology) requires managemente involvement from day one. I wish I knew who I make a business case to invest in a software factory an SDC & Maintenance center
While having a very flat project structure (project administrator, Lead developer, developers) makes communication easy I would say nevertheless I found the "burden" and responsibity they put on the LDEV very big. He combines, in variuos degrees, technology , process and domain knowledge. While writing user stories and talking (and thinking with) the "business" , he also was responsible for keeping the process on track and prefereable also knew the technology. I did not hear anything about system analist or functional analysist or business analist being a part of the team writing the use cases. I wish I knew how I could manage that.
Project assignements go through a process of determing their criticality before being assigned to the development are's withing the KBC group (Low level SLA application and High Level SLA applications). Until now the delivery centers worked on LOLA apps. i wish I knew how I could organise a delivery center for HILA apps with .NET as major technology platform?
The technology pilar changes at a relatively fast pace. When they started in 2005/2006. Some technology from Microsoft wasn't mature enough (Microsoft Software factories) .I wish I knew if it would a safe bet now to go along that path to make our software factory/Delivery center.

Once again, congratualation for the presentors.

Best regards,

Alexander Nowak

Unit tests (Part 2)

2007-10-07T21:56:00.000+02:00

Working definition

Unit testing is a procedure used to validate if “units” of source code are working properly. More technically one should consider that a unit is the smallest testable part of an application. In an Object Oriented program, the smallest unit is a Class or more interesting, its operations.

Unit testing is done by the developers and not by end-users or separate QA. A unit test is a piece of code that calls the unit under test in particular way in order to find errors during development. Each Unit test verifies if it returns the predicted answer.

Automation is key in the success of unit testing because it gives the developers a way to create structured unit tests that can be executed easily and repeatable and can be verified through assertions in the test code.

Benefits

The goal of unit testing is show that the individual parts of the code are working correctly. In doing unit testing you gain several benefits

Raise level QA mindset of the developer
Raise confidence in the quality of the code
Tests are run repeatedly
Early warning detection system for programming and/or design errors or flaws
Living documentation on the API of the “units”
Aid for regression testing
Code refactoring is more stimulated

When writing unit tests you should think like a tester, not just a developer. The time you take to design your unit tests will help reduce the time spent resolving defects later. Focus on the details of your objects: How is data transferred between them? Who consumes them? How easy can you break the object? What happens if I "do this"?

When a developer is programming some functionality he usually writes in some way or another some testing code to see whether his implementation does what is supposed to do and if it doesn’t produce errors. The developer then moves on to program the next piece of functionality. Again he assures that run as expected. Maybe the developer moves on other functionalities without coming back to test previous piece of code until release data. With automated unit testing, the developer has a method so the test can be run repeatedly as the code is developed.

When you have a suite of unit test that effectively test your code, you can be confident that your code will have a low likelihood of errors. The confidence in the quality of the code can be increased.

The act of writing tests often uncovers design or implementation problems. The unit tests serve as the first users of your system and will frequently identify design issues or functionality that is lacking.

Once a unit test is written, it serves as a form of documentation for the use of the target system. Other developers can look to unit tests to see example calls into various classes and members.

Perhaps one of the most important benefits is that a well-written test suite provides the original developer with the freedom to pass the system off to other developers for maintenance and further enhancement. Should those developers introduce a bug in the original functionality, there is a strong likelihood that those unit tests will detect that failure and help diagnose the issue. Meanwhile, the original developer can focus on current tasks.

It takes the typical developer time and practice to become comfortable with unit testing. Once a developer has been saved enough time by unit tests, he or she will latch on to them as an indispensable part of the development process.

Unit testing does require more explicit coding, but this cost will be recovered, and typically exceeded, when you spend much less time debugging your application. In addition, some of this cost is typically already hidden in the form of test console- or Windows-based applications. Unlike these informal testing applications, which are frequently discarded after initial verification, unit tests become a permanent part of the project, run each time a change is made to help ensure that the system still functions as expected. Tests are stored in source control very near to the code they verify and are maintained along with the code under test, making it easier to keep them synchronized.

Unit tests are an essential element of regression testing. Regression testing involves retesting a piece of software after new features have been added to make sure that new bugs are not introduced. Regression testing also provides an essential quality check when you introduce bug fixes in your product.

It is difficult to overstate the importance of comprehensive unit test suites. They enable a developer to hand off a system to other developers with confidence that any changes they make should not introduce undetected side effects. However, because unit testing only provides one view of a system's behavior, no amount of unit testing should ever replace integration, acceptance, and load testing.

In the same line but when refactoring code (changing code without changing the intend) to improve design in terms of maintainability, performance, etc. are more stimulated if you have a battery of unit tests. You’re more confident when you refactor some code. You can easily check if the code change is done correctly.

Automated unit testing should help reduce the amount of time you spend in the debugger. However, if testing results and code coverage do not help in providing the reason why your test is failing, don't be afraid to debug your unit tests. In Visual Studio 2005 Team System, developers can debug their unit testing assemblies using the Debug Selected tests option in Test Manager. You can also debug Nunit test.

Concerns

Unit testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors (except when you consider a component or subsystem as a unit) , performance problems or any other system-wide issues. In addition, it may not be easy to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.

It is unrealistic to test all possible input combinations for any non-trivial piece of software. Like all forms of software testing, unit tests can only show the presence of errors; it cannot show the absence of errors. Writing unit tests for every single class to test every aspect of every function is a tall order. Even with a large number of unit tests, is complete testing possible (or desirable)?

What if there are bugs in the unit tests? Where there is code, there will be bugs. You’ll be getting false positives.

Not all code can be unit tested (at least not easily). It could, of course, be argued that in this case, the code should be simplified until it can be unit tested.
Passing a Unit test only means that no the test it self did not catch any problem. All the assertions were correct. This doesn’t mean that there are no bugs. The success depends on the quality of the unit test as well.

Automation

Unit testing in spirit has existed since the beginning of programming. Every environment, development shop or even programmer had/has its way of conducting these kinds of tests. For example creating a form with 20 –odd buttons calling various part of your code base.
This “Form with buttons where each button performs a functionality of the code to be tested” –approach work s but you need a manual operation for each single test: clicking the button. The main problem is that you need to have the discipline to click on very button after very change. Secondly, it is very easy to drag a button on a form and hook and eventhandler leaving the name of the button to its default. If you revisit the form a after some time, chances are you don’t remember to meaning behind the button. It will be even more difficult for someone else interpreting the test behind all the buttons. Thirdly, knowing if the test succeeded or not , you had to make assertions in your code , or make the assertion visually but also you needed a way to give feedback to the developer ( eg infamous msgbox).

Third-party vendors of course offer solutions to the testing problems but they are usually very expensive and introduce their own, proprietary scripting language to conducts test.

The xUnit framework was introduced as a core concept of eXtreme Programming in 1998. It introduced an efficient mechanism to help developers add structured, efficient, automated unit testing into their normal development activities. This Toolkit lets you develop tests using the same language and IDE that they are using to develop the application. The xUnit framework defines concept of "test runner" as the application responsible for (a) executing unit tests, and (b) reporting on the test results. Automated unit tests are based on "assertions," which can be defined as "the truth, or what you believe to be the truth." From a logic standpoint, consider this statement, "when I do {x}, I expect {y} as a result."

The atomic parts of a unit test fixture are the test methods that make assertions about the behavior of some (or all) public members of the unit you're testing. A test suite is a library assembly that contains one or more such test fixtures, along with optional setup and teardown methods. You normally run this assembly from a GUI app, but these unit tests can run also from a command-line and be activated from other applications (build process).

With the help of this testing automation framework your unit tests are

Test can be written in same language as the tested unit
Structured.
Self-documenting.
Automatic and repeatable.
Designed to test positive and negative actions.

Open questions

Testing should become an integral part of the programming effort , not some things that comes afterwards. But how can you educate your team? How can you convince management
How to ensure that having a lot of unit tests does in fact make tracking bugs easier?
How to make sure your tests are easy to maintain?
How to write readable unit tests

References and recommend reading

Write Maintainable Unit Tests That Will Save You Time And Tears, Roy Osherove http://msdn.microsoft.com/msdnmag/issues/06/01/UnitTesting/
Qualities of a Good Unit Test http://codebetter.com/blogs/jeremy.miller/archive/2005/07/20/129552.aspx
Coder to Developer: Tools and Strategies for Delivering Your Software , Mike Gunderloy , ISBN:078214327X
Pro Visual Studio 2005 Team System ,Jeff Levinson and David Nelson , ISBN:1590594606
.NET 2.0 for Delphi Programmers ,Jon Shemitz , ISBN:1590593863
Extreme Programming Refactored: The Case Against XP Matt Stephens and Doug Rosenberg ISBN:1590590961
Professional Visual Studio 2005 Team System ,Jean-Luc David et al. ISBN:0764584367
Code Complete: A Practical Handbook of Software Construction ,Steve McConnell , ISBN:1556154844
Test-Driven Development in Microsoft .NET, James W. Newkirk and Alexei A. Vorontsov ,ISBN:0735619484
Pragmatic Unit Testing in C# with NUnit: The Pragmatic Starter Kit, Volume II (Volume Set) , Andy Hunt and Dave Thomas ,ISBN:0974514020
Why Agile Software Development Techniques Work: Improved Feedback: http://www.ambysoft.com/essays/whyAgileWorksFeedback.html
http://en.wikipedia.org/wiki/Unit_testing
The Humble Dialog Box (http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf )
NUnit (http://www.nunit.org/ )
http://martinfowler.com/eaaDev/uiArchs.html

Testing types classification

2007-10-05T12:56:00.000+02:00

There are different types of test you can conduct. These test types have different objectives and characteristics. There are several classification criteria to organize these types of test.

Who uses the information of the test results? (development team, end-user, operations , Audit firms, etc)

Phase in development cycles (Requirements gathering, Analysis, Design , Construction , etc)

Phase in life cycle of a application (New versus maintenance)

Who establishes and/or executes the test? (Development team, Test organization, end-user)

Dynamic tests versus static tests

Functional test versus non-functional test (performance, load, security ,etc)

Sometimes the classification name for type means one thing for one person and another for person. I tried to compile a list of test types from several Internet resources and books. I don't give any formal definition but try to specify the test type through its characteristics.

Objective

How are the tests executed?

Who executes the tests?

Who is the main interesseted party in the test-results?

Where are the tests executed?

When are the tests executed?

Static testing

To find errors before they become bugs early in the software development cycle.
Code is not being executed.
Usually take the form of code reviews (self-review, peer–review, walk-troughs, inspections, audits) BUT can also be conducted on level of requirements, analysis and design.
Usage of check-list for verification.
Code analysis preferably executed through tools.
Executor : Individual developer , Development team, Business analyst / Customer in case of requirements testing ,Business analyst in case of analysis reviews, Architect/Senior designer in case of design review
Primary beneficiary is the project team self
Can be executed on the private development environment
Can be executed in separate environment ( test environment , build environment in Continuous Integration scenario)
Code review usually in construction phase; Audits tend to be later in the cycle.
Reviews on requirements, analysis and design are executed early in the project

Unit testing

A.k.a. Program testing , Developer testing
The main objective is to eliminate coding errors
Testing the smallest unit in programming environment (class , function , module)
Typically the work of one programmer
In OO environment usage of Mock Libraries/Stubs to isoloated units
Executed by the developer
Feedback immediately used by the developer.
Test results are not necessarly logged somewhere.
Executed on the private development environment. So Tested in isolation of others developers.
But can be executed in separate environment ( build environment in Continuous Integration scenario)
Executed during construction phase

Unit Integration testing

A.k.a. Component testing, Module testing
To check if units can work together
It is possible for units to function perfectly in isolation but to fail when integrated
Typically the work of one programmer
Executed by the Developer
Test results immediatly used by Developer
Test results are not necessarly logged somewhere.
Executed on the private development environment. So Tested in isolation of others developers.
Can be executed in separate environment ( build environment in Continuous Integration scenario)
Exectued during construction phase

System testing

Compares the system or program to the original objectives
Verification with a written set of measurable objectives
Focuses on defects that arise at this highest level of integration.
The execution of the software in its final configuration, including integration with other software and hardware systems
Includes many types of testing: usually strong distinction between functional and non-functional requirements like usability, security, localization, availability, capacity, performance, backup and recovery, portability
Who's testing depends on sub type but usually a separate tester role within the project team
Test results are used by the Project team or Operations (performance , volume , -> capacity)
Executed on the separate test environment
Generally executed towards the end of construction when main functionality is working.

Functional testing

Finding discrepancies between the program and its external specification
Test result used primarily by the project team
focuses on validating the features of an entire function (use case)
Main question: Is it performing as our customers would expect?
White box approach
Single-user test. It does not attempt to emulate simultaneous users
Look at the software through the eyes of targeted end-user.
Preferable separate tester (non-biased)
Separate test environment
Generally more towards the end of construction

Beta testing

Check how the software affects the business when a real customers use the software
"Test results" used by customer and project team
Usually not a completely finished product
Sometimes used in parallel with previous application on the end-user environment.
Is usually not structured testing. No real test cases established. Application is used in everyday scenarios. Although some end-users use implicit error guessing techniques to discover defects.
Executed by selected end-users
In separate test environment or real production environment
It can be conducted after some construction (iterations) but normally before formal acceptance tests.

Acceptance Testing

Executed by the Customer or some appointed by the customer (Not the delivering party)
Determine if software meets customer requirements and to whether the user accepts (read pays) for the application.
Who defines the depth of the acceptance testing?
Who creates the test cases?
Who actually executes the tests?
What are the pass/fail criteria for the acceptance test?
When and how is payment arranged?
Test results primarily used by customer but are handed over to project team . Could be that application is accepted under some conditions . A.k. known bugs.
Separate test environment or Production environment
Executed after complete testing by delivering party

Performance testing

Searches for bottlenecks that limit the software’s response time and throughput
Results primarily used by development team and Operations
To identify the degradation in software that is caused by functional bottlenecks and to measure the speed at which the software can execute
Kind of system test
Mimic real processing patterns , mimic production-like situations
To identify performance strengths and weaknesses by gathering precise measurements
Don’t intentionally look for functional defects
Executed by separate test role
Executed on the separate test environment
Generally toward the end of construction unless performance-critical components are identified beforehand (calculations, external connectivity through WAN/Internet, etc)

Usability testing

Check if human Interface complies with the standards at hand and UI is easy to work with
Kind of system test
Test results serve the development team and end user
Components generally checked include screen layout, screen colours, output formats, input fields, program flow, spellings, Ergonomics , Navigation , and so on
Preferable separate tester (non-biased) conducts these test (speciality)
Executed on the separate test environment
Generally toward the end of construction

System Integration testing

To check if several sub-system can work together as specified
Kind of system test
Development team
Larger-scale integration than unit integration
Generally combines the work of several programmers.
Separate test role with the project team (delivering party)
Test result used by project team
Executed on the separate test environment
During construction when workable sub-systems are ready to be tested (i.e. basic functionality works in happy path)

Security testing

Tries to compromise the security mechanisms of an application or system.
Kind of system test
Need for separate test environment to mimic production environment
Separate test role
Executed at end of construction unless is primordial in the application

Volume testing

Other terms or specialisations Scalability testing, Load testing, Capacity testing
To determine whether the application can handle the volume of data specified in its objectives (current + future). So what happens to the system when a certain volume is processed over a longer period of time (resource leaks, etc)
Test results important for project team and Operations
Volume testing is not the same as stress testing (tries to break the application)
Separate test role
Executed on the separate test environment
Generally towards end of construction unless load/scalability is a critical success factor

Stress testing

To find the limits of the software regarding peak data volumes or maximum number of concurrent users
Interested parties are : Project team and Operations
Create chaos is the key premises.
Go beyond potential processing patterns
Try to demonstrate that an application does not meet certain criteria such as response time and throughput rates, under certain workloads or configurations
Kind of system test
Don’t look for functional defects (intentionally)
Need for separate test environment to mimic production environment
Special test software used to simulate load /concurrent users
Separate test role
End of construction

Availability testing

To assure that our application can continue functioning when certain type of faults occur.
Test in function of fail-over technology.
Don’t look for functional defects (intentionally)
Need for separate test environment to mimic production environment
Special test software used to simulate certain fault(network outage, etc)
Separate test role
End of construction

Deployment testing

A.k.a Installation testing
To verify installation procedures
Important for Release manager and Operations (start up/ business continuity)
checking automated installation programs
Configuration meta data
Installation manual review
Conducted by separate test role
Separate environment by default (clean machine principle)
Construction during stage promotion for example

Regression testing

To avoid that software goes into regression in light of changes or bug fixes
Regression bugs occur whenever software functionality that previously worked as desired stops working or no longer works in the same way that was previously planned. Typically regression bugs occur as an unintended consequence of program changes
To make sure a fix/change correctly resolves the original problem reported by the customer.
To ensure that the fix/change does not break something else.
On different levels Unit, integration , system, functional
Executor depends on level of test
Primary interested party depends on level of test
Executed during Construction or Maintenance

Several Other Specialised tests

Testing for operational monitoring: Do faults we simulated find their way in the monitoring environment of our application?
Testing for portability: Does our product work on the targeted Operation Systems?
Testing for interoperability;For example does our software work with different targeted database systems?
Testing the localized versions.
Testing for recoverability after certain system failures.
etc.

Test Specification techniques

2007-10-04T09:13:00.000+02:00

Context

I assembled some information about techniques you can use to specify the test cases to see whether the code contains errors while assuring that the test cases covered all the code and possible conditions.

Generally the techniques are divided in two groups ; Black-box and white-box testing techniques where black-box techniques treat the unit only from an API standpoint and the white-box techniques require full understanding of the code.

It is important to note that some formal techniques can take a lot of effort to so I suggest a more pragmatic approach based on a combination of techniques.

There are different kinds of testing but for the discussion of this text we will use the following taxonomy.

Unit testing is a process of testing the individual subprograms (classes), subroutines or procedures (methods) in a program. Most of the times these “units” are tightly coupled with other modules. Sometimes the latter are stubbed out to assure that we only test the unit under test and not the secondary units. Unit integration testing concerns the inter-operation between the different units a developer has made.
Component testing or subsystem testing (unit integration) verifies a particular portion of the application, for example the data access layer code. This for example requires the integration of a database server.
System test is a higher level test that verifies non-functional characteristics at the entry point of a system. For example performance, stress, volume, etc.
Functional tests test the system much like a client would use the application. Things like the order of input , user error messages , etc

Some of the techniques discussed are more fit for a particular type of testing than others . I will concentrate on the techniques you could use in developer testing (unit and unit integration testing).

Test psychology

Is testing something we do to show that a piece program is working as expected or that it contains no errors or to find (all) errors? Or everything mentioned?

This might seem like subtle differences in semantics to some readers but it is actually quite strong because it gives you a different starting point while specifying test cases. Testing should have something destructive in nature. You’re going to write test cases to break the code; At least that should be your mindset while testing.

Testing has the purpose of increasing the reliability of your code. So actually when a test case finds an error , the test case should be mentally regarded as a success because it found an error and we can fix it before it was put through acceptance testing or worse, in production. Of course the meaning of success is double-faced because once we fix the code and execute the test case again we should not find the error anymore. The test-case passed successfully, meaning this time processing the correct result.
So saying that my piece of code is working as expected so not the ideal mindset because you can more easily write test case that will demonstrate that. That being said these “positive attitude“ test cased should nevertheless be included into your collection test cases to assure you code coverage objective.

Proving that your program contains no errors is very difficult to state because the psychological burden to achieve this is too big for a programmer. It is better to uncover as much as errors as possible. By increasing the test case you have, you’re more likely to find more errors. The more errors you find, an of course fix, the more confident you will be of your code. This will reduce the stress you’re feeling.

Typical Software defects and their causes

This is a huge area but ranging from wrong error messages to unable to free unused memory or missing database fields because of incomplete specifications. I will narrow the list to some typical coding errors and their sources.

Error against numeric boundaries
Wrong number of loops
Wrong values for constants
Wrong operation order
Overflows
Incorrect operator used in comparison
precision loss due to rounding or truncation
Unhandled case in logic
Failure to initialize a loop control variable
Failure to (re)initialize
Assuming one event always finishes before another
Required resource not available
Doesn't free unused memory
Device unavailable
Unexpected end of file
Wrong data types used

Test strategies

So how should we go about finding all errors in a program? We could try it by using every possible input condition as a test case. This would lead quickly to huge amounts of test cases with usually negligible added value. So what is out test-case design strategy?

One important testing strategy is black-box testing. Your goal is to be completely unconcerned about the internal behaviour and structure of the program. Instead, concentrate on finding circumstances in which the program does not behave according to its specifications. In this approach, test cases are derived solely from the specifications (i.e., without taking advantage of knowledge of the internal structure of the program).

Another testing strategy, white-box testing, forces you to examine the internal structure of the program. This strategy derives test data from an examination of the program’s logic.
In finding test cases, the following paragraphs sum up several techniques you can use to find appropriate test cases. It is not a “cook-book –like” description. It merely tries to serve as a reminders when you faced with coming up with test cases.

It neither a mutually exclusive set of techniques. As a matter a fact, the recommended procedure for example unit testing is to develop test cases using the black-box methods and then develop supplementary test cases as necessary with white-box methods.

Sources for test case specification

Depending on the type test some sources are more appropriate than others .

Equivalence classes

Exhaustive-input test of a program is impossible. Hence, in testing a program, you are limited to trying a small subset of all possible inputs. Of course, then, you want to select the right subset, the subset with the highest probability of finding the most errors. Instead of randomly selecting a subset, you can structure your effort with technique called equivalence partioning.The equivalence classes are identified by taking each input condition (a sentence or phrase in the specification, or a parameter of a method) and partitioning it into two or more groups. An equivalence class consists of a set of data that is treated the same by the module or that should produce the same result. Two types of equivalence classes can be identified: valid equivalence classes represent valid inputs to the program, and invalid equivalence classes represent all other possible states of the condition (i.e., erroneous input values).These equivalence classes will form the basis for your test cases and test case data

Here are some examples

Boundary value analysis

Boundary conditions are those situations directly on, above, and beneath the edges of input equivalence classes and output equivalence classes. Boundary value testing focuses on the boundaries simply because that is where so many defects hide. For example using > (greater than) in a condition instead of >= (greater than or equal as). "Below" and "above" are relative terms and depend on the data value's units. So rather than selecting any element in an equivalence class as being representative, boundary-value analysis requires that one or more elements be selected such that each edge of the equivalence class is the subject of a test.Another attention point is that rather than just focusing attention on the input conditions (input space), test cases are also derived by considering the result space (output equivalence classes). For example for determining the percentage of discount for a particular order, you could try to come up with a test case that would possible generate a discount more than the foreseen maximum discount. These boundary values will be the test case data values.

Here are some examples.

Decision table testing

In equivalence Class and Boundary Value testing we considered the testing of individual variables that took on values within specified ranges. Sometimes value of one variable constrains the acceptable values of another. In this case, certain defects cannot be discovered by testing them individually.This technique explores the combinations of input circumstances to come up with test cases.The testing of input combinations is not a simple task because the number of combinations can grow quicklyIf you have no systematic way of selecting a subset of a combination of input conditions, you’ll probably select an arbitrary subset of conditions, which could lead to an ineffective test.Decision tables represent actions that have to be taken based on a set of conditions. These are actually the rules. Each rule defines a unique combination of conditions that result in the execution ("firing") of the actions associated with that rule (If this and that is true than do this). If the system under test has complex business rules, presenting the system behaviour represented in this complete and compact form enables you to create test cases directly from the decision table.Create at least one test case for each rule. If the rule's conditions are binary, a single test for each combination is probably sufficient. On the other hand, if a condition is a range of values, consider testing at both the low and high end of the range. In this way we merge the ideas of Boundary Value testing with Decision Table testing.

Consider following psuedo-code

This exercise shows us that we need at least 18 different test case data sets to test all the logic.
You can merge this with explicit boundary checks.
Qty = 0, Qty = 9, qty= 10, qty = 11, qty = 14, qty = 15, qty = 16
Km = 0, km =49, km= 50, km = 51, km =99, km = 100, km=101
Of course combinations between them are also possible. You could even try a test with negative quantity to see if the code can handle that situation.

This all depends if you can look into the code to see how the decision tree is actually implemented. Some situations will result in the same outcome. You could decide to collapse the table. But be careful with collapsed tables. While this is an excellent idea to make the table more comprehensible it is dangerous from a testing standpoint. It is always possible that the table was collapsed incorrectly or the code was written improperly. Try to use, if possible, the un-collapsed table as the basis for your test case design.

Control flow testing

Modules of code are converted to graphs, the paths through the graphs are analyzed, and test cases are created from that analysis. This techniques requires are lot of preparation. Maybe usefull for safety -critical applications.

Depth of test specification

Some factors that come into play of determining the appropriate amount of testing effort (test case, time, depth , etc)
Cost of failure
Testability of the design
Life cycle of the application ; new or maintenance

Programming language

Sometimes the data types available in your programming language can help you reduce the number of test cases. Suppose you have a method in VB.NET that accepts an integer value that, according to the specs, must between 0 and 100. In case of the boundary values we can actually help reduce the number of test case because in VB.NET we have the possibility to use an unsigned integer data type (UInteger). It simply can not hold negative numbers so consequently we don’t have to specify a test that gives a negative number to the method.Enumerations can also help.

Design-by contract

In the design-by-contract approach, modules (called "methods" in the object-oriented paradigm, but "module" is a more generic term) are defined in terms of pre-conditions and post-conditions. Post-conditions define what a module promises to do (compute a value, open a file, print a report, update a database record, change the state of the system, etc.). Pre-conditions define what that module requires so that it can meet its post-conditions.More the code must use a mechanism to assert that these pre – and post conditions are met. Usually these assertions are placed inside the method. In this case the module is designed to accept any input. If the normal preconditions are met, the module will achieve its normal post conditions. If the normal pre-conditions are not met, the module will notify the caller by returning an error code or throwing an exception (depending on the programming language used). This notification is actually another one of the module's post conditions.Based on this approach we could define defensive testing: an approach that tests under both normal and abnormal pre-conditions. On the other hand if we trust the assertion mechanism we could go about and create test cases only for the situations in which the pre-conditions are met.
There are programming environments where the specification of pre-and post conditions are integrated directly in the language or language execution environment.
Should we create test cases for invalid input?

Difficult test conditions

Often modules have code that is executed only in exceptional circumstances—low memory, full disk, unreadable files, lost connections, Database locking, high network load etc. You may find it difficult or even impossible to simulate these circumstances and thus code that deals with these problems will remain untested. Still you would like to know how your code and more over your error handling code reacts on these situations.It is recommend to write that test case down and make the remark that the situation has not been tested.There are tools on the market that can help you simulate some of these conditions and check how your program reacts. For example DevPartner Fault Simulator 2.0. For example using DevPartner Fault Simulator you could observe how the application would handle the failures generated by a simulated network failure instead of you physically disconnecting a cable from the network,

Code coverage analysis

While the previous sections described techniques to specify test cases before or during programming, a code coverage analysis tool investigates your code during the execution of the current set of test cases (assuming you did some upfront test case design) if all code regions have been covered by the sets of test. It can point you the certain code path that haven’t been covered by the current set of test cases. It gives you the opportunity to specify new test case that will execute that part of your code. It can help you to eliminate gaps in your test suite.It is important to note that 100% code coverage does not equal to the fact that all errors are removed from your piece of code. It merely states that all code paths have been gone trough with your current test case suite. You still can miss out on errors simply because you’ve used a wrong comparisons operator ( > instead of >=) . So the previous techniques are still valuable. Code coverage gives you additional information that you can use. It just one of the methods to maximize the ability to find the errors in your code.Microsoft research is currently working on a tool that augments code coverage analysis with the creation of missing test case. Pex performs a systematic program analysis. It records detailed execution traces of existing test cases. Pex learns the program behaviour from the execution traces, and a constraint solver produces new test cases with different behaviour. The result is a test suite with maximal code coverage

Concluding remarks

Writing effective test cases is hard work and not always that easy as it seems to maximize error detection.
First of all, specifying test case requires a “destructive” mindset. You should find pleasure in finding errors. This is the right attitude to find as many errors as possible during your programming effort. So this implies that we write imperfect code. This is not always easy to accept for some people but nevertheless a good starting point to begin specifying test case data.

I summed up several techniques that can help you in finding the right set of test cases to find as many as errors as possible. The most important advice we can give you is to use the different methods in combination. Because only when you look at the problem from different angles, you can reasonably find the test cases that will filter out the bulk of the error conditions within the restrictions of budget and resources.

As you gradually become more familiar with the techniques you will not only come up with effective test cases but you will also think about testing while you’re coding. This will make you think how you can avoid an error condition in your code. Mind though you still will have to write out the test case even if you know your code can handle the situation.

Maybe while looking for test cases will give you more insight into the specifications that lead up to the code you’re investigating, making it more clear what you’re actually programming and how it fits in the whole picture. Maybe you will detect things that were omitted from the specification, either by accident or because the writer felt them to be obvious.

Continuing with this logic of thinking, Test-driven development puts testing completely on the fore-ground. You actually write test before writing any code. In that respect, testing becomes part of your (micro) design process. But that's for another post.

Sources and further reading

Test-driven development by example, Kent Beck, Addison-Wesley 2003
The Art of Software Testing, Second Edition, Glenford J. Myers ,John Wiley & Sons 2004
A Practitioner's Guide to Software Test Design, Lee Copeland ,Artech House 2004
BUG TAXONOMY AND STATISTICS, Boris Beizer , http://inet.uni2.dk/~vinter/bugtaxst.doc
Developing and using defect classification schemes, Freimut, B.,http://publica.fraunhofer.de/eprints/N-8167.pdf
Pex ; Dynamic Analysis and Test Generation for .NET, http://research.microsoft.com/Pex/
Code Coverage Analysis, Steve Cornett, http://www.bullseye.com/coverage.html
How Do You Know When You Are Done Testing? Richard Bender http://www.benderrbt.com/Bender-How%20Do%20You%20Know%20When%20You%20Are%20Done%20Testing.pdf
Don't be fooled by the coverage report, Andrew Glover ,http://www-128.ibm.com/developerworks/java/library/j-cq01316/index.html

Continuous Integration

2007-10-03T10:10:00.000+02:00

Early feedback for quality assurance

Continuous Integration (CI) is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. This approach can lead to reduced integration problems and allows a team to develop cohesive software more rapidly.

Build automation

Just building your application is relatively easy using a IDE : Decide whether you want a debug or a release build, and then select Build Solution or Rebuild Solution from the Build menu in for example Visual Studio .NET (VS .NET). But the manual approach is not sufficient for a project of any reasonable size. If you’re working with other developers, for example, you need to remember to get the latest pieces from your source code control system before building. You might also have unit tests to run through, documentation to update, and so on. The problem with attempting to put all the pieces together by hand is that people tend to forget things. This is where an automated build tool can be helpful with for example Nant or MsBuild.

Continuous integration

If daily builds are good, then even more frequent builds are better. After all, the sooner you spot a problem in a build, the sooner you can get to work fixing it. The logical extension of this line of reasoning is to rebuild your application every time anyone checks a change into your source code control system. This is where continuous integration (CI) steps in.

Continuous integration (CI) in that perspective is a technique to ensure the quality a software product during development where all the necessary assets of the product to build an application are under version control.

In order to integrate CI in your development process you make use of a CI application. This application monitors your source code control repository for check-ins. When it detects an updated file, it gets the current code, instructs to build the project, and look for failures. With instant feedback via e-mail or a web page, it can help you be sure that no one has accidentally checked in untested code.

Introducing CI is not only a technology implementation but also requires soft skills CI demands a certain discipline of the developer and project/team leaders to make CI give its benefits. The developer may not check in untested code and all the team members must interpret the reports generated by the CI application and do something with it if required (build failure, code complexity, etc.)

The CI process consists of the following steps:

SourceControl system monitoring
Building
Unit testing
Coverage Analysis
Static Code Analysis
Documentation Generation

These steps (some are optional) are executed by the CI build server every time a modification is detected in the source code. Each of these steps produces data, which is stored in a repository. A project portal is used to present these data items, in various reports.

All artifacts in order to build a working software product should be put into a source control system like Visual SourceSafe. The CI process that runs on the build server continuously scans the source control system for changes, and starts the build process as soon as a change is detected. The build server gets the latest version of the code base, and builds the software in one or more configurations. If the build fails, the build process is aborted and the build is considered to be broken. A broken build is a serious situation within CI, which should be remedied as soon as possible.

If the build succeeds, the other CI steps will be performed and eventually the build output will be copied to a drop location.

Unit testing and Code coverage analysis

The CI environment will run a set of specified unit test on the build server, each time integration of new functionality takes place. During these unit tests, code coverage is measured and registered by the build server. Code coverage provides developers and testers with information about what code is touched when a test is run. The results of the unit tests and the coverage analysis metrics are published on the project portal.

Static Code Analysis and Code metrics

Static code analysis checks code for conformance to the Design Guidelines and/or naming guidelines. The CI environment can perform Static Code Analysis on the build server, every time new functionality has been checked in to source control. A software metric is a measure of some property of a piece of software or its specifications. Tools can gather some code metrics on basis of the source code. These metrics can be used to track some aspects of your code that can influence some software quality aspects : maintainability, readability, etc.

Documentation Generation

The CI environment can automatically generate documentation during the build process. This is usually technical documentation that describes the internal structure of the application, such as classes and data structures. For example Ndoc or DoxyGen.

CI output

The CI environment will publish all the outcomes of the different workflows on a portal and can inform interested parties about the build outcome.

Conclusion

So the main purpose of CI is the early detection of integration issues and furthermore is quality improvement measure.

CI can help you measure cyclomatic complexity, code duplication, dependencies and coding standards so that developers can proactively refactor code before a defect is introduced.
If a defect is introduced into a code base, CI can provide feedback soon after, when defects are less complex and less expensive to fix.
CI provides quick feedback, via regression tests, on software that was previously working and is adversely affected by a new change.

But you need to understand that CI is more than a technology. It requires some discipline from the developers and involvement of project management. Running CI without doing something with the information will not improve the quality of your software application. Meaning that somebody has to watch the builds and care when they are broken. Furthermore the developer must "fix" any broken CI builds.

Sources and recommened reading

Continuous Integration, Martin Fowler http://www.martinfowler.com/articles/continuousIntegration.html
Coder to Developer: Tools and Strategies for Delivering Your Software Mike Gunderloy Sybex 2004 ISBN 078214327X
Expert .NET Delivery Using NAnt and CruiseControl.NET, by Marc Holmes Apress 2005

Software factory

2007-10-03T09:53:00.000+02:00

Reuse of people,process and technology

A software factory is an approach to build applications with more or less the same structural characteristics through the use of more or less fixed combination of skilled people on the various topics within software development, processes and technology. So you say that software factories is a form of reuse. Not only "code", but also processes and people.

With a high degree of “industrialization”, it is seeking the maximum efficiency & productivity through:

Processes Standardization for project management and software development
Re-utilization & use of frameworks
Automation & tools
Skilled professionals

With these strategies it wants to reduce the development cycles (“Time to Market”) and the total cost of the project while keeping a high degree of quality for the complete cycle.
So if you have lot applications that share same structural characteristics such the development language, code organisation, and usage of third party libraries you can think about creating such a generic software Creation platform. I will be reused for all the concerned applications. More over you will be encouraged to let your future application share as much characteristics as possible. The more standardized to more benefit you’ll gain from re-use.

Usually Software factories are organized around a broad technology platform like .NET, JAVA, or SAP

Factory metaphor limitations

As with all metaphors you must not drive them to far else you’ll end up creating false expectations. Creating software is not the same as making cars or radios where predefined parts are assembled together while moving from one point in the factory to another until they are completed, checked and shipped. Software is not a mass-production item. It is not even tangible. More over, software is never finished. It build adjusted, expanded, reshaped throughout it life-time. Even during “assembly” the shape of the “product” is completely written in stone.

Microsoft definition of Software factories

Microsoft also uses the term software factories but it is not used to describe the assembly-line idea. They use it to name several Development accelerators packages. It frees the developer of creating something from scratch. The packages are materialized in the form of Visual Studio wizards, templates, project structures, components, frameworks but also best practices in the form of white papers.
De developer working with such a software factory in Visual studio will see the structure of the project so that the developer knows exactly which things to complete; which deliverables he has to work on; which are fixed for him, because they represent the architecture of that application.
So this definition does not emphasis on people and processes that much but rather on automation (technology).