Monday, 22 December 2008
VISUG Event : Entity Framework & WCF
Kurt Claeys gave a presentation about using the Entity Framework (EF) in a distributed scenario (Service oriented with WCF, Tight coupled Client-server) for the Visug , the Belgian Visual Studio user group.
His presentation was based on his personal research into the subject so he didn't gave immediately all the answers. Instead he showed us via concrete examples which difficulties he encountered during his endeavor.
The reason for going N-tier and the way to achieve this were out of scope for the session but decoupling was the key concept he used to convey this requirement.
After a very quick introduction into WCF and the EF, Kurt showed us the challenges you are confronted with if going this route and how WCF, EF or something else could provide a solution: Serialization of object graphs, contract sharing , … But the biggest hurdle in the current implementation seems to be change tracking.
In EF, a special mechanism called the ObjectContext, keeps track of all the things you do with the entity instances you retrieve through the EF infrastructure. This means the entity identification , the relationships ….and also the changes you make to the entity instances or the relations between them. Without the objectContext the EF infrastructure cannot create the necessary insert/delete/update statements for your data store.
Now a typical Service-oriented WCF service (cfr. 4 tenets of SOA ) is stateless in nature. This means that an operation that enables to retrieve an object graph through EF registers this action with the EF objectContext , but due to the nature of the service operation, the objectContext doesn’t stay around.
Kurt showed in some examples what this means for us as a developer . He also tried to come up with solution candidates to the problems like for example re-fetching the entity again and using special functionality from the objectContext the apply the changes. It worked but only for a single entity instance without relations. If you would like to go further …you had to do it yourself. Another solution candidate was to re-attach the object and apply all the changes to the re-attached object . If you had relations….you get the picture: DIY was the common factor in the candidate solutions.
There are some efforts in the EF community to give support of object change tracking in a distributed scenario but some of them are not active anymore and some are more research like efforts to look for solution.
Kurt concluded his presentation with what is coming in Vnext of EF , as told on PDC 2008.
What I remember is that if you’re going to use EF in a distributed scenario you should clearly do your homework first and recognize the potential difficulties you about to face in your situation and come up with a strategy to solve them.
Feel free to check the Visug site if you want to browse through the presentation. Maybe Kurt will also post his examples so you can try them out?
What are your experiences with EF?
Thanks for reading,
Best regards,
Alexander
Friday, 28 November 2008
VISUG event : VSTS 2010
Yesterday (27/11/2008) I attended the VISUG presentation about Visual Studio team System 2010 ( previously known as Rosario) . Pieter Gheysens gave a focused presentation on some highlights based on the VSTS 2010 CTP (November 2008).
VSTS 2010 is the third incarnation of the Visual Studio team System product portfolio. Since the previous versions (VSTS 2005, VSTS 2008) it has improved on several levels most notably on Source control management, build management ,test management and architecture & design.
Testing
- A new feature called Team System Lab Management. It is intended to ease up the configuration of testing environments using virtualization technology. (Not included in the CTP though)
- A new standalone tool code-named “Camano” . This tool is intended to help you manage test plans; test configuration, functional test cases.
A functional (manual) test execution assistant (not sure if this is part of Camano or something separate) that helps the tester to perform the tests. Pieter gave demo and showed the potential advantage of this tool. It namely can capture everything the tester was doing while executing the test. Not only the visible step through video capturing the screen but also the system information and the event happing in the application and optionally also the called methods and parameters. When a issue arising during testing the tester can link up with TFS and submit a bug item together with the captured execution data. The developer can use this information while investigating the issue (No more “it works fine on my machine” replies. A question from the audience quickly also showed that you should be careful whet you capture because every thing is stored inside TFS SQLserver databases. It was also not immediately how the test runner gathers all program execution information. Do we need to build a special instrumented version of the application when we give it to the functional tester? - Beside Web UI testing that was already available in previous version, a new test template named “coded Ui test” is available. Like in web UI testing, you can record your actions on a Winforms or WPF application and convert it into code and run it from the VS integrated test environment
- There are also a some new test window s; The Impacted Tests view provides a list of tests that need to be run and which code changes are covered by each of the tests. The Code Changes view provides a list of code changes and which tests must be run in order to validate each of them. Of course the best practice is to run all test again when you change something but I guess this can be a time-saver in some cases.
Architecture
- For better understanding code the code (for example the code base you get on your plate you never seen before and where you need to add or fix something) the new Architecture Explorer enables you to create a visual representation of existing code assets to give better insight , to see how things fit and work together. This can help you also in Impact analysis for changes.
- There are several UML model templates available. The difference with the previous Class diagram for example (beside not being UML ) is the fact that it is not linked with code. An exception is the sequence diagram that you can reverse engineer from existing code. I don’t know if it is two way though. So the most models are merely a communication medium. Being integrated in VSTS saves you the use of Visio or enterprise architect.
- A example of architectural constraint enforcement was also given. The Architecture Layer Diagram enables validation of code against a visual representation of the architecture. For example the presentation layer may not talk to class from the data access layer, you must pass a service layer. With the diagram you can hook up the class to be part of certain layer.
- I still wonder how this all fits in the OSLO vision of model-driven development .
Source control and management
- The most visually appealing demo was the branch visualization. VSTS now can depict branch and merge operation in a viewer ( a picture tells a 1000 words sometimes)
There some other topics on project management and the integration of TFS with Office Excel and Microsoft project , gated check in, workflow and parallel build management , ….and the ones I forgot . It other words there was to little time. I think Pieter have could spend an entire day showing new features of VSTS 2010.
I guess download the CTP and start playing with it.
Also...a nice gesture from Microsoft , they made several books avaible for the attendees. You had to be quick to grab one. But anyway thank you Microsoft.
There are some interesting session coming up so feel free to check out the Visug web-site (Design principles, Entity Framework, WCF)
Thanks for reading.
Best regards,
Alexander
Friday, 17 October 2008
Isolating your Dependency Injection Container
While searching for some background information about Dependency Injection Frameworks (aka Ioc containers ; ex. Windsor Castle, Unity, Spring.NET , StructureMap, etc) I stumbled on this posting by Chris Tavares (Unity creator) regarding the creation of a isolation layer for DI containers.
Because there are several DI containers available , the creation of an extra shield between your code and the DI container of your choice seems not a bad idea if you want to keep a maximum of freedom to swith from container to container somewhere in time. It could be that in the lifetime of an application another DI container might be more suitable for your problem at hand because of certain feature it has.
This is made possible with a generic interface that makes up the operations you can call from a DI container. So your code, when it needs a service from a DI container, only works against this interface. At application startup you of course need to hook up a real DI Container implementation with its own configuration semantics. For several DI containers ther are already adapters written that convert the generic API semantics to the DI container specific API calls.
You call read about it and download the code from the codeplex site.
Maybe some code snippets to get your appetite perhaps started?
In the following code snippet you see the creation of the Castle Windsor Container and the act of resolving an instance of a particular type.
class Program
{
static void Main(string[] args)
{
try
{
IWindsorContainer windsorContainer = CreateWindsorContainer();
ObjectCatalog.IMyObject obj = windsorContainer[typeof(ObjectCatalog.IMyObject)] as ObjectCatalog.IMyObject;
System.Console.WriteLine("Straight Windsor");
System.Console.WriteLine(obj.WhoAreYou());
System.Console.ReadLine();
}
catch (Exception ex)
{
System.Console.WriteLine(ex.Message);
System.Console.ReadLine();
}
}
private static IWindsorContainer CreateWindsorContainer()
{
IWindsorContainer windsorContainer;
windsorContainer = new WindsorContainer(new XmlInterpreter(new ConfigResource("castle")));
return windsorContainer;
}
}
The Castle Windsor config section looks like this
ps: I'm using SyntaxHighLighter. The XML is not shown correctly. It always set to uppercase. I will need to look it to that. (Tips are welcome!)
name="castle"
type="Castle.Windsor.Configuration.AppDomain.CastleSectionHandler, Castle.Windsor" />
id="idIMyObject"
service="ObjectCatalog.IMyObject, ObjectCatalog"
type="ObjectCatalog.MyObject, ObjectCatalog" />
Consequently your code is pretty heavily "connected" to the DI container implementation of Castle Windsor in variuos ways (code + config). Which isn't necessarly all bad because in the beginning of the project you choose Windsor for its abilities. But suppose you want (or you need ; consolidation, standardization , features , ...) to change to another DI container implementation
Let's rewrite everything to the Microsoft P'n'P Unity DI container.
class Program
{
static void Main(string[] args)
{
try
{
IUnityContainer unityContainer = CreateUnityContainer();
ObjectCatalog.IMyObject obj = unityContainer.Resolve();
System.Console.WriteLine("Straight Unity");
System.Console.WriteLine(obj.WhoAreYou());
System.Console.ReadLine();
}
catch (Exception ex)
{
System.Console.WriteLine(ex.Message);
System.Console.ReadLine();
}
}
private static IUnityContainer CreateUnityContainer()
{
IUnityContainer unityContainer;
unityContainer = new UnityContainer();
UnityConfigurationSection section = ConfigurationManager.GetSection("unity") as UnityConfigurationSection;
section.Containers["containerUnity"].Configure(unityContainer);
return unityContainer;
}
}
The config file for Unity for the example looks like this
So besides the config information and the creation of the container, you also must change every location where you use the services of the DI container (that's usually more than than one like in the example:-)
This is where the ServiceLocator comes in. It offers a generic way to talk to your DI container. The only implementation specifics are the config information and the creation of the container. So if you would need to change one day from one DI container to another, the changes kept to a minimum (although changing a DI config info from one format to another is also not to be taken light. Maybe some XSL magic !)
So how would the example look with the ServiceLocator?
class Program
{
static void Main(string[] args)
{
string ioc ="" ;
if (args.Length==0)
{ioc="Windsor";}
else
ioc=args[0] ;
try
{
IServiceLocator myLocator = CreateLocator(ioc);
ObjectCatalog.IMyObject obj = myLocator.GetInstance();
System.Console.WriteLine("Service Locator");
System.Console.WriteLine(" Implementation : " + myLocator.GetType().ToString());
System.Console.WriteLine(obj.WhoAreYou());
System.Console.ReadLine();
}
catch (Exception ex)
{
System.Console.WriteLine(ex.Message);
System.Console.ReadLine();
}
}
private static IServiceLocator CreateLocator(string ioc)
{
IServiceLocator myLocator=null;
if (ioc=="Unity")
myLocator = new UnityServiceLocator( CreateUnityContainer());
else if (ioc=="Windsor")
myLocator = new WindsorServiceLocator(CreateWindsorContainer());
else
throw new System.ArgumentException("Only Unity or Windsor are valid arguments");
return myLocator;
}
private static IWindsorContainer CreateWindsorContainer()
{
IWindsorContainer windsorContainer;
windsorContainer = new WindsorContainer(new XmlInterpreter(new ConfigResource("castle")));
return windsorContainer;
}
private static IUnityContainer CreateUnityContainer()
{
IUnityContainer unityContainer;
unityContainer = new UnityContainer();
UnityConfigurationSection section = ConfigurationManager.GetSection("unity") as UnityConfigurationSection;
section.Containers["containerUnity"].Configure(unityContainer);
return unityContainer;
}
}
So your code now makes use of a DI container generic API in order to hold on to a reference to a concrete DI Container implementation and also the calls to resolve dependencies or get instances from the DI container are written in a generic manner.
The DI Container specific calls are made in the adapters (one for each concrete DI Container). So "GetInstance" will be translated to "Resolve" for the Unity Container.
Thanks for reading.
Best regards,
Alexander
Wednesday, 15 October 2008
VISUG event : ASP.NET MVC
Yesterday (14/oct/2008) , I attended a presentation on ASP.NET Model-View–Controller (MVC) framework by Maarten Balliauw that was organized by VISUG ( a Belgian .NET community). Last web development I did was in Netscape (Who?) LiveWire (What?) and I also some ASP (thus pre .NET era). So I somehow skipped ASP.Net webforms but still followed it from the side-line. So this was an excellent opportunity to keep up-to-date with web-development.
These are some points that I learned from the presentation:
- ASP.NET MVC model is built on top of the ASP.NET framework.
- It is something you have to install separately.
- Fully integrated into Visual Studio
- It is NOT a replacement for the current ASP.NET Webforms framework
- The main goal is to allow programmers to clearly separate responsibilities according the MVC pattern. This clear separation of presentation logic, UI and the “rest” of the application is not only conceptually but also reflected in the code and project structure as created by the ASP.NET MVC project template
- You can use ASP.NET infrastructure in an ASP.NET MVC application. Cohabitation of the webform way of working and the MVC way of working is possible. You can also use ASP.NET providers like membership and roles. You can also re-use ASP.NET session and so fort.
- ASP.NET Server controls can be used as long as they don’t rely on the ViewState mechanism.
- The filter-mechanism to create attributes to annotate parts of application tha can do reccuring activities for you (err handling, logging, etc)
- A special routing mechanism for your requests is available.By default relies on a special convention of naming and structure to kick start the right controller. But you have the ability to intervene in this routing mechanism
- Also the View parts must follow a convention (naming , folders)
- You can more easily test the controller logic in “classic” unit test by controlling the dependencies with dependency injection and mocking frameworks. ASP.NET MVC plays along.
- The “car and Motor-cycle” analogy : Webforms -> car , ASP.NET -> Motor-cycle. I guess it has to do with protection , speed, and so fort . Or with the fact that you can choose between them
Some questions I still have
- Still in CTP. Maarten thinks at PDC 2008 a beta will made available and a V1 will soon follow. But how it will be packaged, I’m not sure.
- What about a MVP style implementation of ASP.NET webforms. Don’t you also get clean(er) separation, testability of presentation logic, understandability , etc ?
- Is the Web Client Software factory from Patterns& practices a viable alternative?
- In the examples Maarten showed I show a lot of in-line code in the View pages. So while server controls in webforms spit out the markup. Apparently I have to do the most your-self though Maarten mentioned some tools that can help you to alter the view rendering (Nhaml)
Some questions from the audience
- Can the controller be in separate assemblies? In the examples shown the relation between view and controller was very tight in the sense of the “actions” you can trigger from a view (so no reference dependencies). So in first line I would think this is a part of the presentation layer But according to Maarten there is someone who tampered with the routing mechanism to achieve this.
- Hooking up Winforms in this MVC mechanisms: It is called the ASP.NET MVC framework. It uses the System.Web namespace. So in the lower parts of the framework I guess all Http-stuff (get, post, etc). Maybe the MVP pattern is better suited? Has anyone used the Smart Client Software Factory?
- There was also a question about comparison , I believe, between Monorail , classic Web forms and ASP.NET MVC. I think this could be a valuable for many in order to have some “framework” to base a decision on what web development model to use (car-motorcycle analogy). One of the bullets is the presenation was : Not for everyone. So I guess there are some points to think about before embarking in a ASP.NET MVC implementation .
I’m sure I forgot some things from the presentation. If you have any comments or would like to add some points (benefits or concerns) , feel free to drop a line.
Visug has some more interesting sessions coming up ;
- ASP.NET 3.5 sp1 novelties,
- Rosario(VSTS 2010) ,
- Entity framework & WCF and
- a session by Juval Lowy (hardcore WCF).
Thanks for reading.
Best regards,
Alexander
Wednesday, 20 August 2008
Pex - Test Case 7
In this Pex test case I will investigate unit test on classes the have dependencies.( feel free to read the previous posts regarding this subject).
I'll first start with how you would unit test this scenario without the help of Pex and afterwards I'll investigate how Pex might help.
I have requirement to calculate the bonus a salesperson earns at the end of the year. The bonus depends on the sales he/she made this year and the year before.
The rules are
- The bonus amounts to 1% of the sales for that person
- If this year's sales is higher than 30000 € AND ALSO this year's sales are higher than the Sales of the previuos year for that salesperson, an extra bonus is added of 250 €
A quick test analysis should result in following test cases
currentYearSales greater or equal than 30000 € | Yes | Yes | No | No |
currentYearSales greater than LastYearSales | Yes | No | Yes | No |
Base bonus (1% currentYearSales ) | X | X | X | X |
Extra bonus (250€) | X | - | - | - |
So three test cases would be sufficient for complete test coverage (last two test cases can be collapsed).
Suppose you implements the rule in a bonus class and to know the sales amounts you have a Sales class that has appropriate methods to give you them. You've also encapsulated the retrival of these sales amounts inside the bonus class method that calculates the bonus.
public class Bonus
{
public double CalculateForEmployee(uint EmployeeId)
{
double baseBonusPercentage = 1;
double extraBonusAmount = 250;
uint minSalesForBonus = 30000;
Sales sales = new Sales();
uint currentYearSales = sales.CurrentYearSalesForEmployee(EmployeeId);
uint lastYearSales = sales.LastYearSalesForEmployee(EmployeeId);
double bonus = currentYearSales * (baseBonusPercentage / 100);
if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
bonus += extraBonusAmount;
return System.Math.Floor(bonus);
}
}
public class Sales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
}
Suppose you want to unit-test the method CalculateForEmployee in the Bonus class. The method calls into a Sales object to retrieve the sales for this year and the year before. Our code-under-test depends on another object to do its job. Nothing prevents you to write a standard unit test calling CalculateForEmployee.
[TestMethod()]
public void CalculateForEmployeeTest()
{
Bonus target = new Bonus();
uint EmployeeId = 0; // ?
double expected = 0; // ?
double actual;
actual = target.CalculateForEmployee(EmployeeId);
Assert.AreEqual(expected, actual);
}
But how do I know which EmployeeId to use and how do I know what the expected value for the bonus should be? In fact the method-under-test has dependencies we don't control in the unit test. The bonus class doesn't control the value of the sales. it's the responsibility of the sales class. As a matter a fact this situation isn't a pure unit test anymore. It's a unit integration test. So how can we control this dependency on the sales object and know in advance the EmployeeId and Sales amounts necessary for our method-under-test. Suppose a database is used to store sales information. We could setup a test database with all the necassary records for retrieving the sales for a particular sales employee. But this would still not classify our test as a unit test (only testing the rules for caculating the bonus). Also you must account for setting up three different test situation in the database.
Rewritting our classes a little bit could helps us. Interfaces, dependency injection and test doubles are techniques I will us to make the CalculateForEmployeeTest easier to test : knowing it advance what the sales amounts will be in order to assert the bonus.
First we extract an interface from our Sales class
public interface ISales
{
uint LastYearSalesForEmployee(uint EmployeeID);
uint CurrentYearSalesForEmployee(uint EmployeeID);
}
public class Sales : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
}
In this example I'll use "manual" dependency injection (constructor injection) to demostrate the dependency injection principle.
public class Bonus
{
ISales mSalesdao;
public Bonus(ISales aSalesDao)
{
mSalesdao = aSalesDao;
}
public double CalculateForEmployee(uint EmployeeID)
{
double baseBonusPercentage = 1;
double extraBonusAmount = 250;
uint minSalesForBonus = 30000;
double bonus;
uint currentYearSales = mSalesdao.CurrentYearSalesForEmployee(EmployeeID);
uint lastYearSales = mSalesdao.LastYearSalesForEmployee(EmployeeID);
bonus = currentYearSales * (baseBonusPercentage / 100);
if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
bonus += extraBonusAmount;
return System.Math.Floor(bonus);
}
}
So how does this help me in my unit test you might wonder? A test-double is a test-specific class that we will use in our unit tests so the code-under-test will not notice it that much. There are several variations of test-doubles. For example a test stub is an equivalent of the dependend class but gives "fixed answers" if we ask for something or does something in a pre-determined way.
So instead of passing a real instance of the type Sales to the constructor of Bonus, I slip in a test stub when I write the unit tests. Because it gives fixed answers , I'll need a test stub for each test case.
public class SalesStub_CYSsmaller30000 : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 29999;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 29998;
}
}
public class SalesStub_CYSgreater30000_CYSgreaterLYS : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 30001;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 30002;
}
}
public class SalesStub_CYSgreater30000_CYSsmallerLYS : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 30003;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 30002;
}
}
The unit tests use these stubs to do their thing. I create a teststub that is appropriate for the test case and inject it into the constructor of bonus.
[TestMethod()]
public void CalculateForEmployee_Lys29999_cs29998_withStub()
{
ISales aSalesDao = new SalesStub_CYSsmaller30000();
Bonus target = new Bonus(aSalesDao);
uint EmployeeID = 0; // Dummy EmployeeID not important for this example
double expected = 299; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
}
[TestMethod()]
public void CalculateForEmployee_Lys30001_cs30002_WithStub()
{
ISales aSalesDao = new SalesStub_CYSgreater30000_CYSgreaterLYS();
Bonus target = new Bonus(aSalesDao);
uint EmployeeID = 0; // Dummy EmployeeID not important for this example
double expected = 550; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
}
//and so
Another implementation for test doubles is the use of a Mock framework. This utility dynamically create test-equivalent objects that are setup (expectations) through the framework facilities . I'll illustrate this with the help of Rhino.Mock.
[TestMethod()]
public void CalculateForEmployee_Lys29999_cs29998_withMock()
{
uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose
Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
ISales salesDao = mr.CreateMock();
Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(29999);
Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(29998);
mr.ReplayAll();
Bonus target = new Bonus(salesDao);
double expected = 299;
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
mr.VerifyAll();
}
[TestMethod()]
public void CalculateForEmployee_Lys30001_cs30002_WithMock()
{
uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose
Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
ISales salesDao = mr.CreateMock();
Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(30001);
Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(30002);
mr.ReplayAll();
Bonus target = new Bonus(salesDao);
double expected = 550; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
mr.VerifyAll();
}
//and so on
The use of a Mock framework doesn't force me to explicitly write numerous of test stubs.
So where does Pex come in? For Pex to do its job I'll need a parametrized unit test. This is a unit test that will take arguments. Consider it as a test refactoring thing. Things that are common in the unit test are extracted and put into separate method/function. Pex needs this setup because it will be in charge with generating "traditional" unit tests with test data values that it will determine during its exploration phase. These generated unit test will call the parametrized unit test , passing the necessary arguments.
In our "mock" example we explicitly setup the mock for each unit test. In analogy with the parametrized unit test we need a parametrized Mock object (see Pex Tutorial).
[TestClass]
[PexClass]
public partial class BonusTest
{
[PexMethod]
[PexUseType(typeof(SalesPexMock))]
public void Calculate(ISales aSalesPexMock)
{
PexAssume.IsNotNull(aSalesPexMock);
uint EmployeeID = 9999; //dummy , not important for our test
Bonus target = new Bonus(aSalesPexMock);
double result = target.CalculateForEmployee(EmployeeID);
PexValue.AddForValidation("result", result);
}
}
[PexMock]
public class SalesPexMock : ISales
{
public uint LastYearSalesForEmployee(uint aEmployeeID)
{
var call = PexOracle.Call(this);
uint lys = (uint)call.ChooseFrom("lastYearSales", new uint[] { 29999, 30000, 30001 });
PexValue.Add("lastYearSales", lys);
return lys;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
var call = PexOracle.Call(this);
uint lys = (uint)call.ChooseFrom("CurrentYearSales", new uint[] {29999, 30000, 30001});
PexValue.Add("CurrentYearSales", lys);
return lys;
}
}
Running the PexExplorartion on the PexMethod will result in three unit test being generated.
[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate01()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)30000u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)30000u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "300");
}
[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate02()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)29999u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)30000u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "299");
}
[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate03()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)30001u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)29999u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "550");
}
Of course meanwhile you verified the test coverage results :-) in the Pex report or via the Visual Studio Test Coverage facility.
So with the PexMock framework expectations for the indivudual unit test are setup , just like we did in our Rhino.Mock example. Of we had to help the Pex mock framework a little bit just like we did with the parametrized unit test.
Thanks for reading. Of course your comments are most welcome.
Best regards,
Alexander
ps: We already had complaints of Salespersons that find this bonus rule not fair. for example sales-persons that are above the minimum sales amount eligable for extra bonus but are not progressing , don't get it. Like wise someone who is progressing doesn't receive extra incentives.... :-)
Monday, 11 August 2008
Pex - test case 6 (revisited...again)
I did some further test on test case 6 . I removed all pex assumptions regarding the dates. This time Pex had no problem finding the test cases necessary for a complete block coverage.
So the PUT states no assumptions regarding the dates;
[TestClass]
[PexClass(typeof(EmployeeServices))]
public partial class EmployeeServicesTest
{
[PexMethod()]
public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
{
PexAssume.IsNotNull(aEmployee);
PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);
PensionContribution result = target.DeterminePensionContribution(aEmployee);
PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);
if (aEmployee.Salary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aEmployee.Salary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}
}
When you run Pex this time it generated none less than 11 unit tests. The generated DateTime value are constructed with the Ticks argument, so it no easy to see the actual dateTime. Actually the first time I run this Pex generated a tick value with a value larger than 32 bit. When running the generated unit tests, an exception was thrown specifying thant the tick value should be between the DateTime.Minvalue.Ticks and DateTimeValue.Maxvalue.Ticks. It seemed to me that i generated a 64-bit value.
I haven't been able to reproduce this error though. May be it is one of the remarks in the release of this version 0.6x. I'm running on Windows XP SP2!
Nevertheless here are the generated unit tests.
public partial class EmployeeServicesTest
{
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution01()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution02()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution03()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution04()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution05()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution06()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution07()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1262076461318144L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution08()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(56623104001835008L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution09()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution10()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "0");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution11()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(633532870975488000L);
DateTime s1 = new DateTime(126226978751643648L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}
}
When you investigate the coverage results for Visual Studio and the output from Pex, you will see that both confirm the 100% coverage result.
Part of the Pex report , coverage section (click on picture for better view):
What about my "business" assumptions? Maybe it would be easier to incorporate them as pre-conditions in the code under test. For example Start date of the contract should be greater than 25/May/1968 in the setter method for Startdatecontract of our Employee class. Non-compliance would than result in an exception being thrown. how will pex cope with them?
Thanks for reading. As always, feel free to comment.
Best regards,
Alexander
Pex - test case 6 (revisited)
In the previous case I tried to come up with a scenario where a method takes a "complex" type instead of primitive types like integers of booleans. The exercise didn't go as expected. By the way , the latest downloadable version of Pex is 0.6.30728.0
To see whether pex still works on a method with primitive types, I re-wrote the method as follows.
public PensionContribution DeterminePensionContribution2(
uint aSalary,
uint aAge,
uint aServiceYears,
Contract aTypeContract)
{
//no pre-codition checks ....
PensionContribution contribution = new PensionContribution();
if (aSalary <= 3000)
{
if (aAge < 45)
{
if (aTypeContract == Contract.FullTime)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 0;
}
}
else
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 25;
}
}
}
else
{
if (aAge < 45)
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
if (aTypeContract == Contract.FullTime)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 25;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
else
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
return contribution;
}
The PUT method looks like this.
[PexMethod]
public void PUTDeterminePensionContribution(
[PexAssumeUnderTest]EmployeeServices target,
uint aSalary,
uint aAge,
uint aServiceYears,
Contract aTypeContract
)
{
PexAssume.IsTrue(aAge > 18 && aAge < 75);
PexAssume.IsTrue(aServiceYears > 0 && aServiceYears < 50);
PexAssume.IsTrue(aSalary > 1400 && aSalary < 10000);
PexAssume.IsTrue(aTypeContract == Contract.FullTime || aTypeContract == Contract.PartTime);
PensionContribution result = target.DeterminePensionContribution
(aSalary, aAge, aServiceYears, aTypeContract);
PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);
if (aSalary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aSalary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aSalary <= 3000 && aAge >= 45 && aServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aSalary <= 3000 && aAge < 45 && aTypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aSalary <= 3000 && aAge >= 45 && aServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aSalary > 3000 && aAge >= 45 && aServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aSalary > 3000 && aServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}
In no time Pex finds the test cases for full block coverage
target | aSalary | aAge | aServiceYears | aTypeContract | BaseContribution | SupplementaryContribution | Summary/Exception | Error Message |
1 | ... | 8192 | 64 | 2 | FullTime | 135 | 30 | |
2 | ... | 8192 | 64 | 30 | FullTime | 135 | 20 | |
3 | ... | 8192 | 31 | 2 | FullTime | 135 | 30 | |
4 | ... | 8192 | 31 | 30 | FullTime | 135 | 25 | |
5 | ... | 1532 | 64 | 2 | FullTime | 80 | 15 | |
6 | ... | 1532 | 64 | 30 | FullTime | 80 | 25 | |
7 | ... | 1532 | 31 | 2 | FullTime | 80 | 15 | |
8 | ... | 8192 | 64 | 2 | PartTime | 135 | 30 | |
9 | ... | 1532 | 31 | 2 | PartTime | 80 | 0 | |
10 | ... | 8192 | 31 | 30 | PartTime | 135 | 20 |
So there is nothing "wrong" with the method under test. The problem when you investigate the output window, lies maybe in the calculation of the "age" and "serviceYears" properties.
00:00:00.0> starting execution
00:00:00.0> reflecting tests
00:00:00.3> TestProject1
00:00:00.3> EmployeeTest
00:00:00.3> DeterminePensionContributionIntern(Employee)
maxconstraintsolvertime - 1s (constraint solver time out after 1 seconds)
[execution] 4 runs
[execution] 15 runs
[test] (run 20) DeterminePensionContribution
[test] (run 21) DeterminePensionContribution
[execution] 21 runs, 49/81 blocks covered
[test] (run 25) DeterminePensionContribution
[execution] 29 runs, 49/81 blocks covered
[execution] 41 runs, 49/81 blocks covered
[modelsearch] could not flip 2 branches in a row
[execution] 50 runs, 49/81 blocks covered
[execution] imprecision at testingPEX.Employee.differenceInYears, offset 0x26
[modelsearch] could not flip 4 branches in a row
[execution] 77 runs (97,40% unique paths), 49/81 blocks covered
timeout - 120s
Timeout, 1 times
MaxConstraintSolverTime, 60 times
[coverage] 49/81 block (60,49%)
00:02:01.4> [finished] 3 generated tests (0 failures), 00:02:01.3757768.
-- 0 critical errors, 0 errors, 0 warnings.
[reports] skipping report generation
EXPLORATION SUCCESS
monitored process exited with Success (0)
finished
Pex needs to find dateTimes in order to test the conditions specified in the method. I used a helper method to calculate the years to date for the age and serviceyears:
private uint differenceInYears(System.DateTime aDate1, System.DateTime aDate2)
{
//missing checks ....
//approximative
uint years = (uint)(aDate1.Year - aDate2.Year);
if (aDate1.Month == aDate2.Month)
{
if (aDate2.Day < aDate1.Day) years = years - 1;
}
return years;
}
Could it be the unsigned integer type declarations? Or are the assumptions wrongly formulated? Or both? What does "could not flip 2 branches in a row" mean?
I'll try to do some more test around this. Feel free to comment!
Thanks for reading.
Best regards,
Alexander
Wednesday, 6 August 2008
Pex - Test Case 6
In the previous Pex posts I used examples that used primitive types and/or strings as parameters for methods under test. In this post I will "test" Pex how it wil cope with a method that accepts a "complex type". This test scenario is similar to the case study in Pex - Test case 3 except we don't work directly with primitive types a method parameters or return types.
Suppose the HR department asked you to include a new module in their Salary-application that will determine the contribution each employee has to make in a private pension fund. The salary administration uses the following desicion table.
While analysing the table you conclude you can simplify the table to help you code this rule into c#.
You already have some classes in the application :
public class Employee
{
//missing elementary checks in order to keep code concise....
public int EmployeeID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
private DateTime _BirthDate;
public DateTime BirthDate
{
get { return this._BirthDate; }
set {
this._BirthDate = value;
this.Age = differenceInYears(_BirthDate, DateTime.Now);
}
}
private DateTime _StartDateContract;
public DateTime StartDateContract {
get { return _StartDateContract; }
set {
_StartDateContract=value;
this.ServiceYears = differenceInYears(_StartDateContract, DateTime.Now);
} }
public uint Salary { get; set; }
public Contract TypeContract { get; set; }
public uint Age { get; set; }
public uint ServiceYears { get; set; }
private uint differenceInYears(DateTime aDate1, DateTime aDate2)
{
//missing checks ....
//approximative
uint years = (uint)(aDate1.Year - aDate2.Year);
if (aDate1.Month == aDate2.Month)
{
if (aDate2.Day < aDate1.Day) years = years - 1;
}
return years;
}
}
public enum Contract
{
FullTime=0,
PartTime=1
}
public struct PensionContribution
{
public uint BaseContribution{get;set;}
public uint SupplementaryContribution {get;set;}
}
The rule has also been translated into c#. The method DeterminePensionContribution has a parameter of the type Employee as described in the above code snippet. The return type is the structure PensionContribution.
public class EmployeeServices
{
public PensionContribution DeterminePensionContribution(Employee aEmployee)
{
//no pre-condition checks .....
PensionContribution contribution = new PensionContribution ();
if (aEmployee.Salary <= 3000)
{
if (aEmployee.Age < 45)
{
if (aEmployee.TypeContract == Contract.FullTime)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 0;
}
}
else
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 25;
}
}
}
else
{
if (aEmployee.Age < 45)
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
if (aEmployee.TypeContract == Contract.FullTime)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 25;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
else
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
return contribution;
}
}
In a "test-after-you-code" scenario, you could use the desicion table to specify the values for actual for the different properties of a employee object that is being passed to the method under test. So 9 unit tests would be sufficient to touch all corners of the code.
A data-driven in test in Visual Studio is somewhat difficult because we are not dealing with simple datatype parameters. We could serialize the different instances and save it in string form into an external data source.
Can Pex help generate the necessary unit test to fully test our code?
Before heading to the PUT (Parameterized Unit Test) I would like to refer you to the Pex tutorial for more background information on setting up PexMethods.
A new element in this case study is the use of a factory method to create our Employee object that will be used in the generated unit test. Pex will call this method with appropriate values that it has discovered during its exploration of the code in the generated unit tests. The cool part is that Pex will first complain about not being able to create a Employee object but then suggest a soulution for it : the Factoty method and generates it for us.
[PexFactoryClass]
public partial class EmployeeFactory
{
[PexFactoryMethod(typeof(Employee))]
public static Employee Create(
int i0,
string s0,
string s1,
DateTime dt0,
DateTime dt1,
uint ui0,
Contract c0
)
{
Employee e0 = new Employee();
e0.EmployeeID = i0;
e0.FirstName = s0;
e0.LastName = s1;
e0.BirthDate = dt0;
e0.StartDateContract = dt1;
e0.Salary = ui0;
e0.TypeContract = c0;
return e0;
}
}
In the generated unit test , this factory method is used as follows:
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_095656_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626406282140467200L);
DateTime s1 = new DateTime(631149005309067264L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
Let's return to the PUT. The following PUT was based on a PexMethod that you can let the Visual Studio Pex integration build for you.
During its explorartion Pex will try out different value for the properties of our Employee object. The assumptions should help Pex create reasonable test cases. The code in this example did not enforce them in order to keep the code somewhat limited in size.
Because this method will be used for all generated unit test, the asserts have to be more general than you would normally do for a single unit test.
[TestClass]
[PexClass(typeof(EmployeeServices))]
public partial class EmployeeServicesTest
{
[PexMethod(MaxConstraintSolverTime = int.MaxValue, Timeout = int.MaxValue)]
public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
{
PexAssume.IsNotNull(aEmployee);
PexAssume.IsTrue(aEmployee.BirthDate < new DateTime(1992, 1, 1) && aEmployee.BirthDate > new DateTime(1955, 1, 1));
PexAssume.IsTrue(aEmployee.StartDateContract < DateTime.Now && aEmployee.StartDateContract > new DateTime(1968, 1, 1));
PexAssume.IsTrue(aEmployee.BirthDate < aEmployee.StartDateContract);
PexAssume.IsTrue(aEmployee.Age > 18);
PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);
PensionContribution result = target.DeterminePensionContribution(aEmployee);
PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);
if (aEmployee.Salary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aEmployee.Salary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5 )
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}
}
Running the Pex exploration several times (and after quite a while) I got results but no full coverage. Maybe the assumptions and/or asserts are not "pex-friendly"?
The generated unit tests only cover half of the code blocks.
public partial class EmployeeServicesTest
{
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_000()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110121_002()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110200_003()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626094836655304192L);
DateTime s1 = new DateTime(633479291125776385L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110201_004()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626094836655304192L);
DateTime s1 = new DateTime(633479291125776385L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_000()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112422_002()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115647_003()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(627392712227689792L);
DateTime s1 = new DateTime(633479779357556736L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115718_004()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(627392712227689792L);
DateTime s1 = new DateTime(633479779357556736L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}
}
If you have any suggestions to improve (re-factor) the test code or the code-under- test (test-friendliness), don't hesitate to drop a line.
Thanks for reading.
Best regards,
Alexander
Thursday, 31 July 2008
Pex - test Case 5
This post is a continuation in a series of post about Pex, a test case generation tool from Microsoft research. This test scenario is a variation of test case 4. (sorry in advance for the table formatting , I still can not get it right :-(
In the Pex tutorial on p37 you'll see an example of testing a Regular Expression. I didn't find one in the samples though. Too simple maybe?
I'm no hero in regular Expressions. Thankfully there's a library loaded with regular expressions for every kind of application. The following regular expressing is for ensuring strong passwords.
Expression | ^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{4,8}$ |
Description | Password matching expression. Password must be at least 4 characters, no more than 8 characters, and must include at least one upper case letter, one lower case letter, and one numeric digit. |
Matches | asD1 | asDF1234 | ASPgo123 |
Non-Matches | asdf | 1234 | ASDF12345 |
The code example where the regular expression is implemented in
public class UserAccount
{
private string mPassword;
public string Password
{
get { return mPassword; }
set {
if (value == null)
throw new System.ArgumentNullException("Please specify a valid password. (non-null)");
else
{
bool valid = RegularExpressions.Regex.IsMatch(value, @"^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{4,8}$");
if (valid)
mPassword = value;
else
throw new System.ArgumentException("Please specify a Strong Password. ");
}
}
}
}
The pex method (keeping in mind the remarks in the comments from the previous test case: the invariant remark)
[PexMethod()]
public void PasswordTest( string aPassword)
{
UserAccount target = new UserAccount();
target.Password = aPassword;
//no problem calling target.password several times, stays the same
Assert.IsTrue(target.Password != null);
Assert.IsTrue(target.Password.Length != 0);
Assert.IsTrue(target.Password.Length >= 4);
Assert.IsTrue(target.Password.Length <= 8);
Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[A-Z]"));
Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[a-z]"));
Assert.IsTrue(RegularExpressions.Regex.IsMatch(target.Password, @"[0-9]"));
Assert.AreEqual(aPassword, target.Password);
}
We could do this without Pex of course. We have have to come up with several input values to trigger an exception (at least that is what we intended to program and now are verifying) and some test values for some "positive" tests. The equivalence class technique can surely help us out.
Valid equivalence classes
- bigger or equal than 4 characters
- smaller or equal than 8 characters
- include at least one upper case letter
- include at least one lower case letter
- include at least one numeric digit
Invalid equivalence classes
- not null, non-existing
- smaller than 4 characters
- bigger than 8 characters
- no upper case letter
- no lower case letter
- no numeric digit
You normally start with searching a physical test case (password value) that covers all the valid equivalence classes and for each invalid equivalence class you should also make a test case (i.e. unit test as well). Try not to overlap any invalid equivalance classes so you can be sure that the forced invalidity throwed the exception.
- asDF1234 (covers all valid classes)
- null (non existing)
- Az1 (smaller than 4 chars)
- ASDf12345 (etc)
- abcd1
- ABCD1
- aBcDefGh
So 7 test cases should be enough ?
What would Pex do? It should be as good as this (if not better , at least the tutorial almost implies so).
If you would let the PexMethod work on the password setter method, you would get following results.
Run | aPassword | Summary/Exception | Error Messag |
---|---|---|---|
1 | null | ArgumentNullException | Value cannot be null. Parameter name: Please specify a valid password. (non-null) |
2 | """" | ArgumentException | Please specify a Strong Password. |
3 | \0 | ArgumentException | Please specify a Strong Password. |
4 | \u8000 | ArgumentException | Please specify a Strong Password. |
6 | \0\0 | ArgumentException | Please specify a Strong Password. |
7 | 8 | ArgumentException | Please specify a Strong Password. |
8 | \0\n | ArgumentException | Please specify a Strong Password. |
10 | \0\0\0 | ArgumentException | Please specify a Strong Password. |
11 | \n | ArgumentException | Please specify a Strong Password. |
12 | \n\0 | ArgumentException | Please specify a Strong Password. |
16 | \0\u8000\u8000 | ArgumentException | Please specify a Strong Password. |
21 | \n\n | ArgumentException | Please specify a Strong Password. |
25 | \u00062\u00f1\u0016\n\n | ArgumentException | Please specify a Strong Password. |
31 | \09\u00ef\u0088f\r | ArgumentException | Please specify a Strong Password. |
75 | \u8000\0\0\0\n\0 | ArgumentException | Please specify a Strong Password. |
83 | 0\u0088o]O< | ||
112 | \u00062Oo\n\n | ArgumentException | Please specify a Strong Password. |
113 | \u00062Oo\n | ||
114 | \u00062Oo\n\n\n\n\n\n\n\n\n\n | ArgumentException | Please specify a Strong Password. |
Pex established 19 test cases. A little bit more if you would go for the manual approach. But are the test input values better? In terms of "readability" no. But Pex generated some input values that I wouldn't have thought of in the first place.
The empty string was added in the Pex generated approach. In our manual exercise with didn't explicitly took that test case because this was implied in the "smaller than 4 chars" equivalence class. But again testing a little bit too much is maybe better (or saver) than testing too little.
Also the Pex generated unit tests include non alfanumeric signs like * or /. Did I intended to do that with my expression? If not I have to re-write it. With my manual technique I "forget" them. So Pex went a little deeper than my test analysis.
What about Escape characters such as "\n" (new line). pex included them and some tests passed. Was this the way I intended it? Were these the specs from the customer?
The examples I used to get acquinted with Pex may seem academic but I believ show potential for a tool like Pex. How to use it in your day-to-day applications will take some time and study. And like the Pex FAQ already states : it is not Pex versus unit testing (or other forms of testing or QA assurance measures like code analysis for example). I hope the Pex team will (continue to) find sponsers within Microsoft to integrate it in the next releases of Visual studio. At least the pex team already took a patent!.
Thanks for reading.
Best regards,
Alexander
Monday, 28 July 2008
Pex - test case 4
I've got one more test scenario for Pex. From comments on my previous post
Case study 3 I understood that the way I specified the PUT was not done correctly (though Pex generated the unit test methods I expected).
A PUT should specify what the intended behaviour of the code under test should do. It should not do this like in a tradiotional unit test for a particular combination of test input but in a more general way.
Sounds reasonable but I find some difficulity in writing them. I'll try to come with a "better" PUT this time.
Our code under test is as follows (Sorry in advance for some bad formatting HTML tables later in the post.):
private string mCode;
public string Code
{
get { return mCode; }
set {
if (value == null )
{
throw new System.ArgumentNullException("Please specify a valid code. (non-null)");
}
else if (value.Length < 4)
{
throw new System.ArgumentException("Please specify a valid code. (bigger than 3 characters.");
}
else if (value.Length > 10)
{
throw new System.ArgumentException("Please specify a valid code. (max 10 characters. including XYZ prefix) ");
}
else if (value.StartsWith("XYZ") == false)
{
throw new System.ArgumentException("Please specify a valid code (starting with XYZ - uppercase !) ");
}
else
{
this.mCode = value;
}
}
With the comments for my PUT in mind , I created the following PUT:
[PexMethod()]
public void CodeTest(string aCode)
{
measurement target = new measurement();
string actual;
target.Code = aCode;
actual = target.Code;
PexValue.AddForValidation("actual", actual);
if (( aCode != null) &&
(aCode.Length > 3) &&
(aCode.Length <= 10) &&
(aCode.StartsWith("XYZ")) )
PexAssert.IsTrue(actual == aCode);
}
Running Pex on this PUT will result in several generated test cases.
(Pex reporting option : Enable via Options/Pex/Reports)
Parameter Values | |||
---|---|---|---|
Run | aCode | actual | Summary/Exception |
1 | null | ArgumentNullException: Value cannot be null.(testingPEX) | |
2 | "" | ArgumentException: Please specify a valid code. (bigger than 3 characters. (testingPEX) | |
3 | \0\0\0\0\0\0 | ArgumentException: Please specify a valid code (starting with XYZ - uppercase !) (testingPEX) | |
4 | new string('\0', 14) | ArgumentException: Please specify a valid code. (max 10 characters. including XYZ prefix) (testingPEX) | |
5 | XYZZZZZ | XYZZZZZ |
Of course we first need to specify the expected exceptions (Pex will suggest these for you !) :
- Add attribute [PexAllowedExceptionFromAssembly(typeof(ArgumentNullException), "testingPEX")]
- Add attribute [PexAllowedExceptionFromAssembly(typeof(ArgumentException), "testingPEX")]
The generated unit tests look like this :
public partial class measurementTest
{
[TestMethod]
[PexGeneratedBy(typeof(measurementTest))]
public void CodeTestString_20080728_141020_005()
{
PexValue.Generated.Clear();
this.CodeTest("XYZZZZZ");
PexValue.Generated.Validate("actual", "XYZZZZZ");
}
[TestMethod]
[ExpectedException(typeof(ArgumentNullException))]
[PexGeneratedBy(typeof(measurementTest))]
public void CodeTestString_20080728_143541_000()
{
this.CodeTest((string)null);
}
[TestMethod]
[ExpectedException(typeof(ArgumentException))]
[PexGeneratedBy(typeof(measurementTest))]
public void CodeTestString_20080728_143620_001()
{
this.CodeTest("");
}
[TestMethod]
[ExpectedException(typeof(ArgumentException))]
[PexGeneratedBy(typeof(measurementTest))]
public void CodeTestString_20080728_143620_002()
{
this.CodeTest("\0\0\0\0\0\0");
}
[TestMethod]
[ExpectedException(typeof(ArgumentException))]
[PexGeneratedBy(typeof(measurementTest))]
public void CodeTestString_20080728_143620_003()
{
this.CodeTest(new string('\0', 14));
}
}
If you would verify the code coverage results in Visual Studio or via the Pex reporting option, you would see 100% Block coverage.
Pex generated for us the strings to serve as input values. I don't know about you but I think this is pretty clever.
Still I'm a little confused about the PUT I created. If I would delete all the pexAssert statements the exercise would still be the same for the generated unit tests.
[PexMethod()]
public void CodeTest(string aCode)
{
measurement target = new measurement();
string actual;
target.Code = aCode;
actual = target.Code;
PexValue.AddForValidation("actual", actual);
}
Thanks for reading.
Best regards,
Alexander