Wednesday 20 August 2008

Pex - Test Case 7

Hello,

In this Pex test case I will investigate unit test on classes the have dependencies.( feel free to read the previous posts regarding this subject).
I'll first start with how you would unit test this scenario without the help of Pex and afterwards I'll investigate how Pex might help.

I have requirement to calculate the bonus a salesperson earns at the end of the year. The bonus depends on the sales he/she made this year and the year before.

The rules are

  • The bonus amounts to 1% of the sales for that person

  • If this year's sales is higher than 30000 € AND ALSO this year's sales are higher than the Sales of the previuos year for that salesperson, an extra bonus is added of 250 €



A quick test analysis should result in following test cases

















currentYearSales greater or equal than 30000 € YesYesNoNo
currentYearSales greater than LastYearSales YesNoYesNo
Base bonus (1% currentYearSales )XXXX
Extra bonus (250€)X---


So three test cases would be sufficient for complete test coverage (last two test cases can be collapsed).

Suppose you implements the rule in a bonus class and to know the sales amounts you have a Sales class that has appropriate methods to give you them. You've also encapsulated the retrival of these sales amounts inside the bonus class method that calculates the bonus.


public class Bonus
{
public double CalculateForEmployee(uint EmployeeId)
{
double baseBonusPercentage = 1;
double extraBonusAmount = 250;
uint minSalesForBonus = 30000;

Sales sales = new Sales();
uint currentYearSales = sales.CurrentYearSalesForEmployee(EmployeeId);
uint lastYearSales = sales.LastYearSalesForEmployee(EmployeeId);

double bonus = currentYearSales * (baseBonusPercentage / 100);

if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
bonus += extraBonusAmount;

return System.Math.Floor(bonus);
}
}



public class Sales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}

public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
}


Suppose you want to unit-test the method CalculateForEmployee in the Bonus class. The method calls into a Sales object to retrieve the sales for this year and the year before. Our code-under-test depends on another object to do its job. Nothing prevents you to write a standard unit test calling CalculateForEmployee.


[TestMethod()]
public void CalculateForEmployeeTest()
{
Bonus target = new Bonus();
uint EmployeeId = 0; // ?
double expected = 0; // ?
double actual;
actual = target.CalculateForEmployee(EmployeeId);
Assert.AreEqual(expected, actual);

}


But how do I know which EmployeeId to use and how do I know what the expected value for the bonus should be? In fact the method-under-test has dependencies we don't control in the unit test. The bonus class doesn't control the value of the sales. it's the responsibility of the sales class. As a matter a fact this situation isn't a pure unit test anymore. It's a unit integration test. So how can we control this dependency on the sales object and know in advance the EmployeeId and Sales amounts necessary for our method-under-test. Suppose a database is used to store sales information. We could setup a test database with all the necassary records for retrieving the sales for a particular sales employee. But this would still not classify our test as a unit test (only testing the rules for caculating the bonus). Also you must account for setting up three different test situation in the database.

Rewritting our classes a little bit could helps us. Interfaces, dependency injection and test doubles are techniques I will us to make the CalculateForEmployeeTest easier to test : knowing it advance what the sales amounts will be in order to assert the bonus.

First we extract an interface from our Sales class

public interface ISales
{
uint LastYearSalesForEmployee(uint EmployeeID);
uint CurrentYearSalesForEmployee(uint EmployeeID);
}




public class Sales : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
//Could be a database call
}
}


In this example I'll use "manual" dependency injection (constructor injection) to demostrate the dependency injection principle.


public class Bonus
{
ISales mSalesdao;


public Bonus(ISales aSalesDao)
{
mSalesdao = aSalesDao;
}

public double CalculateForEmployee(uint EmployeeID)
{
double baseBonusPercentage = 1;
double extraBonusAmount = 250;
uint minSalesForBonus = 30000;
double bonus;

uint currentYearSales = mSalesdao.CurrentYearSalesForEmployee(EmployeeID);
uint lastYearSales = mSalesdao.LastYearSalesForEmployee(EmployeeID);

bonus = currentYearSales * (baseBonusPercentage / 100);
if (currentYearSales >= minSalesForBonus && currentYearSales > lastYearSales)
bonus += extraBonusAmount;

return System.Math.Floor(bonus);
}
}


So how does this help me in my unit test you might wonder? A test-double is a test-specific class that we will use in our unit tests so the code-under-test will not notice it that much. There are several variations of test-doubles. For example a test stub is an equivalent of the dependend class but gives "fixed answers" if we ask for something or does something in a pre-determined way.

So instead of passing a real instance of the type Sales to the constructor of Bonus, I slip in a test stub when I write the unit tests. Because it gives fixed answers , I'll need a test stub for each test case.


public class SalesStub_CYSsmaller30000 : ISales
{

public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 29999;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 29998;
}
}

public class SalesStub_CYSgreater30000_CYSgreaterLYS : ISales
{
public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 30001;
}
public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 30002;
}

}

public class SalesStub_CYSgreater30000_CYSsmallerLYS : ISales
{

public uint LastYearSalesForEmployee(uint EmployeeID)
{
return 30003;
}

public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
return 30002;
}

}

The unit tests use these stubs to do their thing. I create a teststub that is appropriate for the test case and inject it into the constructor of bonus.


[TestMethod()]
public void CalculateForEmployee_Lys29999_cs29998_withStub()
{
ISales aSalesDao = new SalesStub_CYSsmaller30000();
Bonus target = new Bonus(aSalesDao);
uint EmployeeID = 0; // Dummy EmployeeID not important for this example
double expected = 299; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
}


[TestMethod()]
public void CalculateForEmployee_Lys30001_cs30002_WithStub()
{
ISales aSalesDao = new SalesStub_CYSgreater30000_CYSgreaterLYS();
Bonus target = new Bonus(aSalesDao);
uint EmployeeID = 0; // Dummy EmployeeID not important for this example
double expected = 550; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);
}

//and so


Another implementation for test doubles is the use of a Mock framework. This utility dynamically create test-equivalent objects that are setup (expectations) through the framework facilities . I'll illustrate this with the help of Rhino.Mock.


[TestMethod()]
public void CalculateForEmployee_Lys29999_cs29998_withMock()
{
uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose

Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
ISales salesDao = mr.CreateMock();
Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(29999);
Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(29998);
mr.ReplayAll();

Bonus target = new Bonus(salesDao);


double expected = 299;
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);

mr.VerifyAll();


}


[TestMethod()]
public void CalculateForEmployee_Lys30001_cs30002_WithMock()
{
uint EmployeeID = 0; // Dummy EmployeeID not important for this test purpose

Rhino.Mocks.MockRepository mr = new Rhino.Mocks.MockRepository();
ISales salesDao = mr.CreateMock();
Rhino.Mocks.Expect.Call(salesDao.LastYearSalesForEmployee(EmployeeID)).Return(30001);
Rhino.Mocks.Expect.Call(salesDao.CurrentYearSalesForEmployee(EmployeeID)).Return(30002);
mr.ReplayAll();

Bonus target = new Bonus(salesDao);


double expected = 550; //
double actual;
actual = target.CalculateForEmployee(EmployeeID);
Assert.AreEqual(expected, actual);

mr.VerifyAll();
}
//and so on


The use of a Mock framework doesn't force me to explicitly write numerous of test stubs.

So where does Pex come in? For Pex to do its job I'll need a parametrized unit test. This is a unit test that will take arguments. Consider it as a test refactoring thing. Things that are common in the unit test are extracted and put into separate method/function. Pex needs this setup because it will be in charge with generating "traditional" unit tests with test data values that it will determine during its exploration phase. These generated unit test will call the parametrized unit test , passing the necessary arguments.

In our "mock" example we explicitly setup the mock for each unit test. In analogy with the parametrized unit test we need a parametrized Mock object (see Pex Tutorial).


[TestClass]
[PexClass]
public partial class BonusTest
{
[PexMethod]
[PexUseType(typeof(SalesPexMock))]
public void Calculate(ISales aSalesPexMock)
{
PexAssume.IsNotNull(aSalesPexMock);

uint EmployeeID = 9999; //dummy , not important for our test
Bonus target = new Bonus(aSalesPexMock);
double result = target.CalculateForEmployee(EmployeeID);
PexValue.AddForValidation("result", result);
}

}

[PexMock]
public class SalesPexMock : ISales
{
public uint LastYearSalesForEmployee(uint aEmployeeID)
{
var call = PexOracle.Call(this);
uint lys = (uint)call.ChooseFrom("lastYearSales", new uint[] { 29999, 30000, 30001 });
PexValue.Add("lastYearSales", lys);
return lys;
}

public uint CurrentYearSalesForEmployee(uint EmployeeID)
{
var call = PexOracle.Call(this);
uint lys = (uint)call.ChooseFrom("CurrentYearSales", new uint[] {29999, 30000, 30001});
PexValue.Add("CurrentYearSales", lys);
return lys;
}
}


Running the PexExplorartion on the PexMethod will result in three unit test being generated.


[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate01()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)30000u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)30000u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "300");
}

[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate02()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)29999u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)30000u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "299");
}

[TestMethod]
[PexGeneratedBy(typeof(BonusTest))]
public void Calculate03()
{
PexValue.Generated.Clear();
IPexOracleRecorder oracle = PexOracle.NewTest();
((IPexOracleSessionBuilder)
(oracle.OnCall(0, "SalesPexMock.CurrentYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "CurrentYearSales", (object)30001u);
((IPexOracleSessionBuilder)
(oracle.OnCall(1, "SalesPexMock.LastYearSalesForEmployee(UInt32)")))
.ChooseAt(0, "lastYearSales", (object)29999u);
SalesPexMock salesPexMock = new SalesPexMock();
this.Calculate((ISales)salesPexMock);
global::Microsoft.Pex.Framework.PexValue.Generated.Validate("result", "550");
}


Of course meanwhile you verified the test coverage results :-) in the Pex report or via the Visual Studio Test Coverage facility.

So with the PexMock framework expectations for the indivudual unit test are setup , just like we did in our Rhino.Mock example. Of we had to help the Pex mock framework a little bit just like we did with the parametrized unit test.


Thanks for reading. Of course your comments are most welcome.

Best regards,

Alexander


ps: We already had complaints of Salespersons that find this bonus rule not fair. for example sales-persons that are above the minimum sales amount eligable for extra bonus but are not progressing , don't get it. Like wise someone who is progressing doesn't receive extra incentives.... :-)

Monday 11 August 2008

Pex - test case 6 (revisited...again)

Hello,

I did some further test on test case 6 . I removed all pex assumptions regarding the dates. This time Pex had no problem finding the test cases necessary for a complete block coverage.

So the PUT states no assumptions regarding the dates;


[TestClass]
[PexClass(typeof(EmployeeServices))]
public partial class EmployeeServicesTest
{
[PexMethod()]
public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
{
PexAssume.IsNotNull(aEmployee);
PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);


PensionContribution result = target.DeterminePensionContribution(aEmployee);

PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

if (aEmployee.Salary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aEmployee.Salary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}

}


When you run Pex this time it generated none less than 11 unit tests. The generated DateTime value are constructed with the Ticks argument, so it no easy to see the actual dateTime. Actually the first time I run this Pex generated a tick value with a value larger than 32 bit. When running the generated unit tests, an exception was thrown specifying thant the tick value should be between the DateTime.Minvalue.Ticks and DateTimeValue.Maxvalue.Ticks. It seemed to me that i generated a 64-bit value.
I haven't been able to reproduce this error though. May be it is one of the remarks in the release of this version 0.6x. I'm running on Windows XP SP2!

Nevertheless here are the generated unit tests.

public partial class EmployeeServicesTest
{
[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution01()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution02()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution03()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1L);
employee = EmployeeFactory.Create
(2, "", "", default(DateTime), s0, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution04()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution05()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution06()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution07()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(1262076461318144L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution08()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(56623104001835008L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution09()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(634715138118123520L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution10()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(647341024241516544L);
DateTime s1 = new DateTime(1L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.PartTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "0");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContribution11()
{
PexValue.Generated.Clear();
Employee employee;
DateTime s0 = new DateTime(633532870975488000L);
DateTime s1 = new DateTime(126226978751643648L);
employee = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices employeeServices = new EmployeeServices();
this.DeterminePensionContribution(employeeServices, employee);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}

}


When you investigate the coverage results for Visual Studio and the output from Pex, you will see that both confirm the 100% coverage result.

Part of the Pex report , coverage section (click on picture for better view):



What about my "business" assumptions? Maybe it would be easier to incorporate them as pre-conditions in the code under test. For example Start date of the contract should be greater than 25/May/1968 in the setter method for Startdatecontract of our Employee class. Non-compliance would than result in an exception being thrown. how will pex cope with them?


Thanks for reading. As always, feel free to comment.

Best regards,

Alexander

Pex - test case 6 (revisited)

Hello,

In the previous case I tried to come up with a scenario where a method takes a "complex" type instead of primitive types like integers of booleans. The exercise didn't go as expected. By the way , the latest downloadable version of Pex is 0.6.30728.0

To see whether pex still works on a method with primitive types, I re-wrote the method as follows.


public PensionContribution DeterminePensionContribution2(
uint aSalary,
uint aAge,
uint aServiceYears,
Contract aTypeContract)
{
//no pre-codition checks ....

PensionContribution contribution = new PensionContribution();

if (aSalary <= 3000)
{
if (aAge < 45)
{
if (aTypeContract == Contract.FullTime)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 0;
}
}
else
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 25;
}
}
}
else
{
if (aAge < 45)
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
if (aTypeContract == Contract.FullTime)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 25;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
else
{
if (aServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;

}
}

}
return contribution;
}



The PUT method looks like this.



[PexMethod]
public void PUTDeterminePensionContribution(
[PexAssumeUnderTest]EmployeeServices target,
uint aSalary,
uint aAge,
uint aServiceYears,
Contract aTypeContract
)
{
PexAssume.IsTrue(aAge > 18 && aAge < 75);
PexAssume.IsTrue(aServiceYears > 0 && aServiceYears < 50);
PexAssume.IsTrue(aSalary > 1400 && aSalary < 10000);
PexAssume.IsTrue(aTypeContract == Contract.FullTime || aTypeContract == Contract.PartTime);

PensionContribution result = target.DeterminePensionContribution
(aSalary, aAge, aServiceYears, aTypeContract);
PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

if (aSalary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aSalary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aSalary <= 3000 && aAge >= 45 && aServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aSalary <= 3000 && aAge < 45 && aTypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aSalary <= 3000 && aAge >= 45 && aServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aSalary > 3000 && aAge < 45 && aServiceYears >= 5 && aTypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aSalary > 3000 && aAge >= 45 && aServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aSalary > 3000 && aServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}


In no time Pex finds the test cases for full block coverage











targetaSalaryaAgeaServiceYearsaTypeContractBaseContributionSupplementaryContributionSummary/ExceptionError Message
1...8192642FullTime13530
2...81926430FullTime13520
3...8192312FullTime13530
4...81923130FullTime13525
5...1532642FullTime8015
6...15326430FullTime8025
7...1532312FullTime8015
8...8192642PartTime13530
9...1532312PartTime800
10...81923130PartTime13520



So there is nothing "wrong" with the method under test. The problem when you investigate the output window, lies maybe in the calculation of the "age" and "serviceYears" properties.


00:00:00.0> starting execution
00:00:00.0> reflecting tests
00:00:00.3> TestProject1
00:00:00.3> EmployeeTest
00:00:00.3> DeterminePensionContributionIntern(Employee)
maxconstraintsolvertime - 1s (constraint solver time out after 1 seconds)
[execution] 4 runs
[execution] 15 runs
[test] (run 20) DeterminePensionContribution
[test] (run 21) DeterminePensionContribution
[execution] 21 runs, 49/81 blocks covered
[test] (run 25) DeterminePensionContribution
[execution] 29 runs, 49/81 blocks covered
[execution] 41 runs, 49/81 blocks covered
[modelsearch] could not flip 2 branches in a row
[execution] 50 runs, 49/81 blocks covered
[execution] imprecision at testingPEX.Employee.differenceInYears, offset 0x26
[modelsearch] could not flip 4 branches in a row
[execution] 77 runs (97,40% unique paths), 49/81 blocks covered
timeout - 120s
Timeout, 1 times
MaxConstraintSolverTime, 60 times
[coverage] 49/81 block (60,49%)

00:02:01.4> [finished] 3 generated tests (0 failures), 00:02:01.3757768.
-- 0 critical errors, 0 errors, 0 warnings.

[reports] skipping report generation

EXPLORATION SUCCESS


monitored process exited with Success (0)
finished



Pex needs to find dateTimes in order to test the conditions specified in the method. I used a helper method to calculate the years to date for the age and serviceyears:

private uint differenceInYears(System.DateTime aDate1, System.DateTime aDate2)
{
//missing checks ....
//approximative
uint years = (uint)(aDate1.Year - aDate2.Year);
if (aDate1.Month == aDate2.Month)
{
if (aDate2.Day < aDate1.Day) years = years - 1;
}
return years;
}



Could it be the unsigned integer type declarations? Or are the assumptions wrongly formulated? Or both? What does "could not flip 2 branches in a row" mean?

I'll try to do some more test around this. Feel free to comment!

Thanks for reading.

Best regards,

Alexander

Wednesday 6 August 2008

Pex - Test Case 6

Hello,

In the previous Pex posts I used examples that used primitive types and/or strings as parameters for methods under test. In this post I will "test" Pex how it wil cope with a method that accepts a "complex type". This test scenario is similar to the case study in Pex - Test case 3 except we don't work directly with primitive types a method parameters or return types.


Suppose the HR department asked you to include a new module in their Salary-application that will determine the contribution each employee has to make in a private pension fund. The salary administration uses the following desicion table.



While analysing the table you conclude you can simplify the table to help you code this rule into c#.



You already have some classes in the application :


public class Employee
{
//missing elementary checks in order to keep code concise....
public int EmployeeID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
private DateTime _BirthDate;
public DateTime BirthDate
{
get { return this._BirthDate; }
set {
this._BirthDate = value;
this.Age = differenceInYears(_BirthDate, DateTime.Now);
}
}
private DateTime _StartDateContract;
public DateTime StartDateContract {
get { return _StartDateContract; }
set {
_StartDateContract=value;
this.ServiceYears = differenceInYears(_StartDateContract, DateTime.Now);
} }
public uint Salary { get; set; }
public Contract TypeContract { get; set; }
public uint Age { get; set; }
public uint ServiceYears { get; set; }
private uint differenceInYears(DateTime aDate1, DateTime aDate2)
{
//missing checks ....
//approximative
uint years = (uint)(aDate1.Year - aDate2.Year);
if (aDate1.Month == aDate2.Month)
{
if (aDate2.Day < aDate1.Day) years = years - 1;
}
return years;
}
}

public enum Contract
{
FullTime=0,
PartTime=1
}

public struct PensionContribution
{
public uint BaseContribution{get;set;}
public uint SupplementaryContribution {get;set;}
}


The rule has also been translated into c#. The method DeterminePensionContribution has a parameter of the type Employee as described in the above code snippet. The return type is the structure PensionContribution.


public class EmployeeServices
{
public PensionContribution DeterminePensionContribution(Employee aEmployee)
{
//no pre-condition checks .....

PensionContribution contribution = new PensionContribution ();

if (aEmployee.Salary <= 3000)
{
if (aEmployee.Age < 45)
{
if (aEmployee.TypeContract == Contract.FullTime)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 0;
}
}
else
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 15;
}
else
{
contribution.BaseContribution = 80;
contribution.SupplementaryContribution = 25;
}
}
}
else
{
if (aEmployee.Age < 45)
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
if (aEmployee.TypeContract == Contract.FullTime)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 25;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;
}
}
}
else
{
if (aEmployee.ServiceYears < 5)
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 30;
}
else
{
contribution.BaseContribution = 135;
contribution.SupplementaryContribution = 20;

}
}

}
return contribution;
}






}


In a "test-after-you-code" scenario, you could use the desicion table to specify the values for actual for the different properties of a employee object that is being passed to the method under test. So 9 unit tests would be sufficient to touch all corners of the code.

A data-driven in test in Visual Studio is somewhat difficult because we are not dealing with simple datatype parameters. We could serialize the different instances and save it in string form into an external data source.

Can Pex help generate the necessary unit test to fully test our code?

Before heading to the PUT (Parameterized Unit Test) I would like to refer you to the Pex tutorial for more background information on setting up PexMethods.

A new element in this case study is the use of a factory method to create our Employee object that will be used in the generated unit test. Pex will call this method with appropriate values that it has discovered during its exploration of the code in the generated unit tests. The cool part is that Pex will first complain about not being able to create a Employee object but then suggest a soulution for it : the Factoty method and generates it for us.


[PexFactoryClass]
public partial class EmployeeFactory
{
[PexFactoryMethod(typeof(Employee))]
public static Employee Create(
int i0,
string s0,
string s1,
DateTime dt0,
DateTime dt1,
uint ui0,
Contract c0
)
{

Employee e0 = new Employee();
e0.EmployeeID = i0;
e0.FirstName = s0;
e0.LastName = s1;
e0.BirthDate = dt0;
e0.StartDateContract = dt1;
e0.Salary = ui0;
e0.TypeContract = c0;
return e0;
}

}

In the generated unit test , this factory method is used as follows:

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_095656_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626406282140467200L);
DateTime s1 = new DateTime(631149005309067264L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}


Let's return to the PUT. The following PUT was based on a PexMethod that you can let the Visual Studio Pex integration build for you.
During its explorartion Pex will try out different value for the properties of our Employee object. The assumptions should help Pex create reasonable test cases. The code in this example did not enforce them in order to keep the code somewhat limited in size.

Because this method will be used for all generated unit test, the asserts have to be more general than you would normally do for a single unit test.


[TestClass]
[PexClass(typeof(EmployeeServices))]
public partial class EmployeeServicesTest
{
[PexMethod(MaxConstraintSolverTime = int.MaxValue, Timeout = int.MaxValue)]
public void DeterminePensionContribution([PexAssumeUnderTest]EmployeeServices target, Employee aEmployee)
{
PexAssume.IsNotNull(aEmployee);
PexAssume.IsTrue(aEmployee.BirthDate < new DateTime(1992, 1, 1) && aEmployee.BirthDate > new DateTime(1955, 1, 1));
PexAssume.IsTrue(aEmployee.StartDateContract < DateTime.Now && aEmployee.StartDateContract > new DateTime(1968, 1, 1));
PexAssume.IsTrue(aEmployee.BirthDate < aEmployee.StartDateContract);
PexAssume.IsTrue(aEmployee.Age > 18);
PexAssume.IsTrue(aEmployee.Salary > 1400 && aEmployee.Salary < 10000);
PexAssume.IsTrue(aEmployee.TypeContract == Contract.FullTime || aEmployee.TypeContract == Contract.PartTime);


PensionContribution result = target.DeterminePensionContribution(aEmployee);

PexValue.AddForValidation("BaseContribution", result.BaseContribution);
PexValue.AddForValidation("SupplementaryContribution", result.SupplementaryContribution);

if (aEmployee.Salary <= 3000)
Assert.IsTrue(result.BaseContribution == 80);
if (aEmployee.Salary > 3000)
Assert.IsTrue(result.BaseContribution == 135);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age < 45 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 15);
if (aEmployee.Salary <= 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.FullTime)
Assert.IsTrue(result.SupplementaryContribution == 25);
if (aEmployee.Salary > 3000 && aEmployee.Age < 45 && aEmployee.ServiceYears >= 5 && aEmployee.TypeContract == Contract.PartTime)
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.Age >= 45 && aEmployee.ServiceYears >= 5 )
Assert.IsTrue(result.SupplementaryContribution == 20);
if (aEmployee.Salary > 3000 && aEmployee.ServiceYears < 5)
Assert.IsTrue(result.SupplementaryContribution == 30);
}

}


Running the Pex exploration several times (and after quite a while) I got results but no full coverage. Maybe the assumptions and/or asserts are not "pex-friendly"?



The generated unit tests only cover half of the code blocks.

public partial class EmployeeServicesTest
{

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_000()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110101_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110121_002()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(619794039740301312L);
DateTime s1 = new DateTime(631148677322244096L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110200_003()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626094836655304192L);
DateTime s1 = new DateTime(633479291125776385L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_110201_004()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(626094836655304192L);
DateTime s1 = new DateTime(633479291125776385L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_000()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112335_001()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "25");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_112422_002()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(609681269903603218L);
DateTime s1 = new DateTime(621044308151107616L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.PartTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "20");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115647_003()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(627392712227689792L);
DateTime s1 = new DateTime(633479779357556736L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 8192u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "135");
PexValue.Generated.Validate("SupplementaryContribution", "30");
}

[TestMethod]
[PexGeneratedBy(typeof(EmployeeServicesTest))]
public void DeterminePensionContributionEmployeeServicesEmployee_20080806_115718_004()
{
PexValue.Generated.Clear();
Employee e0;
DateTime s0 = new DateTime(627392712227689792L);
DateTime s1 = new DateTime(633479779357556736L);
e0 = EmployeeFactory.Create(2, "", "", s0, s1, 1983u, Contract.FullTime);
EmployeeServices es0 = new EmployeeServices();
this.DeterminePensionContribution(es0, e0);
PexValue.Generated.Validate("BaseContribution", "80");
PexValue.Generated.Validate("SupplementaryContribution", "15");
}

}





If you have any suggestions to improve (re-factor) the test code or the code-under- test (test-friendliness), don't hesitate to drop a line.

Thanks for reading.

Best regards,

Alexander