GAT : StarterKit Pattern

Hello,

Making guidance packages GAT (Guidance Automation toolkit) is also making knowledge available for novice users regarding certain processes or technology. This entry I will discuss a typical scenario of how you can make an example application available through GAT in Visual Studio 2008.

Driving forces
You want to make a working example of how a typical Winforms application is made in your organization available for your developers. The developers must be able to run it , learn from it and make changes to it if they want to.

Solution
Use the GAT to make a Guidance package that unfolds the “Starterkit” application. Install the Guidance Package on the developers machine to make the Guidance Package available in Visual Studio 2008. The developer can choose the Starterkit from the Visual Studio 2008 new project–dialog to create the example application.

Structure

Attention

• GAT template for making Guidance Packages should be stripped to bare minimum before proceeding with own package creation
• References to assemblies, must exist on target machine (GAC or same folder : Hintpath !!!)
  

Alternatives

• Installed example application a demo box
• Example application in SourceSafe
• Setup package (msi) of the application with all the sources
• “Pure” VS2005 templates solution

Best regards,

Alexander

First steps with the Guidance Automation Toolkit (GAT)

Hello,

I recently tried out the Guidance Automation Toolkit and its "run-time" companion Guidance Automation Extensions (GAX/GAT - July 2007 CTP) on VS Rosario November CTP image.

The GAT enables you to extent the Visual Studio 2005/2008 experience. Extensibility comes in different forms and shapes but the GAT is geared helping the developer when creating a VS solution. It gives "guidance" in the form VS project or item templates combined with automation hooks (recipes) that can collect information from the developer (wizards) while the solution structure is being created (un-folded).

GAT find it's origin in the Microsoft Software Factory Initiative. The Software Factories provide you with a set of proven practices for building distributed applications. These practices are exposed through architectural overviews, patterns, how-to topics, reference implementations, automated guidance packages, and application blocks. There are several SW factories you can download today ( Web Service (WSF), Smart Client(SCSF), web client (WCSF))

The idea behind GAT is not completely new. A previuos technology in VS2003, the Enterprise Templates was also an effort for rapidly defining the initial structure of applications and to minimize the need to read through a bunch of of white papers, standards, and policies documents.

In VS2005 we've got the new template mechanism (Visual Studio Templates) . The Export template makes is fairly easy to make shareable templates. Actually GAT uses these exported template. What makes GAT different is the way you can hook automation to these templates.

Wojtek Kozaczynski (http://blogs.msdn.com/wojtek/archive/2005/05/02/414129.aspx) has a nice blog-entry describing the origins of GAT.

Yet another technology, called Domain Specific Languages (DSL), is available to extent VS. The DSL Toolkist (in the SDK) allows you to build visual interaction component that can be used from with in VS just like you would for example use a Winform Form designer. You drag & drop the visual elements on the form and code is generated for you. An example in the SDk is a DSL that enables you to visually build a Wizard-like application. So you get a special toolbar and drag & drop the visual elements on your "canvas" and "model" your wizard application. The model then is used by a code generator to produce executable code.

So GAT and DSL Tools both build on top of existing VS extensibility featueres but with more possibility in guidance experience or modelling experience.

Daniel Cazzulino has some thought about the status of developer guidance in realm of GAT, DSL and Software Factories (http://www.clariusconsulting.net/blogs/kzu/archive/2007/01/08/BuildingSoftwareFactoriesToday.aspx)

So I see certainly advantages

• Productivity can be increased : It includes automation for Visual Studio: with this automation, de-velopers can easily apply guidance in consistent and repeatable ways.
• Adaptable : It is open and customizable.development leads can customize the factory to meet specific needs.
• Accelerated start. It provides an effective way for architects and developers to create a high-quality starting point for their application. This means that projects begin with a greater level of maturity than applications developed from scratch.
• Enforce standards : to enforce standards independent of the solution domain

But I have also some concerns

• Support : I don’t think so. Is not an official product of MS. It’s from the Patterens & practices group. It is still CTP ... since July 2005. Forum exists but I haven't seen any MS employee replying to questions...(http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=78&SiteID=1)
• Some (steep) learning curveExamples only on blogs (http://jelle.druyts.net/CategoryView.aspx?category=Blog%7CProgramming%7C.NET%7CGuidanceAutomation) or you have to dissect the WSF/SMSF/SCSF or you have to study the default Guidance package that is produced when you create a new guidance package project.
• Support for next versions of GAT/GAX once an application has been built with a version of GAT/GAX
• Mistakes are severely punished :-). You can easily scr*w up your registry (like I did) without a warning. Re-installing GAX/GAT can result in something like this

In that respect Victor Garcia Aprea from Clarius Consulting made a GaxTroubleshooter (http://weblogs.asp.net/vga/archive/2007/12/28/troubleshooting-gax-gat-installation-issues.aspx) but I wasn't able to repair my mistakes with it though. Clarius has also a Software Factory Toolkit (http://www.softwarefactoriestoolkit.net/) but I dare not to re-install it because it is rather invasive . It seems to intervene in the default GAX/GAT installation.

Does someone actualy used al these specific VS features to create a custom Guidance package ? What were your experiences? Like to hear from you.

Best regards,

Alexander

Unit testing ADO Entity Framework

Hello,

I was playing with the ADO.NET entity framework beta 3 that I installed on the Rosario November 2007 CTP and I had some problems making unit tests. I kept getting the follwing error.

System.Data.MetadataException: The specified metadata path is not valid. A valid
path must be either an existing directory, an existing file with extension
'.csdl', '.ssdl', or '.msl', or a URI that identifies an embedded resource

When you use the wizard to create a model several "hidden" files are generated for you; .csdl, .ssdl, and .msl files. The information contained in these files is encapsulated in the .edmx file but the files .csdl, .ssdl, and .msl files reside in the bin\Debug or bin\Release directories of the solution. Apperently these files are still necessary at run-time. There not embedded in the assembly.

In a connectionstring you specify these files in the metadata part :

connectionString="metadata=.\Northwind.csdl.\Northwind.ssdl.\Northwind.msl

When you're using for example the VS test framework, the test infrastructure copies the assemblies to a certain test folder (isolation). If you have an app.config or use custom files in your code, you must assure that the test framework put these file in that particular test folder.


Now the .csdl, .ssdl, and .msl are also not copied to that test folder automatically. I used the deployment feature of the VS test environment. I specified the three files and now there copied to the test folder.

Maybe there are other ways? Let me know!

Best regards,

Alexander

Unit Testing Checklist

Hello,

Over time I compiled a list of things regarding unit and unit integration tests; books, articles, forums and other blogs. I did not always kept track where I found some piece of information. So I certainly don't take any credit in this list (except compiling it). Recently I discovered the XUnit Test Patterns website by Gerad Meszaros (http://xunitpatterns.com/index.html). I wished I found that one earlier.

Still I wanted to share with you the following list.

• Do you know how to incorporate the unit testing into the development process?
  Unit tests are written by the developer who write the code and executed by the developer.
  Feedback period between coding and finding error is smaller
  Coder and tester is same person
  Accounted in the development budget
  Continuous testing versus phase-end testing
  Avoid arguments like: I don’t have the time to conduct unit test. I must deliver ASAP.
  Don’t rely on phase-end tests to find your errors.
  
• Do you have a strategy to incorporate unit test code in production code organization?
  Be prepared to see as much production code as test code so good organization primordial.
  Integration in source control system is needed
  
• Are you aware of the type of errors unit testing tries to find?
  Some errors are easier to find with unit (integration) test than in formal phase-end tests. Or at least the effort is bigger to find those errors.
  
• Do you use an automated tool such as NUnit or VS2005 Integrated testing Framework to make the unit-testing process easy and repeatable?
  Same Framework available for every developer. Same framework used by every developer.
  Automation in two ways: invoking the tests and checking the results.
  Can be integrated into automatic build/continuous integration scheme.
  
• Do you know which type (layer) of code benefits the most of unit testing?
  Some type code is more appropriate for unit testing than other for example business logic versus User interface.
  Whenever you are tempted to write something into a debug.writeLine type of thing, write a test for it
  
• Do you know how to deal with code that is to hard to unit test?
  Consider other ways of testing
  GUI skin code,Main methods ,Asynchronous method ,mult-threading
  
• Do you know how to deal with developers that don’t write unit code?
  Education
  Take away resistance by showing advantages
  
• Do you how to avoid that unit-testing becomes de-prioritized when deadlines come near?
  Planning
  
• Do you have the right mindset to conduct unit test?
  Showing that it works versus finding errors versus
  verifying expectations (specification) compared to the realisation (code)
  Are you prepared to scrutinize your work?
  
• Do you know when to write unit test?
  Write the tests as you go along. Don’t leave them for a “later” that may never come.
  Do you know when to execute unit test?
  Run unit test as often as possible to decrease feedback period between coding and finding error is smaller

• Do you know how to integrate unit testing into the build process /continuous integration scheme?
  Unattended execution of unit tests to let the test run as much as possible.
  
• Do you have a policy about how to deal with failed unit test?
  Make sure code passes unit tests and integration tests before you check it into the source control system.
  
• Do you make a distinction between pure unit test and unit integration test and when to use the them?
  Make test for collaborating units in function in stead of big bang approach in system testing. Unit test are good for regression testing
  
• Do you have a policy to distribute knowledge about conduction unit testing and unit (integration) testing and automated testing in general?
  Optimal use of resources
  
• Do you have a policy to use unit test for regression testing?
  Changes to an existing product may not result in breaking unit tests.
  
• Do you have a policy to keep the quality of unit tests high?
  Avoid that the first unit test are good and the remaining unit test are of lesser quality (Coverage) because of time pressure.
  
• Do you consider the unit test code on the same level as production code?
  Be prepared to see as much production code as test code.
  Test must be written and maintained to the same professional standards as your production code
  
• Did you code to Interfaces rather than Classes?
  To plug in different implementations at runtime or test time.
  Makes it easy to provide test stubs that don’t need a sophisticated implementation of collaborators’ functionality and hence are freed from needing to depend on supporting infrastructure.
  
• Do you use Inversion of Control/dependency injection pattern?
  Makes is easier to feed the class under test with alternative implementations of collaborating classes.
  
• Does each class have a well-defined set of responsibilities?
  So it will be more easy to come up with unit test.
  High cohesion , low coupling
  
• Did you expose too much of a class for the sake of testability?
  Don’t mark a method public just for the sake of testability. Keep encapsulation in mind.
  VS2005 test framework can deal with class and methods marked friend or private.
  
• Did you separate unit test code from the code under test?
  Unnecessary code to deploy.
  
• Is there one test class per class under test?
  Code generation of VS2005 test framework encourages it and uses it when you generate additional test method skeletons

• Is there a separate assembly per assembly under test?
  Code generation of VS2005 test framework encourages it and uses it when you generate additional test class skeletons
  
• Is there a separation between tests that need configuration files and the one that don’t need configuration files?
  You can more easily identify which test should be able to run on any machine without setup (eg. Build server)
  Pay attention on place of test data /configuration files because of the way xUnit framework works.
  
• Is there a separation pure unit test and unit integration test?
  With unit integration test you must always verify if the collaborating units didn’t cause an error when a test should fail.
  
• Did you separate database access code test?
  This kind of tests requires much more attention in preparation, configuration and execution speed.
  Is the unit testing code under source control
  Is part of the code base.
  Any developer must be able to use it afterwards.
  An automatic system (Build/CI) must be able to use it.
  
• Can all test methods run autonomous so they don’t rely on other test or require that they are run in a particular order?
  Test method can be selected/de-selected via the Test UI so test must run independent of each other.
  VS2005 test framework allows ordered test runs (more functional like testing, use case scripts)

• Is every test able to run over and over again, in any order, and produce the same results?
  Otherwise automation of the assertions is difficult.
  
• Did you use mocks or stubs to accomplish test repeatability?
  Use mock objects /stubs help you to isolate the item under test and keep it independent from the environment because they give always the expected values.
  
• Did you consider positive unit tests?
  To exercise the code as intended and verify the right result.
  
• Did you consider negative unit tests?
  To intentionally misuse the code and verify for robustness and appropriate error handling.
  
• Did you consider "Stress" tests?
  To check the code for its limits (overflow , etc).
  
• Did you consider data conformance test?
  To check whether an result value conforms to an expected format
  To check how the code deals with input value that conforms with expected data format and vice versa
  
• Did you consider Ordering tests?
  To check whether the set of result values are ordered or unordered as appropriate.
  To check how the code deals with input values that are ordered or unordered.
  Did you consider range tests?
  To check whether the result falls within reasonable minimum and maximum values
  To check how the code deals with input values that fall within reasonable minimum and maximum values.
  To check how the code deals with input values that fall outside the boundaries of reasonable minimum and maximum values.
  
• Did you consider reference tests?
  At the end of the method, post conditions are those things that you guarantee your method will make happen. Direct results returned by the method are one obvious thing to check, but if the method has any side-effects then you need to check those as well.
  
• Did you consider existence tests?
  To test whether the result exist (e.g., is non-null, non- zero, present in a set, etc.)?
  To test how the code deals with empty or missing input values (such as 0, 0.0, “”, or null).
  Did you consider cardinality tests?
  To test whether there are there exactly enough values in the result.
  To see how the code deals with input value list that is of the correct size or contains duplicates.
  
• Did you consider process logic test?
  To check if all control structure constructs (if, else, case, etc.) function correctly.
  Did you consider which part of the code is reasonable for testing invalid parameters?
  Check input at the boundaries of the system, and you won't have to duplicate those tests inside the system. Internal components can trust that if the data has made it this far into the system, then it must be okay. You still can take the pessimistic way but then be prepared to see a lot of extra unit tests.
  
• Do you use code coverage tools?
  In order to verify the thoroughness of the unit tests
  
• Do you write tests for bug-fixes?
  To demonstrate the bug isn’t present anymore.
  
• Do you test for specific number/datetime problems like rounding’s , regional settings?
  Decimal sign, DD/MM/YYY versus MM/DD/YYYY

• Does the unit test method only test one specific thing?
  When a unit test breaks you can more easily pinpoint the problem. Maybe even without debugging.
  
• Did you consider the use of a Mock library or a stub while unit integration testing?
  A stub is a piece of code used to stand in for some other programming functionality. A stub may simulate the behaviour of existing code (such as a procedure on a remote machine) or be a temporary substitute for yet-to-be-developed code. Stubs are therefore useful in unit testing.
  Mock objects are simulated objects that mimic the behaviour of real objects in controlled ways. In a unit test, mock objects can simulate the behaviour of complex, real (non-mock) objects and are therefore useful when a real object is difficult or impossible to incorporate into a unit test.
  
• Are the test suites enough self-documenting?
  Documentation in the form of meaningful test case names and assertion messages is usually more important than comments on test cases itself. When a test fails, the output will show the failed assertion (or an error) and the name of the test in question
  
• Is the name of the test method intention-revealing?
  Name unit test shows up in interactive screen reports and console output
  
• Did you avoid test method names with an appended numbers to distinct between them?
  That doesn’t do a very good job of communicating your intentions
  
• Is there a consistent naming convention for the unit test code?
  So you know what are the test methods and the help methods.
  
• Did you consider putting setup code that is used in every test method put into separate initialization XUnit framework methods that are triggered on test class or test method level?
  Excessive data setup can obfuscate a unit test beyond comprehension so hide it if needed.
  All state is known in advance and will always be the same no matter where or when your test is run.
  To avoid one unit test leaves some kind of dirty data hanging around
  Preconditions that must be true when you want to run the method under test.
  
• Is your unit test fast?
  Many unit tests are launched often so fast execution avoids friction and encourages you to run them often.
  
• Did you consider factory methods that create the object instance under test for you?
  You can then reuse that method to get fresh instances of your class under test in other test methods. This helps to keep the tests maintainable across time and guards your tests from unforeseen changes to the code under test.
  
• Is the actual act of executing the method under test separated from the act of asserting on the result by creating a result variable on a different line?
  The invocation against the object under test may be very long and might make your Assert line stretch all the way beyond the edge of the screen, forcing the test reader to scroll to the right.
  
• Does the variable that contains the result have a readable name?
  Makes your Assert line very understandable and easy to read
  
• Did you consider using separate verification methods?
  This is a reusable method in your test class that contains an Assert statement but that can take different inputs and verify something on them.
  You can use these verification methods when you are asserting the same thing over and over again with varying inputs.
  Even though the Assert is located in a different method, if the Assert fails you'll still get an assert exception and the original calling test will be shown in the test failure output window.
  Can improve readability.
  
• Did you avoid multiple Asserts in a Single Unit Test?
  After a failure, subsequent Asserts aren't executed.
  These unused Asserts could provide valuable data (or symptoms) that would help you quickly narrow your focus and discover the underlying problem.
  Put the additional Asserts should be run in separate, self-contained unit tests so that you have a good opportunity to see what fails.
  
• Did you use an informative assert message?
  A good Assert message should always explain either what should have happened or what happened and why it's wrong. shows up in interactive screen reports and console output
  Test code maintenance
  
• Did you avoid code duplication?
  Refactor the Test Suite as necessary and identify areas in which you can reuse code to speed up writing individual test cases
  
  

References & recommend reading

• xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros, 2007 by Addison Wesley Professional
• The Art of Unit Testing, Roy Osherove, Manning 2008 (MEAP 2007)
• Next Generation Java Testing: TestNG and Advanced Concepts ,Cédric Beust and Hani Suleiman, Addison-Wesley 2007
• Pragmatic Unit Testing in Java with JUnit ,Andy Hunt and Dave Thomas ,Pragmatic Programmers LLC 2004
• Pragmatic Unit Testing in C# with NUnit: The Pragmatic Starter Kit, Volume II ,Andy Hunt and Dave Thomas , Pragmatic Programmers LLC © 2004
• Coder to Developer: Tools and Strategies for Delivering Your Software ,Mike Gunderloy ,Sybex 2004
• EJB Design Patterns ,Floyd Marinescu John Wiley & Sons 2002
• Code Craft: The Practice of Writing Excellent Code ,Pete Goodliffe No Starch Press 2007
• Code Complete, Second Edition ,Steve McConnell ,Microsoft Press 2004

Best regards,

Alexander

MSDN session Application Lifecycle Management with Visual Studio Team System (19/12/2007)

Hello,

Yesterday I attended the MSDN evening session about Application Life cycle Management (ALM) with Visual Studio Team System (VSTS).

It was very interesting and very well presented by Yves Goeleven (Compuware Belgium). He was very knowledgeable about the subject and VSTS in particular. His presentation was a good mix between slides and demo's so you had a good feel what actually was meant.

Still I have some "concerns" about the subject. Concerns are formulated as "I wish I knew how to ...."

• Position VSTS/TFS as an ALM tool or more as an SDLC tool? Yes there are hooks (extensibility)....
• Capture the requirements from the customer in VSTS? And manage them in VSTS? Yes a wokitem will be the "materialization" of a requirement. But is this still the language of the customer?
• To model use case diagrams, UML class diagrams , User stories
• To integrate with operations (deployment, monitoring, incidents , capacity management)

I looked up the the terms ALM and SDLC. Application Lifecycle Management (ALM) is the administration and control of an application from inception to its demise. It embraces requirements management, system design, software development , testing , configuration management and operating the information system. The following picture tries to capture that idea.

Key is that the life of application does not start at the beginning of development. It start somewhere in the “business”. The end of application is not when it is delivered or even installed. It must be kept up and running to serve the business. It is also called life cycle because in effect an “application” is never finished. It is not something like a tangible product that rolls out a factory. It’s an ongoing process where new insights and requirements shape the life of an application, hence application life cycle management. As the business changes, information requirements change, and the cycle continues. Sometimes resulting in the demise of an existing system and the birth of a new application.

The software development Lifecycle (SDLC) on the other hand has everything to do with developing the information system. But it gets kicked of by the business and hands over application to operations. It is also a lifecycle because within one ALM cycle because it is the rare exception that everything is said from the beginning. So there can be several iterations before an applications gets to be delivered to the business. On the other hand of course a change request or new requirements can trigger of a new ALM cycle.

In order to achieve business objectives within an organization via information systems, user functional and non functional requirements must be defined in a consistent manner, prioritized and monitored. The administration and control of the information needs of users (Requirement Management) is therefore primary motor in the ALM and de facto also in the SDLC. All the “knowledge” that’s is necessary to transform these requirements into an information system is produced during the SDLC and put into a repository (or a collection of repositories). This will for the test basis for the test cases that will be used to verify it the information does what needs to do.
The SDLC will take that the requirement will be turned into architecture and design specifications which will eventually lead into code doing what the business expects it to do. Herein lies the objective of testing. Testing actually is conducted throughout the SDLC. Testing preparation can even start as soon as the requirements are written down (testable requirements concept). During development in the form of unit and unit integration testing or system testing (functional, security , performance ,etc). Before the application get rolled-out, the business itself can conduct test in the form of user (functional) acceptance test. Also operations can conducts acceptance tests that verify the operational requirements.

The input for establishing these test case are the expectations of the business. Ideally written down in the form of requirement, detailed functional specification and design documents. So there is a strong link between all the knowledge of what the system must do and all the information of how system is going to be tested.

A integrated system (or an integrated set of tools) should bring some added value to manage this link. Because the objective of testing is to report if the system does what it needs to do. So traceability is a key feature of this tool. Other benefits are accountability (who did what when), visibility (what did we do so far).

So if we would define testing as a kind of “fit-for-purpose” activity, testing is another gatekeeper in the life cycle. With the results of testing the stakeholders (customer/project lead) can take a decision based on facts if there will be release or not (ALM cycle) or within the SDLC cycle if something has to be corrected before release.

Monitoring the information system in production and the actual usage of the application will further keep the ALM cycle running by incident reporting (functional and non-functional).

So were does VSTS/TFS fit in this story?

I tried use the following picture to get some (visual) answers.

I can see several aspects being covered by VSTS while others (at this time) seem to not be (immediately) clear to me. Maybe this will change over time as learn more about ALM and VSTS/TFS in particular? I would be glad to hear your insights on the subject.

Best regards,

Alexander

Trigger TimedSubscription programmatically

Hello,

Subscriptions are a way to kickstart Reporting Service in delivering a report to somewhere instead of you asking for the report through a URL access request (portal or custom app). This is also callled push versus pull.

You can configere a timed event to trigger off those kind of subscriptions. But in some occasions you want some "custom" event to trigger of a subscription. You can not configure something like that through the configuration tools (reportmanger or SqlServer Management Studio).

Lukasz Pawlowski (http://blogs.msdn.com/lukaszp/archive/2005/10/07/478391.aspx) explained a way to tell Reporting Services to kick of a subscription with the SOAP API of Reporting Services. There is a method called FireEvent that can trigger the subscription. Lukasz suggests creating a dummy schedule that is will never run because it's configured in the passed. The report subscription you want to launch uses this dummy schedule. In a program you call the Firevent method on the reporting WebService .... and your report is delivered as you specifed it in your subscription.

I will show some screenshots how I did it, following Lukasz's tips.

So this is the report I want to be delivered on a file share in PDF.

First I make a dummy shared schedule. But one that will never fire a time event.

Now I create a new subscription on the report I showed before. It will use the shared schedule as Time Event generator.

In order to launch this subscription I must simulate a time event. The SOAP API of Reporting Services just gives me that.

This is a code snippet in VB.NET 2005 of how you could just do that.

It is important that you use the ID (ie. GUID) to specify the subscription. You can also look it up in the Subscription table of the reportserver database.

But before you go off and launch this code one thing is left to be configured (besides the folder share of course) . The account under which this program must execute will need "generate event" privilege on Reporting Service level. Because this is a single machine demo setup (november 07 CTP of Rosario). The user tfssetup is also as system admin on RS. So i just checked the checkbox for generating events on the RS syst. admin role .

The code snippet you just so will actually write an entry in the reportserver.Event table. The RS service will pick it up and act upon it. In our example it will write a pdf file into the file share folder (on the local disk ).

Have fun.

Best regards,

Alexander

Navigation button Visual Studio 2005

Hello,

When you open a freshly installed Visual Studio 2005 , you might miss certain things. For example the navigate backwards and forward button. Very handy when you navigate throughout your code.


You can easily add those buttons but nevertheless it is a bit strange that are not included by default (at least not on my machine) In VS 2008 (at least in the Rosario November edition) they are present and even a litlle bit smarter too. You can choose where you want to go back.


Open the "tools" menu and go to the "customize" menu-item.In the "view" category you will find the two missing buttons. When you select them , you can drag-and-drop them on your toolbar.

In Visual Studio 2008 you get the navigation buttons by default at a little bit more!

Best regards,

Alexander