Charles Young

I spent some time today looking at Microsoft's Enterprise Service Bus (ESB) Guidance Toolkit, and this reminded me of an issue which we noticed a year ago, and did attempt to raise with Microsoft at the time, but which I never blogged about.   It's always a little uncomfortable taking the good people at Redmond to task, but, while this is hardly the most urgent problem facing today's world, it is, nevertheless, a bit of a problem and something which should be addressed.

In the article, I discuss the problem, where the Toolkit goes wrong, how UDDI should be used, and a possible remedy.    The article is at http://geekswithblogs.net/cyoung/archive/2008/11/12/126975.aspx.

I'm delighted to report that Microsoft is currently asking for responses to an on-line survey on the use of Microsoft BRE - delighted because it indicates that serious thought is being given to the evolution of this technology.    If your organisation uses MS BRE, do please take a few minutes to provide feedback.

The survey is at:   https://live.datstat.com/MSCSD-Collector/Survey.ashx?Name=BRE_Usage_Survey_Blog

The only issue I would highlight is that the survey deals mainly with present and past use of MS BRE, and not about how its use could be extended in the future.

 

The last couple of weeks have seen a significant increase in terms of announcements and information from Microsoft ahead of the Professional Developer’s Conference next week.   It is a key time for Microsoft’s Connected Systems Division (CSD) as they go public with their plans for .NET 4, Oslo and ‘Dublin’.  Microsoft’s announcement of Dublin led immediately to an interesting, if somewhat predictable, debate within the company I work for, and I think it is worth going over some of that territory here.

 

http://geekswithblogs.net/cyoung/archive/2008/10/15/125848.aspx

Microsoft announced BizTalk Server 2009 today, and gave the green light to talking about the new version.    It’s due for release in the first half of next year, and is shaping up nicely.    Microsoft is casting BizTalk Server 2009 as a major new version in its own right, rather than just an updated 'release' of BizTalk Server 2006.   This is an important move, and one I strongly welcome.   There is certainly enough in BizTalk Server 2009 to warrant thinking of it as a major revision of the product, although it retains the same familiar functionality and tooling we have been using since 2006 (or even 2004).

I've been fortunate in getting my hands on the current non-public CTP in the last month or so, and putting aspects of the new version through its paces.   It's not wise to go into much detail about this first CTP because some of those details will doubtless change in forthcoming betas, driven in part from the feedback Microsoft is getting.   However, I will say a little about the new development and build features and expand just a little on the press releases.   These are the areas I have been looking at in depth.

BizTalk Server 2009 will ship with bindings for Visual Studio 2008, and you will need to upgrade to this version of the IDE if you are still using Visual Studio 2005 (which you will be, of course, if you are a BizTalk Server 2006 user).  It happens that, in previous months, I've seen more than one BizTalk shop where developers are having to run both versions of the IDE side by side in order to continue developing for BizTalk Server 2006 whilst exploiting features like WCF and WF in .NET 3.5.   For many, the changes in BizTalk Server 2009 will come as a relief.   This is not a cynical ploy by Microsoft to force people into upgrades for no additional benefit.   The move to Visual Studio 2008 is accompanied by a very welcome move to a new project format, bringing BizTalk Server 2009 into line with mainstream .NET development.   The new BizTalk project type is defined using MS-Build, just like C# or VB.NET projects.   This has major implications in a number of areas.   First, it means that, if you use TFS Build, you can now build BizTalk Server 2009 projects without having to write complex scripts that shell out to DevEnv.    Just like C# projects, you can let TFS Build do the majority of the grunt work for you, and concentrate your attention more fully on ensuring that your automated build scripts are comprehensive and robust.   This is worth the upgrade in its own right, and removes a major source of current irritation.   Thanks to the rough edges in the current CTP, one of my colleagues has had the opportunity to get to grips with this side of BizTalk Server 2009 in some depth, and we can report that it is all looking very good indeed, once a few remaining gremlins have been chased out.

The other aspect of BizTalk Server 2009 which we have spent some time on is unit testing and debugging.   In one sense, this is not really so much about new functionality, but more about bringing what has previously been considered 'black-belt' into the mainstream.    In Biztalk Server 2006, we currently use BizUnit to drive black-box (grey-box?), end-to-end testing, but we have also created some additional code of our own to support unit testing of BizTalk maps, schemas and pipelines.    Similarly, if you know the undocumented registry key in BizTalk Server 2006, you can get BizTalk to retain the generated C# code for debugging purposes, manually attaching to BtsNtSvc.exe processes in order to debug into orchestrations, etc.    This has proved a life saver in certain situations.  The new version of BizTalk Server is now designed to support these approaches seamlessly.   Of course, it is designed to work with the integrated unit testing features of the IDE, rather than BizUnit.   There are currently some outstanding questions with regard to the debugging support which I won't go into here, because they simply reflect the unfinished state of the first CTP.   However, unit testing certainly works smoothly with BizTalk Server 2009 projects, and will help to raise the bar in terms of the approach that is taken to development of BizTalk solutions.   Again, I will avoid going into further here because I will hit issues that are still to be fully resolved, but things are generally looking good.

I have only mentioned those areas of BizTalk Server 2009 which I have been looking at in any depth.   There is a lot, lot more.   One thing I will monitor closely is the inclusion of ESB Guidance 2.0.   I have some problems with some aspects of ESB Guidance 1.0, like the unfortunate way in which the UDDI resolver violates the UDDI standard!    However, having spent the last year doing little else but designing and implementing service bus patterns using BizTalk Server and WCF (or, in one case, WSE), I regard the inclusion of ESB Guidance 2.0 as a really intriguing, and hopefully worthwhile, aspect of the new version.    It is also intriguing that Microsoft has decided to release their implementation of UDDI 3.0 as part of the BizTalk package.   Let no one tell you that BizTalk has no role to play in building service buses, or that it is to be regarded as 'merely a hub-and-spoke message broker'.   In my experience, the exploitation of the dynamic features of BizTalk (dynamic ports, BAM interception, the rules engine, etc.,) in a rigorous, policy-driven fashion provides good support for implementing many of the core patterns described within the world of ESB.   The core design of BizTalk Server predates later ESB thinking, but is much better aligned to it that some people will admit.

Roll on 2009.

Jurgen Willis, who is Group Program manager of Microsoft's Connected Framework Team over in Seattle, has been in touch, and I offered to forward his request on via this blog site.   Microsoft is currently looking for people to work in their rules area (WF Rules, etc).   They are specifically looking for a Program Manager and a Development Lead

• Community PM
• Activities PM
• Runtime PM

Being known for my interest in rules processing, I quite often get asked to help with problems with MS BRE.   A couple of days ago, I was asked to help investigate an issue occurring in production for a BizTalk Server application.   Occasionally, in a fairly high throughput system, BizTalk logs an error stating that a problem has been encountered while executing a rule set.   That is the only information provided, with no hint of what the problem might be, and because the issue only occurs intermittently under real-world conditions in the production environment, it was not obvious how to obtain further insight without disrupting live operations.

This article investigates one way of handling this deilemma through the use of the compensation handling feature of Microsoft's Business Rules Engine.   It goes on to discuss the broader use of compensation handling in rule processing.

http://geekswithblogs.net/cyoung/archive/2008/01/23/118804.aspx

The recent release of Visual Studio 2008 and .NET Fx 3.5 is causing some confusion.    Microsoft released these two technologies together for good reason.   The wonderful new LinQ technologies introduced in .Net 3.5 rely on explicit compiler-level support, and therefore require LinQ aware compilers in Visual Studio.   The new version of visual Studio provides these compilers, allowing developers to take advantage of the new monadic syntax. In addition, Visual Studio has several new features designed to make it easier to exploit NET 3.5 features such as Ajax and the foundation libraries (WCF, WF and WPF).

The problem is that by tying the release of .NET 3.5 to Visual Studio 2008, the impression is given that, unless you are ready to upgrade to the new version of the IDE, there is no point thinking about upgrading to the new version of the framework.    This is simply not the case.   The .NET framework does not have any built-in dependency on Visual Studio, let alone a specific version of Visual Studio.   More to the point, Microsoft has long since split the versioning of the framework from the versioning of the run-time environment. .NET 3.5 continues to exploit version 2.0 of the CLR. Visual Studio 2005 is perfectly happy to compile your code against .NET 3.5 assemblies.   They are just assemblies.   Even more compelling is the realisation that most of the assemblies in .NET 3.5 are identical to those in .NET 3.0 (same version number).   There are some new assemblies with new features. .NET 3.5 is just .NET 3.0 with extra stuff.

Why is this important?   Well, not everyone is ready to upgrade to Visual Studio 2008.   Apart from the expense this involves, consider the dilemma of BizTalk Server developers.   Currently, there are no Visual Studio 2008 bindings for BizTalk Server (i.e., you can't create BizTalk Server project types in the new IDE).   This, we are assured, will be addressed at some point, but that could be months away.    For the time being, BizTalk developers are stuck with Visual Studio 2005 :-( Hence, some people are currently discounting the possibility of using .Net 3.5 because they believe, quite incorrectly, that it requires an upgrade to Visual Studio 2008.

There are issues, of course.   As well as the absence of compiler support for LinQ, Visual Studio 2005 does not have access to various new project and file templates and tools that support the new version of the framework.   Developers may need to do more coding in Visual Studio 2005 than would be necessary in Visual Studio 2008.   This is often a small price to pay, however, in order to access the improvements in 3.5.    As an example, consider the new integration between WF and WCF, provided in the new System.WorkflowServices assembly.   The integration is provided via the new WorkflowServiceHost class and a couple of new activities.   Visual Studio 2008 has new template support for building workflow services, and comes with a very useful new WCF test harness.   However, exploiting this new functionality in Visual Studio 2005 is trivial.   Create a WF workflow library, add a reference to System.WorkflowServices and add the new activities to your tool box.   Finally, use the WCF Service template to add a service class to your project and you are just about in the same position as you would be in Visual Studio 2008 if you used the new Workflow Service project template.   You'll need to write a couple of lines of code to use WorkflowServiceHost to host your service, of course.   Off you go, and enjoy .NET 3.5.

Noon (Redmond time) on Tuesday 30th October marks the moment in history we can begin to talk publically and openly about Microsoft’s 'Oslo'.   Oslo is the codename for the next generation of Microsoft's SOA platform...'Microsoft SOA vNext' if you will.   It is a huge vision, and one which Microsoft is throwing some serious resource at.   Their strategy will unfold over several years, but the first public view has been provided today. 

Get more details at http://geekswithblogs.net/cyoung/archive/2007/10/30/116456.aspx. 

Earlier this year, Microsoft released a CTP version of a new technology stack called 'BizTalk Services'.   In July, they updated this with a new release.    It is only now, at long last, that I've got around to playing with BizTalk Services, thanks to the gentle pursuasion of my colleague, Andy James (CTO at SolidSoft).   I must say that I am very pleased that Andy applied some pressure.   This is a really interesting development in the BizTalk world, and having now had a chance to stub my toe against the tyres (I'll leave the full-blown kicking until a future date), I feel rather excited about the possibilities this technology opens.

I have provided an overview of BizTalk Services at http://geekswithblogs.net/cyoung/archive/2007/10/09/115944.aspx.   I have also written an article on the Identity Service that is part of the stack.   This is at http://geekswithblogs.net/cyoung/archive/2007/10/09/115943.aspx.   I have published some notes on solving problems when installing the SDK.   These are at http://geekswithblogs.net/cyoung/archive/2007/10/09/115942.aspx.

/*********************************************
 * Group 1 - 'Started' state
 ********************************************/
 
Rule 1
IF
 AND
    Context.CurrentState == "started"
    MyFactA.Property1 > 5
    MyFactB.Property1 < MyFactA.Property
THEN
 MyFactB.Property1 = MyFactA.Property1
 assert MyFactB
 
Rule 2 (priority -1)
IF
 Context.CurrentState == "started"
THEN
 Context.CurrentState = "initialised"
 assert Context
 
/*********************************************
 * Group 2 - 'Initialised ' state
 ********************************************/

Rule 3
IF
 AND
    Context.CurrentState == "initialised"
    MyFactA.Property2 == "USA"
THEN
 MyFactA.SetCountryCode("01")
 
Rule 4
IF
 AND
    Context.CurrentState == "initialised"
    MyFactA.Property2 == "UK"
THEN
 MyFactA.SetCountryCode("44")
 
Rule 5 (priority -1)
IF
 Context.CurrentState == "initialised"
THEN
 Context.CurrentState = "completed"
 assert Context
 assert MyFactA  // convenient place to re-assert MyFactA 
                        // to avoid loops if, for example, 
                        // MyFactA.SetCountryCode() changes the 
                        // state of MyFactA.
 
/*********************************************
 * Group 3 - 'Completed' state
 ********************************************/
 
Rule 6
IF
 AND
    Context.CurrentState == " completed "
THEN
    MyFactA.Property1 = 0 
    MyFactB.Property1 = 0
 
/********************************************/

I had an unpleasant time yesterday afternoon re-stabilising an ASP.NET 2.0 application after it went berserk.   I don't do that much coding in ASP.NET these days, but when I do, I seem to run into lots of problems. A fellow developer on the same project wasn't very helpful or sympathetic - he sat there and informed everyone in a loud voice that he'd just come off another project where he'd built a much larger ASP.NET app without any problems at all.   Yes, thanks for the support!

I had several problems layered on top of one another, including those strange issues where Visual Studio 2005 seems to get into a complete mess somehow with assembly references. I also saw evidence of Visual Studio putting incorrect duplicate file references into the project file. Killing every bin and obj folder and deleting the entire contents of the Temporary ASP.NET Files folder helped no end. However, I had one problem that I couldn't clear for ages.   Certain .aspx pages constantly reported errors associated with their @Page directives.   One page, in particular, constantly reported this error (even after I had re-built the page from scratch), and others intermittently reported the same error!   The error message stated that:

"There is no build provider registered for the extension ''. You can register one in the <compilation><buildProviders> section in machine.config or web.config. Make sure is has a BuildProviderAppliesToAttribute attribute which includes the value 'Web' or 'All'."

Why on earth would I need to register a build provider for non-existent file extensions?   The problem seemed also to manifest itself in an inability to find a master page unless I provided a fully qualified path name.   Another strange thing was that sometimes a different error would be reported by Visual Studio saying that part of the path to the web application directory could not be found.   Needless to say the path was correct and present.   The problem survived cleaning down the project and rebooting the system. 

I tried everything I could think of to solve the issue.   Nothing worked until, by trial and error, I hit on a solution.   In Web.Config, I added the following buildProviders extension entry to the compilation element:

    <compilation debug="true">
      <buildProviders>
          <add extension="*" type="System.Web.Compilation.PageBuildProvider"/>
      </buildProviders>
    </compilation>

Success!   The problem disappeared.   I then had the presence of mind to IMMEDIATELY go back and comment out the buildProvider entry. The problem did not come back!   Whatever happened when I added the build provider seems to have solved the underlying issue.

I can't explain the cause, but I can see a few reports of the same delinquent behaviour when I Google.   Hopefully this will be of help to someone out there.   I really do hope Microsoft has put some effort into stabilising Visual Studio 2008 rather than just concentrating on adding features.

Also, before someone asks, yes, I do have Visual Studio 2005 SP1 installed, and yes, I am using a Web Application project rather than a Web Site project.

I had an interesting time yesterday writing a rule set in order to demonstrate that Bayesian analytics can be combined with rule processing in an entirely natural fashion.   Armed with an understanding of how a rules engine works, I also believe that it is entirely possible for a Rules Engine to implement and apply Bayes theorem in an efficient manner.   I wrote the rule set for a Java-based rule engine called Jess, so please note this is not a BizTalk-related post.   

My reason for writing the rule set was to counter some statements published on the Complex Event Processing Blog site, including references to a 20-year old academic paper that appears to claim, incorrectly, that rules engines cannot efficiently handle the concept of uncertainty.    Bayes theory is one way of handling dependencies amongst uncertain beliefs (i.e., calculating the probability (and changes to probability) of the accuracy of hypotheses based on uncertain beliefs and evidence).   Another approach to handling uncertainty is to employ 'fuzzy logic', which I found myself demonstrating to a client (using MS BRE) just last week.   But that's another story...

If you are interested in this rather obscure discussion, please read on at http://geekswithblogs.net/cyoung/archive/2007/08/27/114988.aspx

 

I got an email today requesting help in deciding the appropriate selection of rule processing technology for a workflow application.   I’ve got requests like this before, so I’ve decided to post a reply publically.

http://geekswithblogs.net/cyoung/archive/2007/07/19/114061.aspx.

Apologies if anyone has been experiencing IE7 crashes when trying to read articles on this site (though I don't really see why I should apologise for IE7 crashing!).   As far as I can tell, this seems only to affect IE7 users on Vista.   IE7 on XP appears to be unaffected, and Firefox is OK.   The problem appears to be related to having a 'blogmap', although it only affects certain pages (articles, rather than posts).   I've removed my blogmap, and the problem appears to have gone away.

 

If anyone has any further problems, please contact me via the site (once you've recovered from a crashed IE7 instance, that is!).

 

Update: The problem continued to re-occur, although far less frequently.   So I played around some more.   I had switched on the Subtext CoComment feature.   I switched this off, and things got much more stable.   So, something about CoComment support in SubText seems to cause an IE7 crash.   I've put the blogmap back in.   Seems to be working OK.

I posted an article on MS BRE side effects yesterday.   I had immediately to withdraw it for a few hours because I realised it was incomplete (and actually a little wrong) in one part.   Then I noticed I had swapped the legend text on the graph making it appear that caching made things slower, rather than faster!   I then discovered today that for the last three years I have been completely ignorant of the fact that you can drag and drop object constructors onto the 'assert' argument in the Rules Composer in order to expolit the built-in CreateObject function!   Big red face.   I have changed the offending paragraph where I moaned at Microsoft for not providing access to this functionality!   Given the number of times I have complained about the lack of comprehensive documentation for BizTalk and related technologies, it is ironic that Microsoft has documented this particular feature very cleary indeed, and even given a brief description of what happens behind the scenes!   I am suitably chastened.

The updated article remains at http://geekswithblogs.net/cyoung/articles/111169.aspx.   If anyone spots any other artefacts of my ignorance or inattention, do please let me know!

For almost two years now, I've been intending to write an article about the mysterious 'side effects' flag used in Microsoft Business Rule Engine policies. Microsoft documents this feature (see http://msdn2.microsoft.com/en-us/library/aa559124.aspx), and describes very briefly how to control it. The mystery that surrounds this flag arises because it is represented by an attribute named 'sideeffects' in Microsoft's BRL (Business Rule Language) although it actually controls a caching mechanism,