Our speciality at Burch Technologies over the last few years has been tackling the hardest integration scenarios out in Enterprise IT land. We often get called in as the ‘project turn around’ guys, after somebody else has screwed something up. We’ve turned around several BizTalk/Integration projects for several of our clients in Sydney over the last half year.
There are basically 3 ways to screw up an Enterprise Application Integration (EAI) project.
1. Not getting the High Level Architecture right. What I mean by this is, EAI/ESB/SOA has its own patterns, its own ‘paradigm’ and its own traps and pitfalls. For instance, if your way of thinking about integration is moving files (flat or xml) around then you’ll be completely oblivious to the advantages of using Line of Business Adapters or database (WCF SQL, Oracle) adapters for transactional, real time, event driven processing. Where ever possible, we avoid moving files around – ask me why - because the disadvantages add up quickly. And this is 12 years of BizTalk/integration experience talking here. Another way of not getting the high level architecture right is to focus on object types (customer, invoice, whatever) and not the messaging sequence required to get that customer object updated in the ERP correctly FOR THAT SYSTEM – If only it was as easy as just calling an update operation. I could go on.
Another gotchya I’ve seen is falling in love with an ESB (Enterprise Service Bus) type of architecture and not understanding the trade offs and costs associated with it. BizTalk folks get polarised by this discussion – needlessly in most cases.
2. Not getting the code/implementation right. Some of the worst BizTalk implementations I’ve seen have been done by “BizTalk Contractors”. This is someone who might have a fair bit of experience on BizTalk but only in one particular area. For example I know of a situation where a “BizTalk Contractor” who knew B2B and EDI well but not Line of Business systems told a client that the BizTalk needed to receive an XML message from a SQL Server database (Which is not true, the database adapters in BizTalk can interoperate natively) and so the client needlessly went down the path of custom coding stored procedures to return XML. One of the advantages of using Consultants over contractors is that as Consultants we move across more client projects, more types of systems and see more types of architectures and patterns than others. And beware of the general consultant who thinks she knows integration because they did a 3 month project. You don’t get a Dynamics CRM consultant to implement a SharePoint site. So don’t get a .Net/WCF consultant to implement BizTalk – call us instead 
.
3. The third major way of not getting projects right is not getting the infrastructure and non-functional requirements right. BizTalk infrastructure requires a careful design. Putting in the installation disk and clicking next 20 times is a recipe for disaster. Additionally, there are simple things we do at Burch Technologies to make our systems easy to manage which payoff in the operator satisfaction/total cost of ownership department. For example when BizTalk is polling for data out of a LOB system the naive implementation is to SELECT * FROM Customers WHERE Status = ‘FOO’. We don’t do that. We wrap that in a storedprocedure, do SELECT TOP PARAMX * FROM CUSTOMERS WHERE Status = ‘FOO’. Then in the BizTalk receive location we can tune the throughput of that process by playing with the polling interval and the number of records (PARAMX) processed in each interval. Its nice, and it stops flood (eg end of month) scenarios from hosing BizTalk and other down stream systems. Naive implementations will never account for this type of non functional niceness. Even better is to not poll for data at all – again, ask me how .
Get those three high level components right and success is much easier to attain.
As with anything, there is a continuum of expertise. Starting with naive through to wisened and battle scarred. Make sure you get the right level of expertise to match the problem space – that's my (self serving) advice.
My other advice is beware of the person or product brochure that tells you integration is easy. At Burch Technologies, we make it LOOK EASY for our clients, but we’re under no illusions about how difficult EAI is.
My next blog posts will be about how cloud technologies are making integration easier. Easier being a relative term.