Archive for August, 2009

muSOAing for 8/26/09

August 27, 2009

Services Design, now before you know it things will get a lot more interesting. Rubber meets the road and all that sort of thing. Let me do a very Quick and Dirty scenario where you want to build out the server side components of one of your services. Remember that the server side is serving a client request, which means it receives a request and returns a response. This means that you will need a Request schema and a Response Schema. For now we will assume that all the calls will be Document style calls.

A quick lowdown on Document style APIs. All request and response objects are sent as SOAP requests and responses. You should all be familiar with SOAP so will not go into the details, suffice to say that in a Document style API, the payload which can be the request of the response XML will be sent in the SOAP Body. The general convention is to suffix this schema with Req such as InvoiceInfoReq. On the server side, when you get a request (via HTTP) you will know the type of request (which function call) thru an important parameter called the SOAP Action. Using this you will determine what you will do to service the request. Using the SOAP Action and some input parameters contained within the SOAP Body payload, you will have to fetch all the required EIS data, map it to the canonical response schema, encapsulate it in the SOAP Body and sent the response back. The convention is to suffix it with Rsp such as InvoiceInfoRsp.

Here is your quick and dirty description of a service interaction. Now how will you actually implement all this. What is the actual service infrastructure you will deploy on the server side (and on the client side) to enable such an infrastructure. Remember that you are now bringing together the work you have done earlier being the Canonical schemas and probably some legwork related to your service APIs such as SOAP Actions, service styles (Document or RPC) and later on the WSDLS to glue all this stuff together.


muSOAing for 8/18/09

August 18, 2009

Services design and development requires a lot of thought and planning. A mature organization will be spending a lot of time taking stock of their existing APIs and service enabling them rather than embarking on a lot of new services development.

Service development is a very tedious task as it requires you to marry the API to your canonical schemas. This task itself will take up a bulk of your development time. While an API is your traditional function call on the server, a lot has to happen for that function to be called as a service. You will first of all need the schema esp if it is a document style API. You will then need the WSDL to chart out all the service characteristics such as the protocol, SOAP Action and so on. The comes the task of building the server side components for service lookup and delegation and a lot of other related tasks. Then the client side development for calling the service.

I have run through this very quickly. In my next posts, I will distill and explain each step in more detail.

muSOAing for 4/16/09

August 16, 2009

The next big challenge for any SOA implementation is services modeling. There is a lot of leg work and planning that needs to be done before you embark on this important task. Ask yourself these set of questions,

– How many existing APIs am I looking at: Do an audit of your existing set of core business APIs that you want to expose
as services both internally and for consumption by trading partners

– SLAs for services: Who is the caller and what are the SLA requirements for these services. Beware that services are
sometimes quite notorius for throwing your SLAs out of whack so tie this down very well right from the beginning.

– Granularity for services: One of the key factors that will determine this is your SLAs. Another factor is whether this will
be a shared service. Another point to note is, services can be composed so whether they are coarse grained or fine
grained will at times not matter if BPM is a key part of your SOA strategy. You an always compose coarse grained
services but you are walking a tightrope here so don’t go overboard. Always do small POCs to validate your findings
instead of going by what you see on paper.

– Services design: Any IT shop will have to deal with two modes of services development. One is to develop services from
scratch and the other is service enabling existing APIs through wrappers. We will get into the gory details of this in future
posts but there you can run into debates about pure web services vs EJB wrappers etc. How do you decide.

– Services Framework: How do you build and deploy your services. The frameworks for the client and server side of
services. This is another very interesting topic and we shall get into the details in future posts.

– Services Security: This is another very important aspects and there are several things to consider. It will primarily be
driven by your overall corporate Security Policy. It can be a federated model or a decentralized model.

Once you have services in place we can get into the details of other aspects such as Governance, Monitoring, Versioning etc.

muSOAing for 8/14/09

August 14, 2009

So that was a whirlwind Canonicals tour. I know it was quick and brief but hey I did touch on and highlight all the key components. If you want more detailed information please don’t hesitate to contact me ( In this post I want to summarize on all the stuff we talked about.

– SOA Competency Center
– SOA Champion
– Appoint Canonical whip/team lead
– Select departmental SMEs who will be on loan to SOA CC
– Whip selects a hands on tech lead
– Audit of existing data representations
– Whip to pull in SME resources as needed
– Pull in Industry standard Schemas
– Taxonomy Normalization and Mapping
– Test drive Canonical through Pilot
– Technical teams to validate and prepare for internal mappings during Pilot and Implementation

One thing that I did not highlight is what do you do if there are no Industry Standard Schemas to leverage. Well then if you have followed all the steps listed above then you probably will have very little or no Taxonomy Normalization and Mapping to do. What you come up with will be your data representation. This may bite you in future if you have to conduct B2B transactions but then if B2B is involved then there will definitely be a standards body that would have thought though and brought out these schemas.

Happy Canonicling. In subsequent posts we will get more tactical and deal with the interesting topic of Service Modeling.

muSOAing for 8/13/09

August 13, 2009

The normalization of taxonomies can be very tricky. If there are vast differences between the taxonomies then the best option is to maintain an internal canonical that maps to your Industry standard one. The mapping exercise is crucial. You certainly do not want to modify the Industry Standard schema to incorporate this canonical. One of the chief reasons that such a modification is ill advised is if you conduct online B2B transactions then in most cases the trading partners should share the same version of the Canonical. I don’t think you can force your trading partner to adopt your taxonomy 🙂

In such a case, some level of mapping and transformation will be required. Keeping in mind that such operations can prove expensive and could throw your SLAs out of whack, keep them to the barest minimum and only where absolutely necessary.

When it comes time to test drive a canonical, make sure that you have budgeted for this and try to achieve this through a Pilot project involving at least two departments that had been hitherto using their own custom representations. Once you have provided the Canonical, there might be some work involved to do inbound and outbound mapping depending on who the producer and consumer is. This is integration work and all of this should be accounted for in the Pilot as well as Implementation Phases.

muSOAing for 8/12/09

August 13, 2009

So after the audit you should ideally end up with the various representations and taxonomies for the canonicals you are trying to define such as Purchase Order, Purchase Order Ack, Sales Order, Inventory etc.

Next comes the really arduous task of mapping these to the industry standard schemas. If you are in high tech then you are in luck as there is a lot of work already done in this area by RosettaNet and Oasis and various schemas such as RosettaNet, cXML and ebXML can be made avail of.

To get access to these schemas a nominal annual membership fee is required and there are a lot of advantages that can be gained from such a membership including access to schemas, participation in standards definition and other meetings.

Next comes the task of mapping and normalizing your taxonomies. It is always better to stick to the industry standard. Things such as Item#, Part#, SKU#, InventoryID can mean the same thing but you will have to adopt an industry standard taxonomy. It is not recommeded that you modify these schemas but in certain cases it becomes necessary that you do so.

These key tasks of Audit, Mapping and Taxonomy normalization will take up the bulk of your time in this process of Canonicalization. This is the stage you might have to pull in additional BAs and SMEs to shepherd this process through. This will also require frequent meetings of the SOA steering committee for inputs.

muSOAing for 8/11/09

August 11, 2009

So how ambitious should you get with Canonicalization. Quite ambitious actually. Any halfhearted measures and short cuts will only lead to lot of heartburn later and it is very easy to take short cuts. There are several patterns here which I will discuss in subsequent posts but suffice to say that upcasting or downcasting by mapping from a superset schema to your own small custom schemas is always not recommended.

The BA who will be driving this effort has to possess certain very unique skills and a bulk of those skills are soft skills akin to a Secretary of State as it involves liaisioning with the key players and department heads, driving and arriving at standards while at the same time ensuring that you do not step on any toes and rub people the wrong way so this person carries a lot on his/her shoulders.

One of the first tasks that has to be undertaken, given the scope of the effort which will normally be company wide as any meaningful SOA implementation has to encompass the entire organization, will be to take stock and do a thorough audit of the existing schemas and data representations being used. You certainly do not want to find late in the game that there are 4-5 representations for Order being used by these 5 different depts. The goals is to normalize and have these 5 depts use one common Order Canonical.

So this audit phase is very crucial and will consume a lot of time. The involvement of the Tech Lead at this point is also very crucial to vet and catalog all these schemas in both a document and a registry for easy lookup.

muSOAing for 8/10/09

August 11, 2009

So how do you start off on Canonicals. You must have completed a few of the critical steps listed for the SOA Competency Center. A core overview and implementation team should be in place. The first step for this is to involve at least two people, a competent Business Analyst with good Subject Matter Expertise. This resource should be fairly technical with fairly good knowledge of XML and Schemas.

Working closely with this BA should be a strong technical resource who works closely with the BA to provide hands on technical support, including implementation of Pilots.

Here is a cheat sheet of how this Canonical Modeling Task should start. As I have mentioned already, this is a strategic effort so in order for it to be effective, you have to go the whole nine yards. It has to encompass the entire organization. Depending of the size of the organization this effort can range from 3m to 1yr even if schemas are available from RosettaNet, HIPAA etc.

muSOAing for 8/9/09

August 9, 2009

Continuing our conversation on Canonicals. The last remark was on not to reinvent the wheel. Some careful planning has to be done for Canonical work. Here are some important ones,

– SOA Competency center: This is needed for any strategic effort and not just for Canonical work. This is just one important function that this center will be responsible for.
– SOA Champion: Appoint a champion for this competency center. He will be the whip, the go getter, the evangelist who will keep mantra of SOA alive and kicking in your organization. An SOA champion is typically a very well liked and competent Technical Leader with lots of soft skills as this will involve lot of liaision with different depts within the organization
– SOA core committee: Nominated reps from the various depts who will work with the SOA champion. This is part of the important liaision work
– SOA implementaition group: This will normally consist of a mixture of BAs, Tech Leads and Hands on technical folks. The BAs and Leads are generally loaned from the various depts for SME and their participation will be part time and decrease over a period of time. The hands on folks are full time resources
– Canonical committee: Overseen by Champion with help of committee. Implemented with the help of relevant BAs and technical resources.

In the next post we will discuss how to kick off this important effort. I cannot stress enough the importance of this effort so I will be making sure to post daily on this topic till it has been covered sufficiently and thoroughly.

muSOAing for 8/8/09

August 8, 2009

Before we start on the subject of service design there is another very important aspect of SOA that we need to consider and that is the Canonical Schema. While it is very easy, convenient and expedient to use one of schemas or make up your own schema for the service, this practice will prove to be very expensive going forward.

If you have initiated a strategic track for your SOA implementation then it is even more important to move Canonical Modeling to the top of your priority list. Canonical Modeling may sound very innocent and infact after an initial study may even seem trivial but you will be very sorry and disappointed if you delve into it with this mindset.

Canonical modeling is anything but trivial. I would even go to the extent of saying that it is one of the most difficult and tedious tasks in your SOA roadmap. If not planned for and implemented properly, it can also turn out to be a very expensive task so approach with caution.

One of the most important things to remember in Canonical Modeling is “Do not reinvent the wheel”. You may think that I am dropping a cliche here but no, I am quite serious. In fact, the importance of Canonicals have been seen very early on and a lot of Vertical Specific work has already gone into them. Some of the earliest and famous ones are RosettaNet and Oasis.