Archive for September, 2009

muSOAing for 9/28/09

September 28, 2009

Got caught up with work. Until now we have focused more on the Design and Implementation aspects of SOA and services. While the building and deployment of services is an ongoing process, the most important task of keeping this process humming along without any glitches is an equally important task.

To achieve this, there are two very important areas that are mainly related to SOA runtime being Governance and Security. Though for Governance itself you should have a well thought out strategy that should ideally start in parallel with your Development activities. Assume for a moment here that your initiative is very Strategic and you want to adopt a top down holistic view towards SOA adoption. In that case you would have initiated your Governance track well in advance.

Your SOA Czar who heads your competency center will have this on the top of his/her to do list. Governance at a very broad level can be divided into two categories, design time and runtime. In the new few posts we will examine in detail what are the main constituents of each of these blocks after which we will tackle Security.

muSOAing for 9/15/09

September 15, 2009

Let us talk about server side webservices. Since this is a vast and very technical subject we will stick to JAX-WS, SOAP (over HTTP) and JAXB which is quite standard and popular when it comes to implementing services. If you are using the JAX-WS container from Glassfish or even the containers from commercial vendors, a alot is accomplished through configuration.

Here the usecase is, a SOAP request is sent over HTTP to a service URL endpoint. The receiving component that services this call is a servlet. The very first thing that happens in the servlet is the dispatch of this call. The Dispatcher’s task is to send this request to the appropriate endpoint, in other words to the implementing class that will service this endpoint or function such as CreditCardCheck or AddressValidate, which will be sent the SOAPMessageContext that includes the complete SOAP message including payload.

Before your business logic to process this SOAP payload and headers actually happens, there is what is called a handler chain (specified in the config file as handler chain(s)). The handler chain classes mainly perform pre-validation and security related tasks. Once all the handlers have finished excuting, the objects are unmarshalled to their respective java objects thru the default JAXB binding mechanism. It is possible to substitute JAXB with another mechanism of your choice such as Castor or XMLBeans (my favorite).

Once processing is done, are response with or without SOAP Faults is sent back. Asynchronous processing is also possible which we will touch upon later. I will post post more detailed examples and information in my other blog ( since they are out of scope of this blog.

muSOAing for 9/11/09

September 11, 2009

One last note about COTS simulators. Keep in mind that these will cost you money. For the features they provide you and what you will end up paying for them, I think they are not worth it. I have done extensive build vs buy studies and have found out that they it is better to build your own simulator at least in my case due to the specialized features that were required.

Of course COTS products will have a lot of fancy bells and whistles such as a nice GUI for setting up your simulations for each service which BTW is not too hard to build in your own home grown simulator. So beware of the high costs and the vendor lockin, you normally will end up paying yearly maintenance and license fees.

Now coming to the tools, especially if you are acting as a broker sitting in the middle between a Portal Framework and the Web Services framework which you are also involved in building but is taking time to come to fruition. Being the agent you have to cater to the needs of the Portal as if you are serving up real services eventough the services by themselves are still being developed and are not available.

Remember that for the Portal they will need the various response XMLs for each screen so one or more response files will map to a particular Portal screen.

You will need a few tools if you do not have the canonicals handy. In order for you construct your SOAP messages for the Simulator, you will at the miminum need the XPath notations of all the elements in your Canonical which you should have during the course of your Canonical development. The Business Analyst is the best person who can provide you with these. From the spreadheet, create the metadata file that will consist of the screenname, XPath notation and the attribute name and have a program use this metadata file to generate the various screenwise XMLs you need.

If the Portlets for each Portal screen are being built dynamically using some technology like XForms then these tools will come in very handy to generate the XML metadata that the XForm engine will need to generate the screen.

Now that I have given you an idea of the kind of tools you may be needing, in the next few posts I will talk about the server side implementation for a web services backbone followed by SOA Governance which we can now talk about since we have services to govern 🙂 Both Design time and run time governance and about SOA Security.

muSOaing for 9/9/09

September 9, 2009

A few final notes on the Simulator before we move onto the tools. I might have gone over briefly the points you need to consider when choosing your Simulator Strategy. So no matter how big or small your services implementation may be, a simulator is absolutely necessary.

Depending on the level of sophistication you need, you might want to avail of a COTS product if you do not want to invest time in building your own and your requirements are quite straightforward and simple, i.e. you want a black box simulator with basic request response capability through configuration. No advanced features like content based responses etc.

By no means should you consider a simulator as a stop gap arrangements. Simulators will be needed and come handy even after you have rolled to production. To test stubbed out processes and to do offline demos.

Now coming to a white box simulator, you can build your own. All you need to do is to implement a SAAJ Servlet coupled with an interface driven simulator framework akin to the one I described for client side services invocation. In each simulator implementation class you will do your content based responses.

Now when it comes to responses, you can generate the requisite XMLs very easily if you have your schemas handy. If not, you will need to use tools which will be based mainly on XPath. We will talk more about these tools in the next posting.

muSOAing for 9/8/09

September 8, 2009

Now let us tackle the Simulator. In my previous post, I did articulate a few important reasons why a Simulator is needed. So here are some important points you need to consider when you want to implement one.

– How many services am I looking at which have the potential for Simulation
– How much flexibility do I need in my Simulator
– Do I need a blackbox simulator with very simple responses
– What level of customization is required, how complex are my services

Take it from me, any shop that is implementing services however big or small will need some level of simulation and there are several reasons for this. First of all, if you are designing a some complex business processes that involve the composing of one or more services, you will need to stub out or simulate a few that are still unavailable so the entire process can be tested and run end to end.

This is also important in an offline environment like a demo at a client location where you will need a simulator that can return the SOAP results that are required for the demo to run.

Guess, I have added a few more reasons for the need of a Simulator. Now let us talk about Simulator features itself. The first question that will be asked is Build or Buy. How do you answer this question. It really depends on what level of sophistication you require of your Simulator. If you want your Simulator to function like a black box. One that does not inspect the incoming request but blindly sends out a response based on configuration then there are several open source and commercial products available that let you do this.

If you need a more sophisticated one that does some intricate content based routing then you might not find a product that suits your needs. Whatever be the case, it is really not too difficult to build your own flexible simulator as I did for my previous engagement. I have found this to be very useful and have used it since in several other ones and others have been able to avail of this as well.

You can build a fairly sophisticated simulator by implementing a SAAJ Servlet that can receive a SOAP message and for the processing of content you can implement an Interface driven framework similar to a services invocation framework. The only difference is, each implementing class will implement a simulation instead of a service. In each class you can have total control over your incoming payload, inspect it and tailor your responses accordingly. As for the responses, you can either have a file driven approach or do something more sophisticated such as cache your responses in memory from a DB and then read off of memory for better performance.

Since these canonicals can be quite intricate and complex or in the case where you cannot leverage Canonicals but are waiting for schemas to be fleshed out (which can happen in a few cases), you will need tools to build out your responses. Ever tried handcrafting response files? Just Kidding!!!!!

For more detailed information of the simulator please feel free to contact me In my next post, I will talk about these tools, mainly the ones I built for my previous efforts.

muSOAing for 9/6/09

September 6, 2009

The next major component of your Services implementation strategy is the Simulator. Never underestimate the power and importance of this especially if you have an established competency center and are implementing a roadmap.

So what exactly is a simulator and how will you go about planning for and implementing it. Here is where I would like to borrow from a very common phrase, I would term this as a classic chicken and egg problem. In the ideal world the Services should be available and when the Portal comes along, the services are there to be consumed when needed. But this is almost always not the case, especially when new services are being designed and developed.

From a strategic viewpoint, services take a lot longer to actually implement than a similar Portal implementation where wireframes and the actual screens can be whipped up very quickly, at least this has been my personal experience in all my SOA implementations. As I have mentioned in previous postings, there are several reasons why Services implementations take time, even if you are ready with your Canonicals. Once the canonicals are ready for consumption, the next step is for the services implementation teams to do the appropriate mappings of the individual attributes against existing backend EIS APIs. This again may not be a straigtforward process of One Canonical to One API. Several backend APIs may have to be composed to map to one Canonical.

So once this Canonical to API mapping is done you have achieved a sort of Logical completion of your implementation task. There are still several steps you need to go through to make the service available for consumption, which is to actually implement the backend code for that particular web service SOAP Action call (web service API call). Here you are servicing the elements described in the WSDL being the SOAP Action and the input and output Schemas.

To summarize, you are in a situation where your wireframes and portlets are ready for services consumption but the services are still being developed. You certainly cannot have the Portal team twiddling their thumbs so as part of your SOA strategy you would have already thought about steps to mitigate this situation so enter the Simulator.

In my next post, I will go into the gory details of the Simulator and how you can go about implementing it as I have done in my previous engagements. So you will be privy to a lot of good information from the SOA implementation front line and I hope it will help and aid your own implementation.

muSOAing for 9/3/09

September 3, 2009

So let us tackle the three principal constituents of Client Side services. Let us start with the services invocation framework. Why is a framework important, well there are several reasons,

– The client (ex. Portal/UI) should be hidden from the implementation details of the services.
All that the client would need are three pieces of information, a. The type of service being
invoked b. The input that the service needs (typically a Request Document XML) and c.
The response that the client can expect
– As new services are being developed, you need a standard mechanism for maintaining and
invoking them. What I mean by this is, as services become available, the client side will
typically consist of these main invocation objects. a. The compiled stubs (using Axis, JAX-
WS etc) b. Your own framework classes that will wrap around these stubs and provide all
the abstractions mentioned in the previous bullet.
– Typically, these packages will be compiled into a shippable unit and be co-located with the
client or it can also be remoted using RMI, EJB or HTTP/S depending on the needs.

Now going into the details of the framework itself. If you expect to keep adding services then it is better to go with a heavyweight pre-build framework like Spring or Struts. If you have a finite list of services say under 50 then you can build your own framework.

Keeping the needs of the Abstraction components in mind, you can follow the standard OO Pattern of Interface and Implementing classes which is quite a standard pattern. When packaged as a jar or war and is colocated, the classes by themselves can be invoked by the client as long as they know the interface. All the other service details are hidden.

In the next post we will discuss Simulation which is a very important component of Services Development and Usage.

muSOAing for 9/1/09

September 1, 2009

Before we delve into the gory details of services development and deployment, I will not have done justice without talking about the client side of the interaction.

We did a Q&D description of the server side but the client side is also quite interesting. There are several components that can come into play in client side development. The main component is of course the invocation part which is mainly driven by the WSDL. There are toolkits available that make this process a breeze. All you do is compile the WSDL, generate the stubs and write some basic invocation code using the stub classes.

Of course, in a true production environment, this will get quite complicated. So here is what you need to keep in mind.
– Services invocation framework
– Services simulator
– Simulator tools (if there is significant invocations thru UI/Portal)

These are the three main constituents and how you plan and build these out are very critical as you are sort of the liaison between the client (UI) and the server (services). So you are actually playing a diplomatic role so you have to tread carefully and manage expectations accordingly.

In the next few muSOAings we will examine each of these constituents in more greater detail.