Archive for the ‘SOA (Service Oriented Architecture)’ Category

muSOAing for 6/17/11 – Hadoop Metrics

June 17, 2011

One big area of interest for me is the metrics for Hadoop. This is the stuff you can see in the default web page thru port 50030 and also available for you thru the JobClient API. As of this writing, this whole infrastructure is undergoing a sea change as is the case with the APIs in general. All the APIs are changing for both Hadoop and HBase.

This is a good sign. It says that the adoption of these tools is increasing and as people are using these products, they are demanding more features. It seems at this point that the primary users of these platforms are from the DW camp. Their interests lie in using alternate platforms to perform the same DW and BI tasks that they have been doing with traditional relational DB oriented infrastructures. Moving to a new platform like Hadoop also requires a different mindset which means unlearning a few of the concepts associated with traditional platforms. It will also mean coming out of the comfort zone and expecting Hadoop+HBase to behave exactly like your Oracle datawarehouse. This is a bit like switching from Windows to Mac. Things will only get better, it is just that you have to get used to it.

I think I have digressed a bit, I started off with Hadoop metrics but then wandered off and done some pontification.

muSOAing for 5/30/11 – Can I use HBase instead of…….

May 30, 2011

This is an oft posed question these days. Folks looking to cut down on their DW licensing costs, looking for alternatives. The Hadoop/HBase platform is definitely a candidate for many popular use cases. This platform is rapidly evolving and is already turning out to be a big challenge to all the traditional DW players. Talk about disruptive technology, this is as disruptive as it can get. Map/Reduce has changed the entire status quo. It has unleashed a veritable Tech Tsunami that is threatening to wash away those who are unwilling to acknowledge and adapt to this rapidly changing landscape. All my blogs in the near future will focus on BigData as there is a lot happening in this space and hence a lot to mull and ponder upon.

muSOAing for 5/13/11 – Are you Hadooping yet?

May 13, 2011

It should be quite obvious by now that Big Data is catching on big time across all Industry verticals. In my personal interactions there hasn’t been one organization that is not looking at Big Data solutions to deal with their data analytical and storage needs. The top on this list is of course Hadoop and it’s ecosystem of offerings chiefly HBase and also Hive, Pig, Zookeeper, Mahout etc. It seems that almost everyday something new is happening in this space. Folks are talking about fault tolerant Hadoop installations, next generation map reduce and a host of companies like NetApp, EMC etc. are offering Hadoop based solutions. These are very exciting times for Big Data and Advanced Analytics.

muSOAing for 4/27/11 – Relax and be RESTful

April 27, 2011

Having always dealt with WSDL based SOAP services, when the RESTful mantra started to be bandied around a few years ago, I was really curious. I was of the firm belief that there should always be a firm contract between the caller and provider of the web service, even if the caller or client is internal to the organization. Having lived in that comfort zone, I was one for dismissing REST services as trivial and not to be taken too seriously. This idea of overloading the URL to send the metadata and then sending just the payload and processing it at the backend did not appeal to me a lot.

However with increased adoption and annotation support from frameworks like Jersey, I have started taking a serious look at REST. The ease with which one can churn out a service warrants a serious second look at this paradigm. I am of the firm opinion now that unless services need to be published for external (B2B) or enterprise wide consumption through a service registry, there is really no need to adopt WSDL based SOAP services. Where a contract is not of prime importance and you can do away with the overhead, REST services will suffice.

muSOAing for 4/17/11 – Write once Read Many?

April 17, 2011

One of the features of a Big Data setup is it’s Write once Read Many paradigm. Any Big Data infrastructure like Hadoop is still a data warehousing infrastructure used for analyzing historical information. Your relational store will still be your repository for ongoing OLTP needs with data being ETLd into your Big Data infrastructure. With data being written to file systems and being analyzed using map/reduce at the lowest level. Advocates encourage the use of higher level tools like Pig and Hive to perform analytics. These tools do execute map/reduce for you but provide you with higher level SQL like interfaces that you are already familiar with to issue your commands which are translated into map/reduce directives under the covers.

With the adoption of Hadoop increasing by the day across all verticals, the need in this area is only going to increase. It also has something for everybody, the technology nerd who can get started on the cheap to your CIO who can now have a multi-node Big Data infra up and running in no time and churning out useful and timely business analytics.

muSOAing for 3/24/11 – The Vertically Integrated Architect II

March 24, 2011

So what are the other qualities. Being able to provide thought leadership and the willingness to evangelize and put forth your thoughts and ideas in various fora such as personal blogs, articles, whitepapers and even downloadable code artifacts. All these make for a well rounded architect aka a Vertically integrated one.

An Architect should be able to see the big picture, the so called 1000 foot view, as well as be able to roll up his/her sleeves and be able to build reusable artifacts. If you do not code then you are not the master of your own domain and you cannot be a true Architect.

Being able to take a problem however nebulous and be able to turn that into a solution, that is the ultimate challenge, being able to bring true value and be able to solve real world problems and at the same time be able to address all the key business drivers such as ROI, value proposition etc. In a sense, I would not be wrong if I said that the end here truly drive the means. The end goal is a well architected system and the means is to bring to bear your full complete set of Architectural firepower.

muSOAing for 3/16/11 – The Vertically Integrated Architect I

March 16, 2011

There are a few yardsticks by which a truly Vertically Integragted Architect should be measured. First and foremost is the passion you have for Architecture and how it manifests. The technology landscape is so fluid and has undergone such radical changes that it looks vastly different from what it was from even last year. Anyone who says that I only focus on this or that field is so 2010. Thinking out of the box and being able to adopt the changes and apply them to solve key business problems is one key element.

In order to do that, one has to do a variety of tasks including copious reading, tinkering in the form of downloading products and toolsets and being able to configure them and build artifacts. Being hands on is the key, anyone who says he does not code and is not willing to cannot be an Architect. This is one of the most basic requirement but there are other manifestations in addition to these key ones.

muSOAing for 3/3/11 – The Vertically Integrated Architect?

March 3, 2011

I am posed with this question frequently, do you consider yourself an Architect or an Enterprise Architect? Or what qualities are expected of a person in this role. I always assume that the questioners are either trying to measure you against their own pre-established yardsticks or they are sincerely seeking an answer to clear out their own confusions.

There are of course the stock answers to such questions. An Architect must see the broader picture, the 1000 ft view. Should be solutions oriented, view the landscape holistically etc. But of late, I have begun to pose this question to myself. Looking at the current landscape where everything is a constant moving target, the grass can tend to grow under your feet even if you rest for that brief moment. So after some intense evaluations, I have come up with this axiom. An effective SOA or Enterprise Architect is one who is the most Vertically Integrated.

Now what does that mean. To answer this, let me take an example. Let us say, of the myriad questions that can be posed to you in a meeting with “C” level execs and fellow Architects, this one is thrown at you. We have been accumulating a mountain of data and are having problems storing and analyzing it. The current architecture just does not cut it and we need something radically different.

Of course, an answer cannot be arrived at in that room, you can only come up with some rough suggestions but let us say you move on to the next steps such as Discovery, findings, roadmap etc. Now it is time for the rubber to meet the road and you have settled on BigData oriented solution. So you plod on with the other steps, POC, Demo and finally implementation. So far you have played the role of Architect to a “T” but the question is, how did you go about doing it. Apart from the role you played, what did you exactly do besides interacting and having face time with all the stakeholders.

muSOAing for 3/2/11 – Are you Vertically Integrated?

March 2, 2011

And you thought this term was long passé well think again. While Ford Motor Co was the epitome of VI, it even owned the rubber plantations that made the tires, manufacturers were only too glad to bury this concept after outsourcing to cheap offshore destinations became all the rage and the term rust belt entered our lexicon.

Fast forward a few decades and VI has be re-habilitated and resurrected by the High-tech industry. Apple Inc has taken the mantle from Ford and believe in controlling all aspects of every product they bring out. One may argue that this is not true VI since the factories that the products are made in are located elsewhere in China but if you think about it, the factories are purpose built for building Apple products and wouldn’t exist otherwise. Oracle is another example with it’s exadata data warehousing appliance. With the acquisition of Sun, it controls all aspects of that product offering.

The reason I started this thread is not to talk about traditional VI but how this applies to you personally and specifically as an Architect or an Enterprise Architect. I feel that, to be an effective Architect you have to be a Vertically Integrated person. What do I exactly mean by that. Watch this space for updates on this.

muSOAing for 2/28/11 – ROI, TCO and all that good stuff

February 28, 2011

Services foster a business first approach. So one should attempt services design from a BPM perspective. Think about the big picture first and then worry about the minituae later. So the first thing to do would be to consider that process that is proving to be the most expensive to maintain for various reasons. It could be that it requires a lot of people to maintain, enhancements take a long time and the resultant feature cannot be reused so all said and done, the TCO for this business function is very high.

Now look at this from a holistic fashion and think along the lines of a SOA based approach to replatform thiis same business process. You basically want to achieve the same function at the end of the day, it is just that you want to eliminate all the inefficiences inherent in the way it is done currently. So if you lay down the yardsticks for the future vision for this process and adopt a SOA based approach then all the rest of the details take care of themselves. A business process can be as complex as something like Order to Cash or something as simple as accepting service requests through the web.