Archive for April, 2012

muSOAing for 4/28/12 – Wazzup Big Data?

April 29, 2012

A very frequent question I keep getting asked is,  what are the different ways in which I can use Hadoop.    I always tell them, the sky is the limit.  Ok, jokes aside,  what can you not  do with Hadoop.    After you get thru the initial challenges of ETL,  data formats,  normalization etc.   the fun really begins.  Now you think, what sort of insights do I want into my data,  is it related to clustering, classification or is it related to stuff like real-time analytics,   information that is targeted to specific users.    Whatever the application may be,  there are tools ready to serve your needs.    I have seen of late tools like Mahout, R and SAS being use increasingly to address these needs.   Complex algorithms apart from k-means,  bayesian are ideal for an HPC environment like Hadoop.

Advertisements

muSOAing for 4/7/12 – Hadoop cures depression

April 8, 2012

Well, the title of this post may sound and seem a bit dramatic but believe me,  it is quite true.    Why this is so is because it lets me do all my work in the very eclectic and democratic world of Linux (if you know what I mean).   Not only is the technology so cool,  a perfect marriage between powerful parallel processing software coupled with powerful server grade hardware,  it lets your creativity take wing with all the other great tools and technologies in the ever growing Hadoop ecosystem.

This ecosystem is being sustained and expanded by a great bunch of individuals with whom it has been my honor and privilege to work with.    Take for instance this case when I was really stuck trying to resolve this quirky network issue when setting up my cluster and what do you know,  this great dude and my close friend and colleague Sujee Maniyam had just the solution I was looking for and not only that he had published this piece on github  (http://sujee.net/tech/articles/hadoop/hadoop-dns/).    This is what makes this whole space so fun and exciting to be in, it almost seems that you can have your cake and eat it too and last but not the least,  I now get to work only on my beloved macbook pro.