muSOAing for 8/19/10 – Troposphere

The whole Big Data space has become a very cool one to be in. It deals with all the aspects of data right from mining it, moving it to your big data platform, managing it and then mining it and all of it is done very differently from your traditional way. A whole other ecosystem has evolved around this concept of storing and accessing big data sets fueled primarily by algorithm and heuristic driven paradigms like Map/Reduce, platforms like Hadoop and tools like Pig, Hive and similar implementations like AsterData, Karmasphere etc.

It is a happy amalgam and confluence of paradigms like MPP, Grid Computing, Algorithms like Map/Reduce, Java, SQL, UNIX like text processing, Inter Process Communication, Data Structures and all the other cool stuff some of which you dealt with in your early college years on a Green Phosper UNIX terminal. The days and nights you spent studying sed, awk, grep, pipes, shared memory, linked lists, semaphores, threads, processes, regular expressions. UNIX was and still is the programmer’s nirvana compared to the dull intellectual wasteland of windoze.

All of it has converged in this utopian landscape of Big Data only pumped up several levels up with addition of Java, commodity servers, innovation, hungry, nerdy, smart and UNIX fueled programmers, venture capital, california wine, sunshine, pale ale, mountain air, sunny beaches…..

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: