Thursday, January 2, 2014

Big Data & Artful Intelligence

Catch me if you can
Yes, do cast a wide net. 
Data is everywhere ... all around you - can you catch 'em - but do focus on quality rather than quantity. 
You have heard this before: We begin with the three V’s: Volume, Velocity & Variety 
(that's the do-re-mi of Big Data)

Large Volumes of Structured data
It’s the usual & the-not-very-exciting-stuff that lives in your internal databases; also in the public clouds. Always useful to have some outside data; gives you the ability to see how you compare with the outside trends.

To this we added the vast Variety of exciting Un-structured data. 
       Go ahead and happily include social perceptions
       Again it is critical to identify the ones that really matter to your business.        
      I can almost imagine some interesting ones. 

             Seriously, was this tweeted?
 Really, what was posted regarding this?
 How does Gen Y perceive this?
That status is so not cool!
What was pinned?
Yes, that too – the video that went viral?

Of course, with a high Velocity, you are almost capturing real-time data as well.
The in-memory columnar format – even better! Who wants a data-warehouse when you can have it alongside your transaction data.

Part or Whole
Do you want to start with a Holistic set of data to get the big picture for your business?
Do you want to start with a small set of Deeper, Segmented  data (think: specialized industries here)?

Analyze this
What story does this data tell you?
Connect the dots and analyze from the business perspective.
Apply algorithms to derive intelligence and insights.

Missing Pieces
Remember though to have a good data representation. Could there be any missing data that will affect the inference? What about the boundaries? Can't provide a skewed perspective, now can we?

Lay it out …
Time to provide some visually compelling and easy to understand insights for decision making.

Fast and Furious
To this ever-changing data apply an evolving logic - logic that progressively learns from its experiences and fine-tunes in an ongoing manner.
You got it ... the logic applied should simply not be hard-coded ... it's smart enough to learn from itself... and does not require any manual logic upgrades. How do we manage that? Bingo!
Usher in - Innovative Cognitive Computing!
(… so la ti do …)