Guess you’ve all seen the sudden explosion of articles about big data recently – you can hardly load a webpage without seeing it mentioned – and a number of questions may have come to mind. Things like

• How can I help my clients take advantage of it?
• How can I leverage it in my business?
• Do I have any of my own?
• Is it really as big as the media says, or is it all hype?

I’m going to look at each of these questions – and probably some others – over the next few weeks, and share what I believe to be useful and how data can be leveraged for you and your clients. I say data in general because from a business perspective it doesn’t really matter if your data is big or small – it’s part of a total approach.

Let’s look at the last question first – is big data really a big deal? To start to answer this question, let’s look at understanding some things about data generally.

Think of data like water. Water is a resource that can be scarce or abundant and historically, we didn’t know what it was made of. We didn’t know its chemical makeup, what properties it held, how pure or impure it was. We only knew it was water. When it was scarce, we moved in deserts from oasis to oasis, getting small amounts out of the ground with great difficulty. In abundance, we crossed oceans made of it, not seeing into its depths and understanding what secrets it held. In essence we held neither a micro nor a macro view – we were at the same level.

Nowadays, we understand water differently. We know what it’s made of chemically and we know that it may contain various different substances while still maintaining the same appearance. We have better ways of detecting it in environments where it’s scarce and better ways of getting it out of the ground. We no longer only float on top of oceans – we build pictures of what lies underneath and detect currents and patterns across large areas.

At one time, a cup of water only quenched a thirst. Now, it holds the answers to many other questions.

With data – particularly big data – we’ve followed a similar path to a greater understanding. In the past for organizations and for society at large, information could be scarce. We’ve known that events have occurred but we haven’t known all of the details about them. Analysts traipse from system to system, copying information from one Excel sheet to another trying to get a picture of what’s happening. We’ve floated on top of information that is unknown, un-described, or both; a bit like old-time mariners, sometimes feeling there’s a storm coming with no way to see the currents that really tell the story, not seeing the shark until it attacks.

Now, information generally has become more abundant and available to both individuals and organizations. We can identify and describe each element of a particular event and save that information for later. We have better ways of discovering and aggregating data, saving the time and effort involved in pulling information together into one place. Now we can choose to float on top of the data, look at its currents and patterns from different perspectives, or we can look at the data itself and understand what parts are needed to answer a particular question and what is not.

So is big data a big deal? Fundamentally yes, but this new abundance has its own challenges. Data now comes out of a fire hose and we have to figure out not just how to sip from it, but how to siphon only what we need. How do we know what we need? This is the next question I’ll be looking into.