Derek Bishop


Finding the needle in the data haystack

Date added: 24th Aug 2017
Category: Innovation Culture

Data Collaboration: helping you to distinguish between the need-to-know, the nice to know and the plain unimportant.

Loyalty cards…internet data…footfall…. It doesn’t matter which way you look at it, the mantra for daily life could easily be ‘if it moves, measure it.’ And there is so much to measure!  According to research by Waterford Technologies in February 2017 [1] the total amount of data in the world now is equivalent to every person in the USA tweeting three times a minute for 26,976 years and the volume of business data is doubling every 1.2 years.

Whatever way you look at it that’s a lot of data.  And that’s great isn’t it?  After all the more you know, the more you can deliver innovative products and services which really make a difference for customers.

Or can you!

When there is just so much data around, how do you distinguish between the need-to-know, the nice to know and the plain unimportant? How do you find that needle in the data haystack which will point your business towards innovation success rather than being misled by shiny bits of straw, stone chips and other distractions? That’s where data collaboration comes into play.

Quite simply, data collaboration is a way of drawing data in from various sources, and then organising and delivering it in a way which will inform rather than mask. Now I know what you’re thinking; if we’ve already got too much data why would we want to draw in more from different sources. Well the problem is that one single source may not provide you with the rounded picture that you need.

Let me give you an example, you are walking through a wood and you notice leaves dropping from certain trees and you wonder why. You might start by seeing how big the problem is, counting species of trees and the numbers of each affected. That will give you lots of data about the size the problem but it won’t necessarily take you any further. So then maybe you go online, researching weather patterns to see if there have been any unusual cold or dry spells which may affect the trees. You may also instigate a research study into the use of the woodland by people, birds and animals, seeing if usage patterns have altered.

Every additional bit of information adds to the picture.  Sometimes it will swiftly become apparent that the data you collect isn’t really adding to your understanding, at other times the information you gather will help you to move one step closer to the solution; particularly if you open up access to others, asking them to bring their specialist knowledge to the problem. And that’s data collaboration in a nutshell; building a picture by collaborating and sharing data from multiple sources.

Of course data collaboration does bring its own problems. When you are combining data from multiple sources you have to be very clear about origin and taxonomy if you are to avoid misunderstandings and incorrect conclusions. This is particularly important when you collaborate outside your own organisation; your jargon may not be universally understood and your internal organisational culture may put different weights and emphasis on certain approaches. Much of this potential confusion can be overcome if you follow the basic principles of collaboration as set out in ISO 44001; in particular choosing your collaborative partners carefully, understanding differences and building common platforms of agreement.

Data collaboration can be an immensely powerful tool, helping you to build an appreciation and understanding which will act as a platform for developing products and services which really meet customer needs. When you have true insight you are well on your way to building a Next Generation culture which will drive innovation and future success.



Leave a Reply

Your email address will not be published. Required fields are marked *