Of all the buzzwords that are circulating in the IT, cloud and web hosting arenas, ‘Big Data’ is probably the most prevalent. To date we have heard that Big Data offers the potential to cure cancer, to accurately forecast the weather (despite issues like global warming), and turn ALL of a company’s data into something executives can use to make business decisions. But what exactly is Big Data and how can it deliver on this promise?Who is using Big Data and what for?
Despite that fact that many companies make reference to Big Data as a means of appearing ‘cutting edge’, few are genuinely utilizing it. As the name suggests, Big Data refers to the processing of massive amounts of data. If you work in an organization of any size, what you might think of data is information like financial records for the previous year and possibly forecasts for the following year. In addition, there might be the data you collect on clients – names, telephone numbers, addresses, etc. But actually, the data your company generates is deeper than this.
What’s the idea behind Big data?
Let’s take for instance, email. Companies often email surveys to customers and the information these surveys reveal can be processed to reveal actionable data – 10% of respondents want this model; 20% want this model, etc. However, the email your company receives on a daily basis is not thought of as being data, because it is not in a raw data format. If you think about it though, every email does contain data.
The average web hosting company might receive hundreds of thousands of emails a year. Perhaps a very small percentage of these emails are making a very general enquiry about how to make a backup to a server – something very minor. While it is possible to make actionable decisions on the actions of your support staff after enquiries have been received (i.e. 0.3% of staff dealt with enquiries on how to back up a server), if there was a means of understanding the emails themselves, then the trend would appear without further analysis.
That’s where Big Data comes in.
Theoretically, if a system could understand the content of the emails a web hosting company receives, it could establish that 0.3% of emails received make reference to this minor point on how to back up a server. This information would allow a company to immediately add backup details to their website, and this would reduce the support staff’s workload. But if it could establish a pattern like this, then surely other patterns would emerge, and all would be equally actionable. As a result, the system would give companies answers to issues within their companies they weren’t even aware of.
If the system could do this with emails, imagine what it could do with telephone calls. What about analysing images? Big Data can establish patterns and trends that it would take humans years to establish, if indeed they asked themselves the right questions to want to be able to investigate a specific area in the first place.
Why is Big Data such a big thing right now?
If you consider the business landscape, you can understand why Big Data has come to the forefront. Emails, websites, mobile phone calls, Facebook messages, Tweets – this is after all the Information Age and more information is being generated than ever before. Big Data is an attempt to comprehend all the data the modern era generates and make it understandable, meaningful, and usable.
What can I do with Big Data right now?
Big Data helps with diagnostic analysis that aims to establish cause and effect relationships, but it also enables us to predict things more accurately by finding relationships between elements we did not know where connected. Big Data also helps us establish strategies to manage future events and situations that might adversely impact our businesses.
Why is Big Data important to a company?
Data is the lifeblood of any company, and the companies that use data in the most efficient way have the edge. Big Data offers companies the opportunity to thrive by solving problems before they even exist. But understanding such huge amounts of data is often impossible, even when it is processed. That is why Big Data is joined at the hip with ‘Virtualization’ – the process of putting data into a visual form – very much like Microsoft Excel creates graphs and pie charts, but on a much, much bigger scale. Unless Big Data represents huge amounts of data in a fashion a member of a company can understand, it is essentially useless.
What will Big Data offer us in the future?
Well, it looks like Big Data will merge with Artificial Intelligence (AI). IBM has a world famous resource known as Watson. Watson is renowned for taking part in the US television program ‘Jeopardy’ – recognizing human speech, Watson was able to beat its human contestants in a general knowledge quiz. The same AI infrastructure is being used to support IBM’s Big Data efforts to forecast weather patterns (the company recently bought ‘The Weather Channel’ incidentally). Imagine the savings if companies could accurately foresee the impact the type of blizzard that hit the Mid-Atlantic and Northeast United States in January 2016. Google on the other hand is looking at using AI and Big Data to underpin its search – imagine asking Google Search a question and getting the exact information you need. But that’s the future, what about now.
How can I learn more about Big Data?
Big Data sounds daunting, but then again, so was learning to drive. There are masses of text and video tutorials available on the Internet that you can take a look at to give you some insight into what Big Data can do and how to use it. YouTube is an obvious resource – this video for instance offers the basics on Hadoop – a Big Data tool. Obviously the big cloud players have their own tutorials. Google has a solid starters guide, while IBM has its own Big Data university. Microsoft has its own video academy that covers Big Data, while SAP provides significant insights. Training sites like Coursera offer Big Data training at cost - there’s a lot of choice available and the cost represents a good investment.
There’s still time to make another new year’s resolution – this time you should promise yourself that you are going to get to grips with Big Data and see how it can benefit YOUR company in 2016.
