What’s the big deal with big data and IoT?

Everyone is talking about big data. Maybe it has you thinking, “What is big data and why is it such a big deal?” Big data is not just a buzzword — it is so much more. Big data and IoT are transforming business, and some might even say it’s changing the world.

Big data from connected trains, connected farms, and even connected bars is being used in business to increase efficiency and improve the customer experience.

What Is IoT Data?

IoT data is data collected from Internet of Things (IoT) technology such as sensors, wearables and connected hardware. From this vast network of internet-connected devices and sensors around the world, companies can take many measurements, track activities, and monitor trends — think of a smartphone or fitness watch.

How Is IoT Big Data Stored?

A corner store that’s logging a couple hundred transactions a day is obviously going to have different ways of dealing with their data, compared to a larger enterprise that collects millions of rows of data each day from devices around the world.

Industries with large data collection systems may take an approach like archiving, backing up, and/or replicating databases in order to preserve the data without slowing down production systems. These backups or archives aren’t always easily accessible. When they are accessible, it can be painstakingly slow to run analysis on the database(s) due to the large quantity of data.

Any company that stores information uses some form of database system. Database systems like Microsoft SQL Server, PostgreSQL, Oracle, and MySQL are some examples. These applications allow users to capture, store and analyse data with relative ease. In cases where companies want to capture and analyse data in real-time on a massive scale, data warehouses that are specifically tuned for such tasks come into the picture. Examples of this are Google BigQuery and Amazon Athena.

Top Benefits of Big Data

Big data can help businesses with:

• Future Planning — through predictive and prescriptive insights
• Customer Service — by providing a clearer view into customer satisfaction and requirements
• Risk Management
• Cost Savings Opportunities

Big data systems are able to work with such large sets of data quickly and efficiently due to the advanced architecture and processing methods designed for these systems. An example of this is a method that was developed by Google, called MapReduce. 

The way it works is that when queries are run on the large data sets, the queries are split and distributed across parallel nodes and are processed in parallel as well. The results from the parallel actions are then gathered and delivered to the end user.

To return to our question above, companies can use big data systems to not only provide storage of historical datasets, but also to give them the ability to run complex data analysis on their historical data. There are many different options to choose from when it comes to big data systems, from self-hosted to cloud based, and Google to Microsoft.

See also: Big data use cases for telematics

IoT Big Data Applications: Telematics

A good example of the power of big data and IoT comes from the fleet management industry. Many fleets support management with the use of telematics or vehicle tracking devices, which act like an IoT hub for the vehicle when connected with third-party devices and sensors, transmitting engine data, location, trips history, driving behaviour data, temperature, tyre pressure, and more.

Vehicle and driving data is fundamental to businesses or government agencies that want to measure their operations. From there, organisations can make important decisions for improvement.

How fleets use telematics data:

• Evaluate drivers
• Increase fuel efficiency and reduce CO2 emissions
• Manage driving habits such as speeding or idling
• Comply with regulations for food safety or electronic logging
• Benchmark performance

Insight on Driving and Risk

Telematics collects, stores and sends information between end users and vehicles through telecommunication devices. Big data extends the value of that data.

The platform of data mining and analysing makes the monitoring of vehicles, planning of optimised routes, online or offline driver assistance, and other supporting telematics-related industries possible. Big data delivers many services to the telematics world. Routing optimisation, breakdown alerts, maintenance reminders, and driving behaviour analysis can all be provided with the help of big data.

There are numerous benefits of analysing driving behaviour with big data. A long list of driving behaviours can be noticed and analysed through designed intelligent algorithms, such as speeding, stopping, and turning. By monitoring these activities, systems can be put in place to coach and instruct drivers on better driving habits and fleet safety like an in-vehicle verbal coaching tool (more here).

Big data analysis on driving behaviours also contributes to fleet risk management. Building accurate risk models based on driving behaviour can provide a precise assessment for the whole fleet.

IoT Big Data Challenges

Working with big data brings its own challenges, including gathering the data, storing and processing it, and even deciding how to put the information to use.

Due to the fact that IoT is new to security professionals, IT personnel may lack the experience to handle IoT security threats. A 2017 report from Thales and 451 Research found that 63% of senior security executives surveyed believed their organisations didn’t have the right security to properly protect their data. In the same study, although 85% of executives confirmed they were using IoT technology, only one-third of them expressed they were “very concerned” about that data.

Open data adds another layer to the issue. Before releasing data for public use, businesses must take extra care in planning and establishing security.

If you want to know how Geotab ensures security, go here.

Geotab’s Big Data Environment

There are billions of data records that pass through Geotab’s servers every day, including GPS data, engine data, accelerometer data, and more. Geotab has taken that leap forward and is diving into big data with some big ideas.

Big data has become so fundamental to Geotab as a company, that each team in the company has a person training in using big data tools.

Imagine selecting a route in MyGeotab to display on the map and being able to see the traffic flow rates for certain times during the day for that particular route. Or, what about being able to see an overlay on the map of cell network coverage based on the type of devices that are in your fleet. This way if one of your devices hasn’t updated recently you can quickly see on the map that they have entered an area with poor network coverage. The advantage of this coverage map is that it can be built from Geotab’s device logs so that the coverage maps are specific to Geotab devices and not generic maps provided by the cellular providers.

IoT and Big Data: Connecting the Dots

Statista estimates there will be 31 billion IoT connected devices in the year 2020. This number represents a massive potential for what we can do with this data (both for business and in everyday life).

For Geotab, we continue to dig deeper into big data and look for ways to unlock insight. Geotab has built a unique data platform to allow users to gain free insight from fully anonymised datasets called data.geotab.com. Intelligence datasets include weather, urban infrastructure, and location analytics. We also offer a smart city data package with hazardous driving areas, parking search and idling info, cell coverage dark spots, hyper-local temperature and barometric pressure.

Tap into IoT data Intelligence. Learn more by visiting: data.geotab.com

Originally published Apr 25, 2014. Updated May 14, 2018.