What’s the big deal with big data and IoT?

Everyone is talking about big data. Maybe it has you thinking, “What is big data and why is it such a big deal?” Big data is not just a buzzword — it is so much more. Big data and IoT are transforming business, and some might even say it’s changing the world.

In business, big data from connected trains, connected farms, and even connected breweries, is used to increase efficiency and improve the customer experience.

What Is IoT Data?

IoT data is data collected from Internet of Things (IoT) technology such as sensors, wearables and connected hardware. From this vast network of internet-connected devices and sensors around the world, companies can take many measurements, track activities, and monitor trends — think of a smartphone, fitness watch or connected car.

How Is IoT Big Data Stored?

A corner store that’s logging a couple hundred transactions a day is obviously going to have different ways of dealing with their data, compared to an larger enterprise that collects millions of rows of data each day from devices around the world.

Industries with large data collection systems may take an approach like archiving, backing up, and/or replicating databases in order to preserve the data without slowing down production systems. These backups or archives aren’t always easily accessible. When they are accessible, it can be painstakingly slow to run analysis on the database(s) due to the large quantity of data.

Any company that stores information uses some form of database system. Database systems like Microsoft SQL Server, PostgreSQL, Oracle, and MySQL are some examples. These applications allow users to capture, store and analyze data with relative ease. In cases where companies want to capture and analyze data in real-time on a massive scale, data warehouses that are specifically tuned for such tasks come into the picture. Examples of this are Google BigQuery and Amazon Athena.

Top Benefits of Big Data

Big data can help businesses with:

• Future Planning — through predictive and prescriptive insights
• Customer Service — by providing a clearer view into customer satisfaction and requirements
• Risk Management
• Cost Savings Opportunities

Big data systems are able to work with such large sets of data quickly and efficiently due to the advanced architecture and processing methods designed for these systems. An example of this is a method that was developed by Google, called MapReduce. 

The way it works is that when queries are run on the large data sets, the queries are split and distributed across parallel nodes and are processed in parallel as well. The results from the parallel actions are then gathered and delivered to the end user.

To return to our question above, companies can use big data systems to not only provide storage of historical datasets, but also to give them the ability to run complex data analysis on their historical data. There are many different options to choose from when it comes to big data systems, from self-hosted to cloud based, and Google to Microsoft

IoT Big Data Applications: Telematics

A good example of the power of big data and IoT comes from the fleet management industry. Many fleets support management with the use of telematics or vehicle tracking devices, which act like an IoT hub for the vehicle when connected with third-party devices and sensors, transmitting engine data, location, trips history, driving behavior data, temperature, tire pressure, and more.

Vehicle and driving data is fundamental to businesses or government agencies that want to measure their operations. From there, organizations can make important decisions that improvement.

How fleets use telematics data:

• Evaluate drivers
• Increase fuel efficiency and reduce CO2 emissions
• Manage driving habits such as speeding or idling
• Comply with regulations for food safety or electronic logging
• Benchmark performance

Insight on Driving and Risk

Telematics collects, stores and sends information between end users and vehicles through telecommunication devices. Big data extends the value of that data.

The platform of data mining and analyzing makes the monitoring of vehicles, planning of optimized routes, online or offline driver assistance, and other supporting telematics-related industries possible. Big data delivers many services to the telematics world. Routing optimization, breakdown alerts, maintenance reminders, and driving behavior analysis can all be be provided with the help of big data.

There are numerous benefits of analyzing driving behavior with big data. A long list of driving behaviors can be noticed and analyzed through designed intelligent algorithms, such as speeding, stopping, and turning. By monitoring these activities, systems can be put in place to coach and instruct drivers on better driving habits and fleet safety like an in-vehicle verbal coaching tool (more here).

Big data analysis on driving behaviors also contributes to fleet risk management. Building accurate risk models based on driving behavior can provide a precise assessment for the whole fleet.

IoT Big Data Challenges

Working with big data brings its own challenges, including gathering the data, storing and processing it, and even deciding how to put the information to use.

Due to the fact that IoT is new to security professionals, IT personnel may lack the experience to handle IoT security threats. A 2017 report from Thales and 451 Research found that 63% of senior security executives surveyed believed their organizations didn’t have the right security to properly protect their data. In the same study, although 85% of executives confirmed they were using IoT technology, only one-third of them expressed they were “very concerned” about that data.

Open data adds another layer to the issue. Before releasing data for public use, businesses must take extra care in planning and establishing security. Read more in Geotab's white paper on open data privacy and security.

If you want to know how Geotab ensures security, please visit our security page.

Geotab’s Big Data Environment

There are billions of data records that pass through Geotab’s servers everyday, including GPS data, engine data, accelerometer data, and more. Geotab has taken that leap forward and is diving into big data with some big ideas.

Big data has become so fundamental to Geotab as a company, that each team in the company has a person training in using big data tools.

Imagine selecting a route in MyGeotab to display on the map and being able to see the traffic flow rates for certain times during the day for that particular route. Or, what about being able to see an overlay on the map of cell network coverage based on the type of devices that are in your fleet. This way if one of your devices hasn’t updated recently you can quickly see on the map that they have entered an area with poor cellular coverage. The advantage of this coverage map is that it can be built from Geotab’s device logs so that the coverage maps are specific to Geotab devices and not generic maps provided by the cellular providers.

IoT and Big Data: Connecting the Dots

Statista estimates there will be 31 billion IoT connected devices in the year 2020. This number represents a massive potential for what we can do with this data (both for business and in everyday life).