How to architect your Machine learning based IoT system?
The adoption of IoT has not been as successful as promised 5–10 years ago. There are many reasons for the unsuccessful adoption of IoT and the article Predictive analytics to the rescue of IoT list many of the reasons.
The key problem is just gathering big data is not enough: You can put a beacon in the shop, gather some information on when someone comes near the shop but that does not go far enough. This information has not much value to the retailers unless you can derive interesting insights about the customer and potentially predict customer’s future buying behavior.
Predictive analytics using Machine learning algorithms can achieve that. It will not only tell someone what the past data was but has valuable insights for future. This will have phenomenal value for businesses and end-users.
Figure 1 outlines a simple IoT architecture with predictive analytics.
Although the above architecture looks simple, there are a lot of complexity. For the sake of explanation in this article, we will take the use case of vehicle telematics. In this use case, driver and driving data are collected to make predictions on the risk of future accidents.
Data diversity and Databases
In, Vehicle telematics, and in most IoT use cases, two kinds of data are collected:
a. User and domain structured (and relational) data. Forex, Data about user details — name, age, address, car details. Typically such data is stored in a relational database.
b. IoT generated semi-structured or unstructured data. Forex, time-series data about where and when a driver was driving. At what points he was over speeding or accelerating or braking hard. Since this is a big data, typically these data are stored in a NoSQL database like Mongo, Dynamo or Casandra.
Top 3 Most Popular Ai Articles:
1. TensorFlow Object Detection API tutorial
Now if I want to make a simple query to the system like:
‘Give the list of all drivers with age less than 35 and who were over speeding in road X’
To find the output, the system has to merge the user data (stored in a relational database) and speeding data (stored in NoSQL database).
So one has to create an application logic on top of the two databases to split the query and merge the query outputs.
The above complicated the architecture. There are ways to solve the problem by using databases like Timescale (SQL time-series database).
Data volume and speed
The Internet of Things generates a massive amount of data compared to human-generated data such as Twitter and Google. Forex, a connected car generates 25GB of data per hour. The generated data must be ingested, evaluated, and used to trigger various actions. Besides the data volume, the speeds at which data is generated is massive too. In the vehicle telematics case, we were generating data from the car every millisecond. The raw data was first stored using simple storage service like S3 for historical context. A queue service was used to make sure we do not lose the data coming from the IoT device. Once stored, we were processing the data and storing the relevant data in the Dynamo DB.
The above architecture works decently well and I am not aware of any better alternative.
Data processing (analysis vs transactional)
The next challenge is data processing. Depending on the use case, one may want to use a Data warehouse (OLAP) or Databases (OLAP).
A data warehouse is an analytic platform to do sophisticated analytics. In our use case, we did not use OLAP as our needs were fulfilled with OLTP. We needed fast query response and our queries were relatively simple. But if you need to do complex queries which do not need real-time processing and data reside on various databases, OLAP can be a good choice.
Machine Learning algorithms
The final piece is which Machine Learning algorithm to use. Below we will look at four possible learning algorithms, briefly explain how they work and when to use them.
Classification or Clustering using Neural Network
Classification (and Clustering) can be defined as the grouping of things by shared features, characteristics and qualities. [Reference]
In our case, once we have the data about driver’s driving pattern we can cluster using unsupervised learning into good or bad drivers. Drivers with similar features were clustered together and we can evaluate good/bad drivers by checking the individual clusters.
But due to the absence of past labelled data, we are not able to use Supervised learning in this case.
On the other hand, if you have labelled data then Classification can be used. Forex, while identifying a good or a bad sales prospect, classification can be used as explained in this article.
word2vec
word2vec is a neural network with one hidden layer using semi-supervised learning. Originally it was trained to predict the next word in a given sentence but it showed some other interesting behaviors.
Each input to the network can be represented into a word embedding (picture below).
When the word embeddings were plotted into a 2D vector space, it was found that the 2D space preserved the semantic relations between the words. The figure below shows how the 2D space looks when sentences with capitals and countries are trained using word2vec and then the words are plotted using Principal component analysis of the word embeddings.
Similarly, word2vec can be used to train on other use cases also and it will work specifically where there are repeated patterns. Forex, word2vec can be trained on the usage patters of the different electric appliances. So the 2D vector space will store the semantic relations between the different appliances. It will know if appliance 1 was used then appliance 2 will also be used. And this can help us to predict electricity consumption.
Long Short-Term Memory (LSTM) algorithm
LSTM is able to connect previous information to the present task. These networks are suitable if you want to remember past historical activities (often over longer period of time) and predict future. Here is an article explaining how LSTM can be used to train sales processes.
A use case for using LSTM can be where weather data is combined with soil condition data to predict harvest yield.
Reinforcement learning algorithm
Reinforcement learning depicts a human way of learning. It is a learning based on real-time feedback and not via training data.
The learning algorithm learns best actions based on rewards and punishments it receives after executing an action in real world. This algorithm can be used in cases where there are not many past data to use and one is able to analyze the reaction from the current action. Thus this algorithm is best suited in a closed environments. A detailed explanation of reinforcement learning is beyond the scope of this article and I will write a longer article on it.
I hope this article was helpful to understand how to build an IoT system with Machine learning and what are the challenges of building such a system. Feel free to share or like if you enjoyed reading it. Also looking forward to your comments.
