With the advent of IoT (Internet-of-Things), sensor-based applications are no longer confined to the factory floor, but sensor-enabled systems are all around us: in cars, appliances, smartphones, buildings, cities etc. From a back-end perspective, IoT systems are no longer based on closed, fit-for-purpose architectures as their M2M predecessors, but are able to interact with other applications and open to future innovations.
Most of us will acknowledge that great applications can be created by combining sensor data streams with other information available in either company internal IT systems or API information available on the Internet. Integration with IT line-of-business applications such as supply chain or support systems allows embedding IoT in business processes to create true business value. Combining IoT with online services extends the idea of an API-programmable Internet to a programmable world where automation of trivial tasks saves us time, money or creates new value. A couple of examples to clarify what i mean: Get notified in real-time with traffic information customized for your current location; get a coupon on your smartphone based on your location, time-of-day and your customer profile; adjust the building climate taking into account the outside temperature and weather forecast. What all of these examples have in common is that they combine information available in API-format on the Internet with information coming from real devices in order to enrich the application and to contextualize it. In the rest of this post, i want to zoom in on one particular technical challenge when preparing for such the programmable world.
In the Internet and broader in the IT world, most data retrieval models are query-based, i.e. whenever you need a piece of information, you query an API to retrieve that piece of information. For example, to know whether a server is up, i can periodically ping it. To know the current weather in my city, i can query an API service such as OpenWeathermapAPI. When i want to know whether there are still spare parts of a particular machine component, i can query the API of an inventory system.
Contrary, IoT is the world where sensor data are produced, and rather than waiting for you to fetch the data, these sensor data gets pushed to you. MQTT, the popular IoT messaging protocol, is based on a publish-subscribe mechanism where you subscribe to events or data streams from a particular source. Critical in IoT is the ability to process data in real-time as they come in, i.e.the ability to act on data in motion. With periodic polling, some scenarios will just break down: if I arrive at the office (an event based on my location), i want to lights to switched on right away, not with a 15 minute delay. I guess you get what i mean.
Much of the existing technology is either
optimized to work query-based or to determine patterns or correlate event
streams. In other words, they are optimized at solving a subset of the
problem space. So here is the vision, in order to bring IT and IoT closer
together, we need technology that effortlessly can blend event-based and
query based data in real-time, not one or the other, hiding underlying
technical complexity from users and extending the idea of a programmable
Internet to a real-time programmable
What do you think?