Share

This is the third part of a three-part blog series. You can read the first part here and the second part here.

Introduction

In this final part of the series, I will be using the real-life example of an elderly care field trial, in order to explain how we combine different AI techniques, in our case using the Waylay platform. The DIoTTO project was developed by three partner companies (Studio DottSensolus and Waylay) and was funded through the Flanders Care Living Labs and supported by the VOKA Health Community. Studio Dott prototyped the app and was leading the overall concept, Sensolus provided sensors for the project and Waylay worked on data mining and rules implementations.

The central goal of the project was to provide a solution that would re-assure caretakers and family that the person in their care was alright. The diagram below explains the concept and presents all involved actors.

There are a number of reasons why this use case comes with difficult challenges:

We often say, data speaks for itself, so let’s first see what sort of data we got during one of the trials. We placed sensors on fridges, front doors, cabinets, curtains and in one case even on the person’s wheelchair. Next to using sensor data, what made this trial different was that seniors, caretakers and family members were able to use the app through which they could share pictures and send messages. Here are histograms of a few typical days, form different houses:

I will not go into much technical detail here on how we processed the data (as this post’s subject is more about combining different ML/AI techniques in automation rules), but just by looking into these histograms we can see what sort of information we can deduct:

 

Rules based on input from seniors and caretakers

As mentioned earlier, some of the rules that we want to apply should be expressed without the need for any “learning”. During the project, Studio Dott came with a great way to capture these rules:

  1. Select an object (fridge, curtain, closet, door etc.)
  2. Select “when” condition (morning, evening etc)
  3. If the object moves or not in the time window, over weekday or weekend
  4. More than or less than X times
  5. Send SMS and/or email to a person if conditions met

Finding whether seniors have developed a new sleeping disorder

In this rules template below, we combine motion sensors, a likelihood sensor, a day/night sensor and an hourly sensor to deduct whether the person has a sleeping problem. Note that we are taking into account the movement profile of the person, so even if they wake up in the night, and they do it often, we will not label that night as being with a sleeping problem. This way, we are only looking into changes of the pattern.

Appliances not switched off after a period of time

If the person forgot to switch off the appliance, we can send the alarm after a period of time (with different periods based on the type of appliance):

In this example, based on the sensor input and type of the object (a TV for example), Waylay sends the alarm in case the appliance was switched on and not switched off again (the exact window can be controlled by the delay sensor). We also discard if multiple on/off events have been registered within the same time window (to avoid on-off-on type of situations ).

Modelling Markov Chains

A Markov chain is a random process that undergoes transitions from one state to another.

Let’s now assume that we have a house with four rooms and the connection between the rooms is shown in the picture above. The total movement probability in every room should be equal to 1 (labelled by the same color). One of the problems modelling this way is that the transition table can change during a day. (Note that this is not the same as the “memorylessness” feature of Markov chains, since the overall process might not only depend on the previous state).

Now, I often say that engineering is the art of applying scientific hacks, so before we get to paralysis/analysis mode on how to solve this problem, let’s see how else we can simplify the model without loosing any valuable information.

For that, we use simple event correlation (how often events come close to each other) and the eviction policy feature of Waylay, together with an AND gate.

In this animation, we can see how we capture a pattern where person moves from bedroom, takes stairs, pass the living room and opens the fridge. In the same rule, we send a SMS if the fridge is not open at least once before 11.00 AM.

Putting everything together – combining AI and IoT

With a platform that allows seamless blending of different AI and ML techniques, you can use information coming from:

 

Want to know more? Watch our TSA webinar here.

You might also like these articles

IoT Insights

Put your IoT data to better operational use: 5 Key learnings from equipment manufacturers

Product News

Succeeding with remote operations in IIoT

Rule Engines for IoT

How to automate visual quality inspection, with Waylay ML ops