PI2-SecurHome Deliverable - Report 2 Individual Project Follow-Up 2

The main objective of this document is to present and describe all the procedures carried out by the research team of the individual project 2 of the coordinated SecurHome programme within the framework of the development of the PI2-SecurHome ecosystem ("Mobile application in charge of domestic interaction by means of television"). Special attention is paid to the description of the structuring of this ecosystem, highlighting the server-side processing and database management components. 

The PI2-SecurHome ecosystem, supported by the iNeighbour TV system, is materialized in the following set of client applications, complemented with an API to access the respective data by the system associated to the individual project. 

Making available to senior users for recording television interactions and viewing reminders (medicines and consultations).

Making caregivers available to users for the management of their dependents and the issuance of medical reminders.

Providing caregivers for users to follow up on dependents under their care and receive alerts.

From the technological point of view, a code-based development scheme was used, compatible with all applications and allowing the creation of a development environment. 
The system is integrated, capable of responding effectively to these needs. 

For the implementation of the TV application, commercial STBs of the MEO commercial IPTV system were used. The fact that these STBs are equipped with Mediaroom middleware means that the server-side code development was done in C# while the Admin application (backoffice of the whole system) was developed using the framework.NET. The TV application was developed with the Presentation Framework. 

The project was complemented with an SQL database in which the data related to all the applications of the project are stored. 

2.1.2 Specific application areas 

Login 

When the application is launched, the user is prompted to choose one of the available users at home and enter that user's PIN. If the PIN is correct, the user logs into the application and switches to the Menu. Otherwise, the user receives information that the PIN will be incorrect and returns to the user choice dialog. Up to this point, the application is controlled through Default.aspx and Access.aspx. The Default.aspx file displays the welcome screen and redirects the application to the Access.aspx page where the user can log in. 

Menu 

After logging in, the SecurHomeTV menu appears at the top of the TV, while the TV broadcast remains available. By selecting each of the menu options (apart from the menu option to the "Off" option), the different submenus of the chosen area will appear and a menu panel will scroll to the next screen up to the lower limit of the TV, with the first submenu information loaded by default. 

Submenu and Detail 

The content of each submenu varies depending on the submenu, although in several areas the structure and operation is similar. The detail of each chosen option can arise through a slide animation of the panel that reveals the information by pressing a button at the top of the panel that allows you to return to the previous area or, alternatively, through a pop-up window with a reduced transparency to allow you to focus your attention on it and not on television content. 

The selection of options that appears is controlled through the EventsTopLayer.aspx page. This page has been designed to work independently of the rest of the application. In redirection 
for this page, which is performed through a SubmitAction, two parameters must be specified: the id parameter of the button that triggers the pop-up action and a keyword used on the EventsTopLayer.aspx.cs page to identify which options to display in the pop-up window. On the same page is created a virtual xml that will save the options to display in the popup, as well as the actions to perform if an option is chosen.

The xml will serve as a bridge between the information to be made available and the TvPhysicsControl network that displays the information on the TV. Depending on the keyword that is collected in the file at the top of the page, the xml is built with two main fields, text to display on the TV and url for which NavigateAction runs. If you only need to choose one option, then the option displays the message "Option saved" and closes automatically. Otherwise, and through an output parameter determines that it should not close and with the new keyword knows which options to display. The process is repeated until all options are chosen. To smooth the appearance of the pop-up window, two of the framework's own animations are used: FadeAnimation and ZoomAnimation. At any time the user can close the pop-up window without selecting an option using the Back button on the remote control, such as the one referred to in the message below the pop-up window. There are also a number of validations that depend on each option. In the case of date selection, for example, the current date is always taken into account so that the user cannot choose an earlier date. Days, months, years, hours and minutes are displayed, not in ascending order, but in a circular panel with the initial focus on the current option (today, current month...). In the case of the choice of users to send invitations, as users are chosen, they disappear from the pop-up window, so that the user does not send the same invitation more than once to the same person. 

The treatment of the selected data is carried out by means of JavaScript code that is included in the Mainpage.aspx page through the TvScript control. The communication between the options chosen in the pop-up window and the Mainpage.aspx option is done through SendEventAction. In SendEventAction the data collected in Json is communicated through the options selected in the pop-up window. These are collected by Javascript through its Eval function and converted into objects that will be saved or displayed in the TV directory as required. 

2.2 Admin Application  

The Admin application is, like the rest of the project, hosted in the virtual machine of the AlticeLabs platform. As for the development of the web page, a basic structure has been used that allows the creation of a global design of the application. Therefore, and using the .NET Framework capabilities, two master pages were linked that serve as the body of all the pages of the website. 
The "Master.master" page has the mission to create the global structure of the application while the "Menu.master" page has the mission to create the menu that allows interaction with the users of the site, as well as the widgets that are always present throughout the application. 

After the creation of these two pages the application executes in a logic of pages where the html and the code of the server (C#) necessary for the operation available in each one of the respective areas are found. 

However, the code that is repeated in several areas focused on the file "inWebPage.cs". whose class extends to all kinds of site-specific pages. 

Parallel to this code, the application also uses, like the TV application, the function Interaction Layers iNeigDAL and iNeigBLL through which it communicates with BD and provides all data related to the user. 

2.3 Mobile application 

The mobile application, entitled SecurHome Mobile, is aimed at carers of older people. In this application, the caregiver can access television consumption data and thus monitor senior activity in real time in order to prevent possible accidents in the home. 

Server Routine 

In addition to the manual monitoring of the caregiver, a Console Application was developed to check, during a certain period of time, the activity of the elderly person, crossing it with their daily consumption habits. Based on standard deviations and using specific information from the ITPV application (events and medication schedule), the system issues alerts to caregivers with different degrees of severity. 

These alerts, ranging from a green chromatic scale (without alerts) to a red one (the most serious) are received by email or SMS by the caregiver, who can ignore them, inform the system that there is no problem with the elderly person concerned, or take other precautions to resolve the problem detected by the system. 

2.4 Alert system 

The mode of operation of the alert generation algorithm is sequential and cumulative. All users are analyzed one by one by the system using a set of sequential criteria that can simply interrupt the process or make it go in different directions depending on the data analyzed above. 

A user's analysis process goes through 3 phases. The first, a general analysis is the same for all users; the second depends on the status of the user in the application (online/offline); and the third, which consists in the generation and diffusion of the alert. 

This division seeks to ensure, as will be seen below, the hierarchy in the following criteria to ensure greater optimization of the code and, therefore, a faster analysis of users 

The following table shows the chromatic scale associated with each alert level allowed, therefore, a correct interpretation of the colors and warnings whenever addressed throughout this document:

2.4.1 Phase 1 - Triage 

In this phase, users will be submitted to four steps that will allow them to filter the status of the user in the application. Next, we explain in detail how this process takes place, knowing that initially the assigned alert code is 000. 

Before starting the user interaction analysis process in the IPTV application, the system analyzes the permissions for verification. Therefore, the first step is to know if there is authorization to perform the user analysis. In case of not having permission, the system assigns a code 000|300 if the lack of permission is due to the user being on vacation or away from home; or a code 000|400 if this impediment is due solely to privacy issues. Including this, the system directs all resources to users interested in analyzing their state, immediately eliminating those who do not intend, for one of the reasons mentioned above, to have the system active. This functionality, defined in the future by the caregiver in the mobile application, also has the function of eliminating warnings during periods when routines are interrupted by the absence of the elderly person in their home. If everything is authorised, the system starts the individual analysis. 

The next step is to check if there is an alert in the application, i.e. if this user starts the analysis process with his usual situation or if an alert has already been issued in the application previously. This criterion does not imply any interruption of the process, it means only an initial mapping. If the user already has an alert, the code 001 is assigned, as long as it does not remain with the 000. 

In the next step, the system checks if there is an emergency alert required by the yellow button of the application that has not been read by any of its caregivers. This question, like the previous one, does not end the process abruptly, but serves once again to better frame the user in relation to the analysis. Thus, if the answer is yes, the user is assigned a 002, while if not, it remains with the previous code.

In the next step the system checks if there is an emergency alert required by the yellow button of the application not read by any of its caregivers. This question, like the previous one, does not end the process abruptly, but serves once again to better frame the user in relation to the analysis. Thus, if the answer is "yes", the user is assigned 002 while if not, it remains with the previous code. 

Afterwards, the analysis criteria are the delayed medicines. The system checks the medication history in the last 48 hours at this stage. This 48-hour value is easily configurable and has been adjusted several times to ensure a degree of effectiveness in the process of generating alerts. Bearing in mind that, in the PI2-SecurHome component, it was considered that 48 hours would be a reasonable time. For this reason, if you are late with your medication, your analysis code changes to 003, otherwise the previous code will be retained. In practical terms, having delayed medicines means automatically issuing a Blue alert (signalling). After collecting the generic data from the detection algorithm, the system enters the phase of analyzing the user's status in the application.  Depending on whether the user is connected or disconnected, the process follows different paths according to the differences that each state entails. 

2.4.2 Phase 2 - Connected Users 

If the user is logged in, the system calls a specific method in the algorithm, and the first step is to check if there is a presence notification in the application, that is, if there is a question "Are you still there? If there is no notification, the system proceeds to the next step without modifying the codes you have transited from the previous steps. If it is launched, the average response time to these notifications is verified based on the data collected during the use of the IPTV application. The purpose of this verification is to understand whether it is common for this user to be absent from the tele for long periods of time, which will result in the appearance of such notifications and, ultimately, to respond later. If it is lower, the assigned indicator is 102. In both cases the assessment stops at this point and an alert is issued to the caregiver based on the assigned codes. Without notifications, the analysis continues. 

Knowing that there is no notice posted, the system continues to analyze data to detect if there is a problem with the user. At this stage, the system checks the date and time of the last interaction with the television. To do this, it uses the analysis of various indicators such as the last change of channel, the notification (presence or medication) that has been responded to or the last entry in the TV application through the history of iNeighbour TV interaction record. 

As soon as it has that date, the system subtracts it by the current date, starting with the interval that proceeds to the analysis of this parameter. The first step of this verification is to confirm that the last interaction took place less than twenty minutes ago. The "twenty minutes" parameter is configurable but its adoption was due to the fact that it is half the time needed for the SecurHome TV application to start the presence notification with the question: "Are you still watching TV?", which is encrypted in 40 minutes. 

In case the last interaction was less than twenty minutes ago, the system assumes that everything is in place, as well as the user. However, if the time is longer, a detection check is performed. This evaluation makes it possible to anticipate up to a maximum of twenty minutes the publication of the notification and thus increase the effectiveness of the system in the event of an accident. 

Thus, screening consists of crossing the broadcast program with the user's habits. The system checks whether the program being viewed is customary for the user. The usual concept in this case is refined by crossing the vision patterns. 

At this stage any program whose display is greater than 5% of the user's total consumption is considered a regular program. The 5% value allows, for example, to exclude all programs never seen before, but guarantees the inclusion of programs with a single weekly transmission. If the program is considered normal, a second level code 103 is returned and whenever there is no return, a second level code 104 is performed. 

Once all these verification steps have been completed, the system issues the alert based on the associated first and second level code and sends it, through mechanisms adjacent to the generated level, to the caregiver, thus completing the verification process. 

Once the user has logged on, the algorithm's verification possibilities depend to a large extent on the presence notification and subsequent response, so the verification method for logged on users is considerably simpler and easier to understand than the method for logged off users presented below. 

2.4.3 Phase 2 - Disconnected users 

Like the process activated for connected users, the dedicated mode for disconnected users is called by phase 1 (triage). This is activated by a specific method in the algorithm. When entering this mode, a second level code with the value 200 is automatically added. The analysis is then started based on the criteria defined for disconnected users. 

The first step of the check is to analyze if the cause of the disconnection of the application is to participate in some event or query. To do this, the system takes advantage of the existence of these data on the PI2-SecurHome ecosystem by crossing them. To increase the effectiveness of this verification, the system adds thirty minutes to the hours defined as start and end of queries. If the current time is within this period, the system assumes that the absence of the user is due to participation in a properly scheduled consultation on the IPTV application. Verification is immediate and this fact is reflected in the mobile application so that the caregiver can know the reason why they are offline. If the user is not participating in an event the second level code 201 and in case of not being in a query 202, in both cases the system continues the analysis. 

Another common problem detected during the development period was the generation of alerts for users who live with someone and whose partner is connected. The fact that there is only one STB per house can trigger the initial issuance of alerts for the offline user. 

As such, it is verified if there is another user connected to the house. If this user exists and is activated, the system assumes that this is the cause of the deactivation state and ends the process by assigning a second level code with the value 210. If no one is connected, it is assigned a value of 203 and the system proceeds. 

Knowing that the user had nothing programmed for this date, and that no one is home using the component SecurHomeTV, the next step is to understand when was the last session of the user. This is done by checking the session history of the user in question. 

As soon as this date and time is collected, the system analyzes if this session was less than an hour ago. If the user has been logged in for less than sixty minutes, the system assumes that there is nothing unusual and concludes the process. 

On the other hand, if the last session lasts longer than three days, the system assigns a second level indicator 209. The only reason to opt for the three days is the prevention of weekends, i.e. avoiding a two-day absenteeism. The three days thus guarantee the supposed return to the weekly routine which, in this age group, can mean, for example, the return home after a weekend in the home of a child. In this case, the system moves on to the next step. 

And it is precisely in the next step that television consumption assumes the primary role in detecting user deviations. In this phase, it is checked whether programmes of the user's preference are being transmitted. In order to determine this preference, the user's television consumption habits are collected and programmes with more than 25% viewing are considered preferable. This 25% value is configurable. 

The existence or not of favorite programs to transmit does not represent in this phase any reflection of the process, the system simply stores this information and uses it in the following steps after verifying the probability that the user is connected in that period of time. In this way, this option avoids that the alert is emitted in a precipitate way only depending on the preferences of the user and without analyzing any other data that is considered fundamental for this criterion. 

Without leaving the same level, the system seeks to complement the previous information by assigning a value to the probability that the user is connected in this period of time. Thus, the last 15 days of use of the application are verified and this analysis results in the probability of being connected. 

If the probability of being connected is greater than 50% and favorite programs are broadcast, a second level value of 205 is added to the code. If the probability is greater than 25% (and less than 50%), a code 206 is returned and if this value is added to the fact that there are also favorite programs to be broadcast. Finally, if the probability is greater than 10% and favorite programs are being broadcast, the code returned is 207.  

If none of these conditions are met, the system assumes that there is no favorite program to be broadcast and the probability is less than 10% it is very unlikely that the user is connected and assumes this fact as the cause of the absence of the user not issuing any alert and immediately ending the process. 

By crossing all these values at this stage, we seek to detect all errors caused by changes in routines caused by live or weekend programs whose programming is considerably different from that of other days. For example, a user who watches a soap opera every day at 9pm may not do so just because the program being broadcast on that day is a football match. The same can happen if a senior watches a program every morning but does not do it at the same time on a Saturday or Sunday because they are broadcasting children's programs. 

Once all the criteria have been met, the system returns the generated warning code to the main function. In doing so, the system enters the last phase of the alert process. To do this, the code, composed of the first and second levels, is executed and transformed into an alert, as explained in the next topic. 

2.4.4 Phase 3 - Issuance of alerts 

This moment represents the final phase of a user's analysis process. It is, therefore, here where all the management of the application alerts and their subsequent sending is carried out. 

In this phase of the process, an object of this class is created and the user ID and the code returned from the previous phase are passed as parameters. This code is collected and broken in order to interpret the different levels and causes of the warnings. This analysis results in the following action. 

In this stage it is fundamental that the system decides what it can do with the alert, based on the returned codes the system can decide to insert a new alert, reduce or increase an already released one or simply ignore it in case the caregiver has assumed that everything was fine less than 12 hours ago. The value of 12 hours is editable and its purpose is to avoid that several alerts are generated consecutively by the application for the reasons that led a caregiver to say that there was no reason for it. However, if the alert is of the highest level (red), this criterion is ignored and the alert is inserted. 

The issue of alerts is, like the whole application, a highly hierarchical process that we are now trying to explain. As soon as an integer is obtained, the system tries to insert that alert into the DB. However, this only occurs if this value is greater than 1 (without description) and if the description is less than the one already issued. 

However, the system has the ability to remove alerts when certain conditions are not met. If someone connects inside the house and an alert is issued for the other member, he deletes the alert previously issued.

Repetition of alerts 

Another recurring problem in this system is the result of successive warnings (when the warning is as severe as the previous one) to caregivers if the warning is not resolved. For this to occur in a balanced manner, it is necessary to provide the system with conditions that allow it to do so accurately and according to the severity of the alert. 

Therefore, if the alert is level 2, only a new alert will be inserted after 1 day. If the alert in question is level 3, this repetition will take place over a period of 12 hours or more. For level 4 this period is 3 hours, while for an extremely serious alert (5) the repetition takes place every hour. 

With these periods we try to achieve a redundancy depending on the severity and thus ensure that the caregivers assume the responsibility of withdrawing the alert and assume before the system that everything is fine. All these values are easily configurable in the application. 

Sending e-mails 

The message sent to the caregiver is titled "Alert @grade detected in the SecurHomeTV application". The same message has as body the explanation of the warning informing the caregiver of the cause that caused the error and its severity. Finally, this message consists of a hyperlink through which the user can remove the alert assuming that there is no problem with the person in charge.

However, the body of the message still depends on the recipient, as these can be sent to the caregiver, the caregiver and the system administration. 

Sending SMS 

When sending written messages, the system uses an external API. The message received by the user indicates that an alert has been triggered with a certain degree of severity. Due to the limitation of characters, the user is not informed of the cause for which the alert has been triggered. 

Ending the Process 

As soon as the alerts are activated by the corresponding channels, the user verification process ends. Here, the system advances to the next user, in case he is running all of them, or simply completes the process. 

Emergency button 

In addition to the data collection mentioned above, the TV application has a mechanism that allows the senior citizen to send an emergency request. This panic button is available to the user through the yellow button on the remote control. 

If an elderly person presses this button, the system immediately sends a written message and an e-mail to the caregiver, as well as automatically changes the alert level of the elderly person to red, the lowest level in the hierarchy (see Scales). 

3.2 User: senior 

The data that could be shared between the PI2-SecurHome component and the PI1-SecurHome component using the respective methods of the Web API are identified below. 

3.2.1 Existing data 

Television Data 
- Application status (offline, online...) 
- Real-time television consumption 
- Last television programmes viewed in a given time zone 
- Time of last interaction with television 

Medical data 
- List of medications and dosages 
- List of future medical appointments 
- List of scheduled exams 
- History of medications with the time of taking them and their status (take/not take) 

Personal data 
- User profile (name, sex, date of birth, city, profile picture, interests and skills) 
- Current notifications 
- Messages received 

3.2.2 Methods 

User status 
- List of user status in the application (offline/online) + last interaction with the TV + real time TV consumption 

LastTVShows 
- Lists the last programs viewed in a time interval. 

LastMedication 
- Historical list of medications with time of taking and state (take/not take) 

ReceiveMessages 
- List of messages received by the user 
User - List of user data by ID 

Medical appointments 
- List of Consultations 

Medical Exams 
- List of exams 

3.3 User: caregiver 

3.3.1 Existing data 

Personal data 
- Profile data 
- List of Dependents 
- Messages sent 

Access data 
- Login list for web/mobile applications 
- Time of last access to web/mobile applications 

3.3.2 Methods 

Caregivers 
- Caregiver Data List by ID 

Login 
- Displays the logon history of Web and mobile applications. 

Dependents 
- Lists caregiver dependents 

SentMessages 
- List of alerts received by the caregiver 

4 Final summary 

The developments reported in this document have been technically evaluated, with the following results the correct functioning of the PI2-SecurHome ecosystem was verified, both at the level of its 3 client applications and at the level of the API developed. 

Therefore, the conditions for this are met: 

- within the framework of the individual project 1, existing data can be accessed to compare the performance of the internal warning system (of the PI2-SecurHome ecosystem) with the system developed by the partners. 

- Within the framework of the individual project 3, the respective field evaluation is carried out with the end-users (elderly people and their carers).