Add event detection with email notifications

In this section we explain how you can customize your Lambda to identify events (e.g. a DTC, high temperature, harsh braking, …), notify the user via email/SMS - and create an event summary table in your data lake. The table can be visualized via our Grafana-Athena event summary dashboard. This can be useful in e.g. predictive maintenance and diagnostics.

Add event detection via Lambda

To detect custom signal events via your Lambda, follow below steps:

  1. Download our example functions.py below and modify it as per your needs
  2. Follow the steps to set up a custom Lambda function incl. ARN layer

signal alert function

Function explained

The example function does the following:

  1. Start by uploading the regular Parquet files to the S3 output bucket
  2. Next, loop through a number of user-defined ‘event’ objects
  3. For each event, load the trigger message(s) in messages_filtered_list into a data frame
  4. Identify the start/stop timestamps of the event based on specified signal thresholds
  5. Store these in a Parquet file along with meta information (incl. optionally GPS data[1])
  6. For the 1st ‘start’ event in the file, publish a message to the Lambda ‘event’ SNS topic
  7. Upload the new event table to the S3 output bucket

Below is an illustration of the function for identifying high speed events:

custom-lambda-event-trigger-example-speed

Special case: J1939 source address handling

A common message of interest for alerts and event identification is the J1939 DM01 message containing Diagnostic Trouble Code (DTC) information.

By default, the Lambda creates a unique message name per PGN source address (e.g. CAN1_DM01_DTC_01_S00, CAN1_DM01_DTC_01_SAB, …), each of which you may want to check for DTC events. In the example alert function, we show how you can re-use the same code to loop across all different source address values.

Special case: Multiple trigger signals per message

You can specify multiple trigger signals per message. For example, you may have a message with 4 tire pressure signals (one per wheel). By adding all of them in the trigger_signals list, the code will evaluate each signal based on the same event settings.

Stateless events and rising/falling edges

In the example function, it is important to understand that the Lambda does not have any information beyond the triggering log file. Therefore, event identification has to be done based on the in-file signal ‘transitions’ - rather than the ‘absolute values’.

For example, a ‘rising edge’ start event (rising_as_start = True) is defined as an event where and trigger_signal >= upper_threshold after trigger_signal <= lower_threshold in the period since the last event (within the log file).

Note

As a result, the example function cannot identify transitions that occur between log files[2]

Hysteresis intervals

For continuous signals (e.g. temperature, speed) we recommend using a hysteresis interval by adding a gap between the values of upper_threshold and lower_threshold. This helps event detection when your trigger signal fluctuates around your upper_threshold.

For boolean signals you can set lower_threshold = 0 and upper_threshold = 1.

Email notifications via SNS topic

By default, if the function detects a ‘start event’ in a decoded file, it publishes a message to the pre-deployed Lambda ‘event’ SNS topic. This triggers an email notification to all topic subscribers.

To enable this, add your SNS ARN value in the function (found in the CloudFormation outputs).

Use the SNS subscriptions overview to e.g. add/remove subscribers or notification types (e.g. SMS).

Warning

We strongly recommend to test your function before deployment to production - and to optionally disable SNS messages for the initial period (or use a test email initially)[3]

[1]GPS data is a ‘nice-to-have’ meta information that helps you better understand and browse events. The example function will attempt to add GPS data (if found), but the GPS data is not ‘required’ to perform the event identification.
[2]For example, it may be that trigger_signal < lower_threshold near the end of log file 1. Log file 2 may then start with lower_threshold < trigger_signal < upper_threshold and later go to trigger_signal > upper_threshold. In such a scenario, a ‘rising edge’ event occurred - but the function will not detect it. To minimize such ‘false negative’ events we recommend that you avoid using high-frequent file splitting. Alternatively, you can incorporate basic state management by storing the last value of trigger_signal in a object on S3 (unique per device) and fetching this as part of the function.
[3]If you make errors in your function or your data exhibits unexpected behavior, you can end up sending an email notification for every log file processed - which can be inconvenient to clean up in your inbox.