S3 management

This section covers how the CANedge interfaces to an S3 server and how to manage devices.

Buckets

Each CANedge device is associated with a specific S3 bucket name1. The device will (depending on the configuration) e.g. push files and look for configuration updates within that specific bucket.

Note

The device is not able to create a bucket. Buckets must be created on the server before the device is connected

Device ID

Each CANedge device carries a globally unique 4-byte device ID (e.g. A1B2C3D4). The device ID is used in the communication with S3 so that multiple devices connected to the same bucket do not conflict. The root path of a given device is given by the Bucket name and the unique ID:

[BUCKET_NAME]/[DEVICE_ID]

Log files

The device uploads closed log files to the S3 server. When a log file has been successfully uploaded, it is deleted on the SD-card. Log files pushed to the server are named according to the following pattern:

[BUCKET_NAME]/[DEVICE_ID]/[SESSION_NUMBER]/[SPLIT_NUMBER]-[EPOCH_TIME].[EXT]

  • EPOCH_TIME: Epoch upload time in HEX (e.g. 2020/01/01 00:00:00 becomes 5E0BE100)

  • EXT: one of the possible file extensions supported by the device.

Note

Upload of many small log files has a higher network overhead and will result in lower throughput. Select a higher log file split_size (refer the Logging Configuration Section) for better throughput if the network stability allows it. For reliable networks, aim for at least 10 MB files.

Note

If the server-to-device confirmation that a log file has been uploaded successfully is lost (e.g. due to network problems), the device will upload the log file again. This can result in multiple identical log files with different EPOCH_TIME timestamps. Copies of the same log file (same SESSION_NUMBER and SPLIT_NUMBER) should be ignored or removed.

Note

If log file time splitting is used (rather than file size based splitting), consider to use the split_time_offset to avoid that all devices upload new log files to the server at the same time. For more information, refer the Logging Configuration Section.

Meta data

S3 allows additional meta data to be attached to each file. The device adds below S3 meta data:

  • X-Amz-Meta-Hw: The device hardware version

  • X-Amz-Meta-Fw: The device firmware version

  • X-Amz-Meta-Net: The name of the network which was used to upload the file/object

  • X-Amz-Meta-Timestamp: The last-modified UTC timestamp of the file/object in format YYYYMMDDTHHMMSSZ

  • X-Amz-Meta-Put-Index: S3 PUT operation counter

Heartbeat (Device File)

The device can be configured to periodically push a Heartbeat object (Device File) to the server:

[BUCKET_NAME]/[DEVICE_ID]/device.json

Configuration Over-The-Air (COTA)

The device configuration can be managed directly from the S3 server. Both the Rule Schema and the Configuration File are synchronised with the S3 Server if OTA (over-the-air) is enabled.

Rule Schema

The Rule Schema is automatically uploaded once for each power cycle. This ensures that the server always knows the configuration rules of a specific device. If the firmware is updated, the device automatically uploads the Rule Schema matching the new firmware.

The Rule Schema is placed in the device root path and named as below:

[BUCKET_NAME]/[DEVICE_ID]/schema-01.02.json

Configuration File

The device periodically checks the S3 Bucket for updates to the Configuration File. If the Configuraton File is updated on the server it is automatically downloaded and applied by the device (causing a power cycle). If no Configuration File is located on the server it is automatically uploaded by the device.

The Configuration File is placed in the device root path and named as below:

[BUCKET_NAME]/[DEVICE_ID]/config-01.02.json

Warning

Configuration changes made via the SD card will be overwritten if a Configuration File is on the server. To apply local changes, first remove the server Configuration File

Firmware Over-The-Air (FOTA)

The device periodically checks the S3 bucket for firmware updates if OTA (over-the-air) is enabled.

Updating firmware over-the-air works similarly to updating firmware via the SD card. The Firmware File is placed in the device S3 root path and named firmware.bin:

[BUCKET_NAME]/[DEVICE_ID]/firmware.bin

If a valid Firmware File is added to this path, the device downloads and stores it on the SD. If the new firmware is a MAJOR or MINOR upgrade, an updated Configuration File must also be provided.

FOTA sequence with same MAJOR/MINOR version (example: 01.02.03 to 01.02.04)

  1. The firmware.bin Firmware File is placed in [BUCKET NAME]/[DEVICE ID]/firmware.bin

  2. The device downloads the Firmware File and reboots

  3. The device continues to use the same Configuration File

FOTA sequence with new MAJOR/MINOR version (example: 01.02.03 to 01.03.00)

  1. The new Configuration File config-01.03.json is placed in [BUCKET NAME]/[DEVICE_ID]/config-01.03.json

  2. The firmware.bin Firmware File is placed in [BUCKET NAME]/[DEVICE_ID]/firmware.bin

  3. The device downloads the Firmware and Configuration Files and reboots

  4. The device uses the updated Configuration File

Note

Upgrading to a new MAJOR or MINOR version requires an updated and compatible Configuration File. If this is not found, the update is aborted

Server certificates over-the-air

The device periodically checks S3 for server certificate updates if OTA (over-the-air) is enabled.

Updating server certificates over-the-air works similarly to Local installation of certificate(s). The Server Certificate Bundle is placed in the S3 device root path and named certs_server.p7b:

[BUCKET_NAME]/[DEVICE_ID]/certs_server.p7b

If a valid Server Certificate Bundle file is added to this path, the device downloads and stores it on the SD and reboots. Refer to Prepare certificate(s) on how to prepare a Server Certificate Bundle.

Warning

When updating the Server Certificate Bundle over-the-air, special care must be taken to ensure that the device can access a server both before and after the update. It is recommended to always include both the current and new certificates in the bundle while any transition is performed.

Below provides recommended Server Certificate Bundle update sequences. Refer to Device file for information on how to verify loaded certificates (used in the sequences below).

Upgrade a server connection from HTTP to HTTPS (TLS)

  1. Generate a Server Certificate Bundle (certs_server.p7b) with the certificate(s)

  2. Upload the Server Certificate Bundle to the S3 server

  3. Wait for the device file to show that the certificate has been loaded

  4. Update the configuration file to enable HTTPS

Replace a soon-to-expire-certificate

  1. Generate a Server Certificate Bundle (certs_server.p7b) with both certificate(s) (the soon-to-expire and the new).

  2. Upload the Server Certificate Bundle to the S3 server

  3. Wait for the device file to show that both certificates have been loaded

  4. Generate a new Server Certificate Bundle without the soon-to-expire certificate and repeat step 2 and 3.

Warning

Above sequence assumes that both certificates are valid during the process.

Change to another S3 endpoint

  1. Generate a Server Certificate Bundle (certs_server.p7b) with the current and new certificate(s)

  2. Upload the Server Certificate Bundle to both the current and new S3 servers

  3. Wait for the device file to show that both certificates have been loaded

  4. Upload the new configuration file to both the current server with the updated server information (pointen to the new server)

  5. Wait for the device to connect to the new server (monitor either if the configuration file or device.json is uploaded)

  6. Generate a Server Certificate Bundle (certs_server.p7b) with only the new certificate(s)

  7. Upload the Server Certificate Bundle to new S3 server

  8. Wait for the device file to show that only the new bundle has been loaded

1

Multiple CANedge devices can share the same S3 bucket