Log file

This page documents the log files stored on the device SD-card.

Format

The CANedge logs data in the industry standard MDF4 format, standardized by ASAM. MDF4 is a binary format which allows compact storage of huge amounts of measurement data. It is specifically designed for bus frame logging across e.g. CAN-bus, LIN-bus and Ethernet. MDF4 is widely adopted by the industry and supported by many existing tools.

Specifically, the CANedge uses MDF version 4.11 (file extension: *.MF4).

Timestamps

Each record is timestamped with 50 us resolution.

Finalization & sorting

The CANedge stores log files as unfinalized and unsorted to enable power safety. Finalization3 and sorting4 can be done as a post-processing step to speed up work with the files.

Note

It may be necessary to finalize/sort a log file before it is loaded into some MDF tools

Additional metadata about the device is captured in the files, including many of the fields exposed in the device file.

  • serial number: Device unique ID number

  • device type: Device type (CANedge1 = 0000001D)

  • firmware version: Firmware version

  • hardware version: Hardware version

  • config crc32 checksum: Configuration File checksum

  • storage total: The SD-card total space in MB

  • storage free: The SD-card free space in MB

  • storage id: The SD-card identifier

  • session: File session counter

  • split: File split counter

  • comment: Configurable device string (e.g. application name)

Naming

Log files are organized by the following path structure:

LOG/[DEVICE_ID]/[SESSION_COUNTER]/[SPLIT_COUNTER].[FILE_EXTENSION]

The path is constructed from the following parts:

  • LOG: Static directory name used to store log files

  • DEVICE_ID: Globally unique device ID

  • SESSION_COUNTER: Increased by one for each power cycle1

  • SPLIT_COUNTER: Resets to 1 on each power cycle and increased by one for each file split

  • FILE_EXTENSION: The file extension selected in the configuration (MF4|MFC|MFE|MFM)

For details on log file splits and related limits, see the Logging Configuration section.

File extension

The default extension is MF4. With compression/encryption enabled the extension changes:

Compression enabled

Encryption enabled

File extension

.MF4

X

.MFC

X

.MFE

X

X

.MFM

With both compression and encryption enabled, the data is first compressed, then encrypted.

For details on compression and encryption, see the Logging Configuration section.

Path example

Example: Log file path: LOG/3B912722/00000004/00000189.MF4

  • LOG: The static directory common for all log files

  • 3B912722: The unique ID of the device which generated the log file

  • 00000004: Generated during the 4th session / power cycle

  • 00000189: Is log file number 189 of the session

  • MF4: File type


Generic header

While plain MDF files are saved as MF4, encryption and/or compression uses a custom header to identify and store relevant information for the files. All file headers consist of a generic 20 byte header, followed by any specialized fields.

The generic header starts with an identifying sequence of the ASCII code for Generic File5. Following are information of the header version (V Ge, currently 0x01), file type version (V FT), file type (FT) and file sub-type (FTI). Finally, the device ID is stored. All numbers stored in the generic header are unsigned and big endian formatted.

|<-                     8 bytes                       ->|
| Byte | Byte | Byte | Byte | Byte | Byte | Byte | Byte |
|  'G'    'e'    'n'    'e'    'r'    'i'    'c'   ' '->|
|<-'F'    'i'    'l'    'e' | V Ge | V FT |  FT  |  FTI |
|   Device ID (Uint32, BE)  |

If required, a generic file may contain a footer as well, as specified by the format.

Encrypted files

Encrypted files have a file type of 0x11. The device supports AES encryption in Galois Counter Mode (GCM), with a file sub-type of 0x01. The current version of the format is 0x00. The encrypted file header stores three additional fields:

  • The 12 bytes long initialization vector

  • The number of hashing iterations for the key, stored as a 32 bit unsigned number in big endian format

  • 16 bytes of salt data for the hashing of the key

|<-                     8 bytes                       ->|
| Byte | Byte | Byte | Byte | Byte | Byte | Byte | Byte |
|                       IV/Nonce                      ->|
|<-        IV/Nonce         |  Iterations (Uint32, BE)  |
|                         Salt                        ->|
|<-                       Salt                          |

The encrypted file contains an additional footer. This stores the 16 byte tag generated when AES runs in GCM mode. When decrypting, this tag should be checked to ensure the validity of the decrypted data. There is no alignment requirement for the footer.

|<-                     8 bytes                       ->|
| Byte | Byte | Byte | Byte | Byte | Byte | Byte | Byte |
|                        GCM Tag                      ->|
|<-                      GCM Tag                        |

Compressed files

Compressed files have a file type of 0x22. At present, the only supported compression format is heatshrink based. This is denoted by a file sub-type of 0x01. The current version of the format is 0x01. The additional header data are two unsigned 32 bit numbers: Lookahead and window sizes.

|<-                     8 bytes                       ->|
| Byte | Byte | Byte | Byte | Byte | Byte | Byte | Byte |
|   Lookahead (Uint32, BE)  |    Window (Uint32, BE)    |

Following the header is the compressed data stream. Following the data stream is a footer with a checksum over the compressed data. There is no alignment requirement for the footer. The checksum format is often found online as CRC32 JAM or JAMCRC.

|<-       4 bytes         ->|
| Byte | Byte | Byte | Byte |
|     CRC32 (Uint32, BE)    |

Encrypted and compressed files

If the file is both encrypted and compressed, it has been processed in two steps/streams. First the data is piped through a compression step, next it is piped through an encryption step. Each step can have its own version.


1

The session counter is also increased by one if the counter of splits in one session exceeds 256

3

The MDF file header includes information on how to finalize the MDF file before use

4

Sorting refers to an organization of the log records which enable fast indexing. It is not related to sorting of timestamps.

5

Generic File maps to 12 bytes of ASCII, with no zero termination of the string.