Reduce your log file size

By default, the CANedge records all raw CAN/LIN frames at their original frequencies - but you can use filters, prescalers, compression and control signals to drastically reduce file size.

In this section we explain how you can reduce the size of your CANedge log files.

Use filter builder tool

The simplest way to build filters is via the ‘filter builder’ editor tool:

  1. Load your config in the config editor, then open the filter builder from the bottom menu
  2. Convert an ‘unfiltered’ log file[1] to CSV via the MF4 converter mdf2csv and load it in the tool
  3. Optionally load your DBC file(s) with CAN channel prefixes (can1-, can2-, …)
  4. Use the tool to add batches of ID filters to your Configuration File and review the size impact

Tips & tricks

  • Optionally use your DBC files (without CSV data) to set filters
  • For J1939/ISOBUS/NMEA data, the tool lets you add PGN filters
  • The search box lets you search entries based on channel, ID, name, signal names and more
  • Use the ‘replace’ option if you only wish to record the selected IDs
  • Use ‘append to top’ to e.g. prescale certain IDs while recording all other IDs as-is
  • Use ‘unmatched’ to select all IDs not matched by DBCs to e.g. add them as rejection filters
  • Evaluate your added filters via the ‘show summary with current filters’

Note

For details on how filters/prescalers work, see the CANedge Docs (Configuration/CAN/Filters)

Filter example

Below example would result in the following:

  1. Only the CAN IDs 70B, 72A and 18FFA43D are recorded
  2. 70B is recorded once every 500 ms
  3. 72A is recorded once every 2nd time it occurs
  4. 18FFA43D is recorded at the original broadcast frequency
CAN-ID-filter-example-canedge

Enable data compression

You can enable compression (in the LOG section) to reduce file size by 50-70%.

Note that your files will be converted from MF4 to MFC in this case. The MF4 converters/decoders natively process MFC files, so for these tools there will be no ‘downside’ to this. If you use asammdf, you must first convert MFC files to MF4 via the MF4 converter mdf2mdf.

Note

If you record very large amounts of data (3+ MB/min), we recommend to use the 256 byte window instead of the 1024 byte window[4]

Setup control signals

Finally, if you only need to record data under certain circumstances, you can use the Control Signal to start/stop logging based on CAN signal thresholds. For detailed examples of how this can be used, see the CANedge Docs.

You can use the control signal to e.g. start logging based on a signal exceeding a limit, or a DTC being flipped. You can also control the behavior of CAN 1 based on a signal on CAN 2.

In the internal GNSS/IMU section we provide an example based on GPS speed.

Below we show how to start/stop logging based on Engine Speed (J1939):

  1. In the editor, show the ‘Advanced’ view and select CAN 1
  2. Set the ‘Reception (rx) initial state’ to Disable
  3. In the Control section, set the ‘Control reception (rx) state’ to Enable
  4. In the Start/Stop sections, enter the signal decoding details[3]
  5. Set the Start trigger low to 200 and the high to an arbitrarily high number
  6. Set the Stop trigger low to 0 and the high to 199

Below ‘partial config’ contains this example[2]:

FW 01.09, J1939 engine speed control signal 'partial config'

j1939-pgn-filter-example-canedge
[1]Make sure the log file is recorded with the ‘default filters’ (where the device records everything at the raw frequency)
[2]You can load this into your own Configuration File with the ‘Partial config loader’ editor tool
[3]You can find the decoding details via your DBC file. Optionally, you can use our DBC editor to load your DBC file to identify the various fields
[4]If you are recording a lot of data per minute, the highest compression window can result in reduced performance for other tasks, e.g. affecting the upload speed of CANedge2/CANedge3 devices. For such cases, it is recommended to use a smaller compression window