Parquet decoder

The Parquet decoder stores the decoded output as Parquet files. For more information the Parquet format itself, see: https://parquet.apache.org/

Note

All major programming languages have Parquet support, see: https://arrow.apache.org/docs/

Tool support

Examples of some specific tools/languages supporting the Parquet format:

Output

The Parquet output data-schema always uses the following structure:

  • One timestamp value (t) using datatype Int64 (MICROS) and snappy compression

  • One or more signal values using datatype double and snappy compression

The row-group-size is set to 1.000.000 (1e6).

The signal names are constructed from the database used for decoding, as in the example below:

         t              Speed    SpeedAccuracy    SpeedValid
____________________    _____    _____________    __________

22-Apr-2022 14:14:43    0.01         2.006            1
22-Apr-2022 14:14:44    0.01         2.152            1
22-Apr-2022 14:14:45    0.01         2.290            1

If specific values exceed the MIN/MAX as defined in the database, they are included in the output as NaN.

Warning

Output records are skipped if all values are NaN.

Changelog

2.4.0

  • First release

Download

Windows AMD64 / x86-64 (64-bit)

  • 2.4.0.zip (MD5: c0eb390675562de6cfbf2deff7f74060)

Linux AMD64 / x86-64 (64-bit)

Linux ARM64 (64-bit)