DuckDB Data Accelerator

To use DuckDB as Data Accelerator, specify duckdb as the engine for acceleration.

datasets:
  - from: spiceai:path.to.my_dataset
    name: my_dataset
    acceleration:
      engine: duckdb

Configuration

The DuckDB accelerator can be configured by providing the following params:

duckdb_file: The name for the file to back the DuckDB database. If the file does not exist, it will be created. Only applies if mode is file.

Configuration params are provided in the acceleration section for a data store. Other common acceleration fields can be configured for DuckDB, see see datasets.

datasets:
  - from: spiceai:path.to.my_dataset
    name: my_dataset
    acceleration:
      engine: duckdb
      mode: file
      params:
        duckdb_file: /my/chosen/location/duckdb.db

Limitations

The DuckDB accelerator does not support schemas with field types: nested arrays/lists, UTF8/string arrays/lists, structs, nested structs or map fields. For example:
- Supported:
  - SELECT [1, 2, 3];
  - SELECT ['1992-09-20 11:30:00.123456789', 'epoch'::TIMESTAMP]
- Unsupported:
  - SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]
  - SELECT {'x': 1, 'y': 2, 'z': 3}
  - SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])
  - SELECT ['duck', 'goose', 'heron'];
The DuckDB accelerator does not support Decimal256 (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits.
The DuckDB accelerator does not support using the same duckdb_file for multiple accelerated datasets.
Updating a dataset with DuckDB acceleration while the Spice Runtime is running (hot-reload) will cause DuckDB accelerator query federation to disable until the Runtime is restarted.

Configuration​

Configuration