Datasets

Custom Datasets and Views documentation.

The Spice.ai platform comes pre-loaded with a variety of community datasets.

In addition, you can define and create your own custom and private Datasets and Views, which can then be queried with SQL, cached in Spice Firecache, and published publicly to be shared with others.

Defining a Dataset

To define a dataset, first ensure your Spice app is connected to a GitHub repository, then add a dataset manifest file to the GitHub repository in the .spice/datasets path.

For example:

# .spice/datasets/recent_blocks.yml
name: eth.recent_blocks
type: append
firecache:
  enabled: true
  trigger: number
  time_column: timestamp

Once the manifest file is committed to the GitHub repository, navigate to the Datasets section. The newly defined dataset will appear in the datasets list.

List of synced Datasets.

Deploy the Dataset

Click the dataset Deploy button. Because this dataset was Firecache enabled, the firecache status will now turn to Ready.

Deployed dataset eth.recent_blocks is now firecache Ready.

View Dataset details

Clicking the dataset will show its details along with it's deployments.

Dataset details page.