The execution mode for querying against Databricks. The default is spark\_connect. Possible values:
DROP FEATURE v2Checkpoint [TRUNCATE HISTORY];
```
For more details on dropping Delta table features, refer to the official documentation: [Drop Delta table features](https://docs.databricks.com/en/delta/drop-feature.html#:~:text=Databricks%20provides%20limited%20support%20for,data%20files%20backing%20the%20table.)
* When using `mode: spark_connect`, correlated scalar subqueries can only be used in filters, aggregations, projections, and UPDATE/MERGE/DELETE commands. [Spark Docs](https://spark.apache.org/docs/latest/sql-error-conditions-unsupported-subquery-expression-category-error-class.html#unsupported_correlated_scalar_subquery)
{% hint style="warning" %}
**Memory Considerations**
When using the Databricks (mode: delta\_lake) Data connector without acceleration, data is loaded into memory during query execution. Ensure sufficient memory is available, including overhead for queries and the runtime, especially with concurrent queries.
Memory limitations can be mitigated by storing acceleration data on disk, which is supported by [`duckdb`](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/duckdb.md) and [`sqlite`](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/sqlite.md) accelerators by specifying `mode: file`.
* The Databricks Connector (`mode: spark_connect`) does not yet support streaming query results from Spark.
{% endhint %}
# Debezium
Debezium Data Connector Documentation
[Debezium](https://debezium.io/) is an open-source platform that enables Change Data Capture (CDC) for efficient real-time updates of locally accelerated datasets. Spice supports connecting to a Kafka topic managed by Debezium to keep datasets up-to-date with the source data.
```yaml
datasets:
- from: debezium:my_kafka_topic_with_debezium_changes
name: my_dataset
params:
debezium_transport: kafka # Optional. Only `kafka` is currently supported.
debezium_message_format: json # Optional. Only `json` is currently supported.
kafka_bootstrap_servers: broker1:9092,broker2:9092,broker3:9092 # Required. A comma separated list of Kafka broker servers.
kafka_security_protocol: SASL_SSL # Default is `SASL_SSL`. Valid values are `PLAINTEXT`, `SSL`, `SASL_PLAINTEXT`, `SASL_SSL`.
kafka_sasl_mechanism: SCRAM-SHA-512 # Default is `SCRAM-SHA-512`. Valid values are `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`.
kafka_sasl_username: kafka # Required if `kafka_security_protocol` is `SASL_PLAINTEXT` or `SASL_SSL`.
kafka_sasl_password: ${secrets:kafka_sasl_password} # Required if `kafka_security_protocol` is `SASL_PLAINTEXT` or `SASL_SSL`.
kafka_ssl_ca_location: ./certs/kafka_ca_cert.pem # Optional. Used to verify the SSL/TLS certificate of the Kafka broker.
kafka_enable_ssl_certificate_verification: true # Default is `true`. Set to `false` to disable SSL/TLS certificate verification.
kafka_ssl_endpoint_identification_algorithm: https # Default is `https`. Valid values are `none` and `https`.
acceleration:
enabled: true # Acceleration is required for the debezium connector.
engine: duckdb # `duckdb`, `sqlite` and `postgres` are supported acceleration engines for Debezium.
refresh_mode: changes # Optional. If specified, this is required to be set to `changes` - any other value is an error.
mode: file # Persistence is recommended to not have to rebuild the table each time Spice starts.
```
## Configuration
### `from`
The `from` field takes the form of `debezium:kafka_topic` where `kafka_topic` is the name of the Kafka topic where Debezium is notifying consumers about any upstream changes. In the example above it would listen to the `my_kafka_topic_with_debezium_changes` topic.
### `name`
The dataset name. This will be used as the table name within Spice.
```yaml
datasets:
- from: debezium:my_kafka_topic_with_debezium_changes
name: cool_dataset
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
| Parameter Name | Description |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `debezium_transport` | Optional. The message broker transport to use. The default is kafka. Possible values:
kafka: Use Kafka as the message broker transport. Spice may support additional transports in the future.
|
| `debezium_message_format` | Optional. The message format to use. The default is json. Possible values:
json: Use JSON as the message format. Spice is expected to support additional message formats in the future, like avro.
|
| `kafka_bootstrap_servers` | **Required**. A list of host/port pairs for establishing the initial Kafka cluster connection. The client will use all servers, regardless of the bootstrapping servers specified here. This list only affects the initial hosts used to discover the full server set and should be formatted as `host1:port1,host2:port2,...`. |
| `kafka_security_protocol` | Security protocol for Kafka connections. Default: SASL\_SSL. Options:
PLAINTEXTSSLSASL\_PLAINTEXTSASL\_SSL
|
| `kafka_sasl_mechanism` | SASL (Simple Authentication and Security Layer) authentication mechanism. Default: SCRAM-SHA-512. Options:
PLAINSCRAM-SHA-256SCRAM-SHA-512
|
| `kafka_sasl_username` | SASL username. |
| `kafka_sasl_password` | SASL password. |
| `kafka_ssl_ca_location` | Path to the SSL/TLS CA certificate file for server verification. |
| `kafka_enable_ssl_certificate_verification` | Enable SSL/TLS certificate verification. Default: `true`. |
| `kafka_ssl_endpoint_identification_algorithm` | SSL/TLS endpoint identification algorithm. Default: https. Options:
|
### Acceleration Settings
{% hint style="warning" %}
Using the Debezium connector **requires** [acceleration](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/README.md) to be enabled.
{% endhint %}
The following settings are required:
| Parameter Name | Description |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled` | Required. Must be set to `true` to enable acceleration. |
| `engine` | Required. The acceleration engine to use. Valid values: `duckdb`, `sqlite`, `postgres` |
| `refresh_mode` | Optional. The refresh mode to use. If specified, this must be set to `changes`. Any other value is an error. |
| `mode` | Optional. The persistence mode to use. When using the `duckdb` and `sqlite` engines, it is recommended to set this to `file` to persist the data across restarts. Spice also persists metadata about the dataset, so it can resume from the last known state of the dataset instead of re-fetching the entire dataset. |
# Delta Lake
Delta Lake Data Connector Documentation
Delta Lake data connector connector enables SQL queries from [Delta Lake](https://delta.io/) tables.
```yaml
datasets:
- from: delta_lake:s3://my_bucket/path/to/s3/delta/table/
name: my_delta_lake_table
params:
delta_lake_aws_access_key_id: ${secrets:aws_access_key_id}
delta_lake_aws_secret_access_key: ${secrets:aws_secret_access_key}
```
## Configuration
### `from`
The `from` field for the Delta Lake connector takes the form of `delta_lake:path` where `path` is any supported path, either local or to a cloud storage location. See the [examples](#examples) section below.
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: delta_lake:s3://my_bucket/path/to/s3/delta/table/
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to reference a secret, e.g. `${secrets:aws_access_key_id}`.
| Parameter Name | Description |
| ---------------- | ----------------------------------------------------------------------------------------------------------- |
| `client_timeout` | Optional. Specifies timeout for object store operations. Default value is `30s`. E.g. `client_timeout: 60s` |
## Delta Lake object store parameters
### AWS S3
| Parameter Name | Description |
| ---------------------------------- | ---------------------------------------------------------------------------------- |
| `delta_lake_aws_region` | Optional. The AWS region for the S3 object store. E.g. `us-west-2`. |
| `delta_lake_aws_access_key_id` | The access key ID for the S3 object store. |
| `delta_lake_aws_secret_access_key` | The secret access key for the S3 object store. |
| `delta_lake_aws_endpoint` | Optional. The endpoint for the S3 object store. E.g. `s3.us-west-2.amazonaws.com`. |
### Azure Blob
{% hint style="info" %}
**Note** One of the following auth values must be provided for Azure Blob:
* `delta_lake_azure_storage_account_key`,
* `delta_lake_azure_storage_client_id` and `azure_storage_client_secret`, or
* `delta_lake_azure_storage_sas_key`.
{% endhint %}
| Parameter Name | Description |
| ---------------------------------------- | ---------------------------------------------------------------------- |
| `delta_lake_azure_storage_account_name` | The Azure Storage account name. |
| `delta_lake_azure_storage_account_key` | The Azure Storage master key for accessing the storage account. |
| `delta_lake_azure_storage_client_id` | The service principal client id for accessing the storage account. |
| `delta_lake_azure_storage_client_secret` | The service principal client secret for accessing the storage account. |
| `delta_lake_azure_storage_sas_key` | The shared access signature key for accessing the storage account. |
| `delta_lake_azure_storage_endpoint` | Optional. The endpoint for the Azure Blob storage account. |
### Google Storage (GCS)
| Parameter Name | Description |
| ------------------------ | ------------------------------------------------------------ |
| `google_service_account` | Filesystem path to the Google service account JSON key file. |
## Examples
### Delta Lake + Local
```yaml
- from: delta_lake:/path/to/local/delta/table # A local filesystem path to a Delta Lake table
name: my_delta_lake_table
```
### Delta Lake + S3
```yaml
- from: delta_lake:s3://my_bucket/path/to/s3/delta/table/ # A reference to a table in S3
name: my_delta_lake_table
params:
delta_lake_aws_region: us-west-2 # Optional
delta_lake_aws_access_key_id: ${secrets:aws_access_key_id}
delta_lake_aws_secret_access_key: ${secrets:aws_secret_access_key}
delta_lake_aws_endpoint: s3.us-west-2.amazonaws.com # Optional
```
### Delta Lake + Azure Blob
```yaml
- from: delta_lake:abfss://my_container@my_account.dfs.core.windows.net/path/to/azure/delta/table/ # A reference to a table in Azure Blob
name: my_delta_lake_table
params:
# Account Name + Key
delta_lake_azure_storage_account_name: my_account
delta_lake_azure_storage_account_key: ${secrets:my_key}
# OR Service Principal + Secret
delta_lake_azure_storage_client_id: my_client_id
delta_lake_azure_storage_client_secret: ${secrets:my_secret}
# OR SAS Key
delta_lake_azure_storage_sas_key: my_sas_key
```
### Delta Lake + Google Storage
```yaml
params:
delta_lake_google_service_account_path: /path/to/service-account.json
```
## Types
The table below shows the Delta Lake data types supported, along with the type mapping to Apache Arrow types in Spice.
| Delta Lake Type | Arrow Type |
| --------------- | ------------------------------------- |
| `String` | `Utf8` |
| `Long` | `Int64` |
| `Integer` | `Int32` |
| `Short` | `Int16` |
| `Byte` | `Int8` |
| `Float` | `Float32` |
| `Double` | `Float64` |
| `Boolean` | `Boolean` |
| `Binary` | `Binary` |
| `Date` | `Date32` |
| `Timestamp` | `Timestamp(Microsecond, Some("UTC"))` |
| `TimestampNtz` | `Timestamp(Microsecond, None)` |
| `Decimal` | `Decimal128` |
| `Array` | `List` |
| `Struct` | `Struct` |
| `Map` | `Map` |
## Limitations
* Delta Lake connector does not support reading Delta tables with the `V2Checkpoint` feature enabled. To use the Delta Lake connector with such tables, drop the `V2Checkpoint` feature by executing the following command:
```sql
ALTER TABLE DROP FEATURE v2Checkpoint [TRUNCATE HISTORY];
```
For more details on dropping Delta table features, refer to the official documentation: [Drop Delta table features](https://docs.delta.io/latest/delta-drop-feature.html)
# Dremio
Dremio Data Connector Documentation
[Dremio](https://www.dremio.com/) is a data lake engine that enables high-performance SQL queries directly on data lake storage. It provides a unified interface for querying and analyzing data from various sources without the need for complex data movement or transformation.
This connector enables using Dremio as a data source for federated SQL queries.
```yaml
- from: dremio:datasets.dremio_dataset
name: dremio_dataset
params:
dremio_endpoint: grpc://127.0.0.1:32010
dremio_username: demo
dremio_password: ${secrets:my_dremio_pass}
```
## Configuration
### `from`
The `from` field takes the form `dremio:dataset` where `dataset` is the fully qualified name of the dataset to read from.
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a dataset with mixed-case characters, wrap each case-sensitive part in double quotes: `dremio:my_source."MixedCaseDataset"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
{% hint style="warning" %}
**Limitations**
Currently, only up to three levels of nesting are supported for dataset names (e.g., a.b.c). Additional levels are not supported at this time.
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: dremio:datasets.dremio_dataset
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
| Parameter Name | Description |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `dremio_endpoint` | The endpoint used to connect to the Dremio server. |
| `dremio_username` | The username used to connect to the Dremio endpoint. |
| `dremio_password` | The password used to connect to the Dremio endpoint. Use the [secret replacement syntax](#secrets) to load the password from a secret store, e.g. `${secrets:my_dremio_pass}`. |
## Examples
### Connecting to a GRPC endpoint
```yaml
- from: dremio:datasets.dremio_dataset
name: dremio_dataset
params:
dremio_endpoint: grpc://127.0.0.1:32010
dremio_username: demo
dremio_password: ${secrets:my_dremio_pass}
```
## Types
The table below shows the Dremio data types supported, along with the type mapping to Apache Arrow types in Spice.
| Dremio Type | Arrow Type |
| ----------- | ------------------------------ |
| `INT` | `Int32` |
| `BIGINT` | `Int64` |
| `FLOAT` | `Float32` |
| `DOUBLE` | `Float64` |
| `DECIMAL` | `Decimal128` |
| `VARCHAR` | `Utf8` |
| `VARBINARY` | `Binary` |
| `BOOL` | `Boolean` |
| `DATE` | `Date64` |
| `TIME` | `Time32` |
| `TIMESTAMP` | `Timestamp(Millisecond, None)` |
| `INTERVAL` | `Interval` |
| `LIST` | `List` |
| `STRUCT` | `Struct` |
| `MAP` | `Map` |
## Limitations
{% hint style="warning" %}
**Limitations**
* Dremio connector does not support queries with the EXCEPT and INTERSECT keywords in Spice REPL. Use DISTINCT and IN/NOT IN instead. See the example below.
```sql
# fail
SELECT ws_item_sk FROM web_sales
INTERSECT
SELECT ss_item_sk FROM store_sales;
# success
SELECT DISTINCT ws_item_sk FROM web_sales
WHERE ws_item_sk IN (
SELECT DISTINCT ss_item_sk FROM store_sales
);
# fail
SELECT ws_item_sk FROM web_sales
EXCEPT
SELECT ss_item_sk FROM store_sales;
# success
SELECT DISTINCT ws_item_sk FROM web_sales
WHERE ws_item_sk NOT IN (
SELECT DISTINCT ss_item_sk FROM store_sales
);
```
{% endhint %}
\`\`\`\`
# DuckDB
DuckDB Data Connector Documentation
DuckDB is an in-process SQL OLAP (Online Analytical Processing) database management system designed for analytical query workloads. It is optimized for fast execution and can be embedded directly into applications, providing efficient data processing without the need for a separate database server.
This connector supports DuckDB [persistent databases](https://duckdb.org/docs/connect/overview#persistent-database) as a data source for federated SQL queries.
```yaml
datasets:
- from: duckdb:database.schema.table
name: my_dataset
params:
duckdb_open: path/to/duckdb_file.duckdb
```
## Configuration
### `from`
The `from` field supports one of two forms:
| `from` | Description |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `duckdb:database.schema.table` | Read data from a table named `database.schema.table` in the DuckDB file |
| `duckdb:*` | Read data using any DuckDB function that produces a table. For example one of the [data import](https://duckdb.org/docs/data/overview) functions such as `read_json`, `read_parquet` or `read_csv`. |
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a table or schema with mixed-case characters, wrap each case-sensitive part in double quotes: `duckdb:my_database."MySchema"."MyTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: duckdb:database.schema.table
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
The DuckDB data connector can be configured by providing the following `params`:
| Parameter Name | Description |
| -------------- | ---------------------------------------- |
| `duckdb_open` | The name of the DuckDB database to open. |
Configuration `params` are provided either in the top level `dataset` for a dataset source, or in the `acceleration` section for a data store.
## Examples
### Reading from a relative path
A generic example of DuckDB data connector configuration.
```yaml
datasets:
- from: duckdb:database.schema.table
name: my_dataset
params:
duckdb_open: path/to/duckdb_file.duckdb
```
### Reading from an absolute path
```yaml
datasets:
- from: duckdb:sample_data.nyc.rideshare
name: nyc_rideshare
params:
duckdb_open: /my/path/my_database.db
```
### DuckDB Functions
Common [data import](https://duckdb.org/docs/data/overview) DuckDB functions can also define datasets. Instead of a fixed table reference (e.g. `database.schema.table`), a DuckDB function is provided in the `from:` key. For example
```yaml
datasets:
- from: duckdb:database.schema.table
name: my_dataset
params:
duckdb_open: path/to/duckdb_file.duckdb
- from: duckdb:read_csv('test.csv', header = false)
name: from_function
```
Datasets created from DuckDB functions are similar to a standard `SELECT` query. For example:
```yaml
datasets:
- from: duckdb:read_csv('test.csv', header = false)
```
is equivalent to:
```sql
-- from_function
SELECT * FROM read_csv('test.csv', header = false);
```
Many DuckDB data imports can be rewritten as DuckDB functions, making them usable as Spice datasets. For example:
```sql
SELECT * FROM 'todos.json';
-- As a DuckDB function
SELECT * FROM read_json('todos.json');
```
{% hint style="warning" %}
**Limitations**
* The DuckDB connector does not support enum, dictionary, or map [field types](https://duckdb.org/docs/sql/data_types/overview). For example:
* Unsupported:
* `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])`
* The DuckDB connector does not support `Decimal256` (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits.
{% endhint %}
# DynamoDB
DynamoDB Data Connector Documentation
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. This connector enables using DynamoDB tables as data sources for federated SQL queries in Spice.
```yaml
datasets:
- from: dynamodb:users
name: users
params:
dynamodb_aws_region: us-west-2
dynamodb_aws_access_key_id: ${secrets:aws_access_key_id} # Optional
dynamodb_aws_secret_access_key: ${secrets:aws_secret_access_key} # Optional
dynamodb_aws_session_token: ${secrets:aws_session_token} # Optional
```
## Configuration
### `from`
The `from` field should specify the DynamoDB table name:
| `from` | Description |
| ---------------- | --------------------------------------------- |
| `dynamodb:table` | Read data from a DynamoDB table named `table` |
{% hint style="info" %}
If an expected table is not found, verify the `dynamodb_aws_region` parameter. DynamoDB tables are region-specific.
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: dynamodb:users
name: my_users
params: ...
```
```sql
SELECT COUNT(*) FROM my_users;
```
### `params`
The DynamoDB data connector supports the following configuration parameters:
| Parameter Name | Description |
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `dynamodb_aws_region` | Required. The AWS region containing the DynamoDB table |
| `dynamodb_aws_access_key_id` | Optional. AWS access key ID for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `dynamodb_aws_secret_access_key` | Optional. AWS secret access key for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `dynamodb_aws_session_token` | Optional. AWS session token for authentication |
### Credential Sources
If AWS credentials are not explicitly provided in the configuration, the connector will automatically load credentials from the following sources in order:
1. **Environment Variables**:
* `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
* `AWS_SESSION_TOKEN` (if using temporary credentials)
2. **Shared AWS Config/Credentials Files**:
* Config file: `~/.aws/config` (Linux/Mac) or `%UserProfile%\.aws\config` (Windows)
* Credentials file: `~/.aws/credentials` (Linux/Mac) or `%UserProfile%\.aws\credentials` (Windows)
* The `AWS_PROFILE` environment variable can be used to specify a named profile.
* Supports both static credentials and SSO sessions
* Example credentials file:
```ini
# Static credentials
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
# SSO profile
[profile sso-profile]
sso_start_url = https://my-sso-portal.awsapps.com/start
sso_region = us-west-2
sso_account_id = 123456789012
sso_role_name = MyRole
region = us-west-2
```
{% hint style="info" %}
To set up SSO authentication:
1. Run `aws configure sso` to configure a new SSO profile
2. Use the profile by setting `AWS_PROFILE=sso-profile`
3. Run `aws sso login` to start a new SSO session
{% endhint %}
4. **Web Identity Token Credentials**:
* Used primarily with OpenID Connect (OIDC) and OAuth
* Common in Kubernetes environments using IAM roles for service accounts (IRSA)
5. **ECS Container Credentials**:
* Used when running in Amazon ECS containers
* Automatically uses the task's IAM role
* Retrieved from the ECS credential provider endpoint
6. **EC2 Instance Metadata Service (IMDSv2)**:
* Used when running on EC2 instances
* Automatically uses the instance's IAM role
* Retrieved securely using IMDSv2
The connector will try each source in order until valid credentials are found. If no valid credentials are found, an authentication error will be returned.
{% hint style="info" %}
**IAM Permissions** Regardless of the credential source, the IAM role or user must have appropriate DynamoDB permissions (e.g., `dynamodb:Scan`, `dynamodb:DescribeTable`) to access the table.
{% endhint %}
## Required IAM Permissions
The IAM role or user needs the following permissions to access DynamoDB tables:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:Scan",
"dynamodb:DescribeTable"
],
"Resource": [
"arn:aws:dynamodb:*:*:table/YOUR_TABLE_NAME"
]
}
]
}
```
### Permission Details
| Permission | Purpose |
| ------------------------ | --------------------------------------------------------------- |
| `dynamodb:Scan` | Required. Allows reading all items from the table |
| `dynamodb:DescribeTable` | Required. Allows fetching table metadata and schema information |
### Example IAM Policies
#### Minimal Policy (Read-only access to specific table)
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:Scan",
"dynamodb:DescribeTable"
],
"Resource": "arn:aws:dynamodb:us-west-2:123456789012:table/users"
}
]
}
```
#### Access to Multiple Tables
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:Scan",
"dynamodb:DescribeTable"
],
"Resource": [
"arn:aws:dynamodb:us-west-2:123456789012:table/users",
"arn:aws:dynamodb:us-west-2:123456789012:table/orders"
]
}
]
}
```
#### Access to All Tables in a Region
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:Scan",
"dynamodb:DescribeTable"
],
"Resource": "arn:aws:dynamodb:us-west-2:123456789012:table/*"
}
]
}
```
{% hint style="warning" %}
Security Considerations
* Avoid using `dynamodb:*` permissions as it grants more access than necessary.
* Consider using more restrictive policies in production environments.
* When using IAM roles with EKS, ensure the [service account is properly configured with IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
{% endhint %}
## Examples
### Basic Configuration with Environment Credentials
```yaml
version: v1
kind: Spicepod
name: dynamodb
datasets:
- from: dynamodb:users
name: users
params:
dynamodb_aws_region: us-west-2
acceleration:
enabled: true
```
### Configuration with Explicit Credentials
```yaml
version: v1
kind: Spicepod
name: dynamodb
datasets:
- from: dynamodb:users
name: users
params:
dynamodb_aws_region: us-west-2
dynamodb_aws_access_key_id: ${secrets:aws_access_key_id}
dynamodb_aws_secret_access_key: ${secrets:aws_secret_access_key}
acceleration:
enabled: true
```
### Querying Nested Structures
DynamoDB supports complex nested JSON structures. These fields can be queried using SQL:
```sql
-- Query nested structs
SELECT metadata.registration_ip, metadata.user_agent
FROM users
LIMIT 5;
-- Query nested structs in arrays
SELECT address.city
FROM (
SELECT unnest(addresses) AS address
FROM users
)
WHERE address.city = 'San Francisco';
```
{% hint style="warning" %}
**Limitations**
* The DynamoDB connector currently does not support filter push-down optimization. All filtering is performed after data is retrieved from DynamoDB.
* Primary key optimizations are not yet implemented - retrieving items by their primary key will still scan the table.
* The DynamoDB connector will scan the first 10 items to determine the schema of the table. This may miss columns that are not present in the first 10 items.
{% endhint %}
## Data Types
The DynamoDB connector supports the following data types and mappings:
* Basic scalar types (String, Number, Boolean)
* Lists and Maps
* Nested structures
* Binary data
Example schema from a users table:
```sql
describe users;
```
```bash
+----------------+------------------+-------------+
| column_name | data_type | is_nullable |
+----------------+------------------+-------------+
| email | Utf8 | YES |
| id | Int64 | YES |
| metadata | Struct | YES |
| addresses | List(Struct) | YES |
| preferences | Struct | YES |
| created_at | Utf8 | YES |
...
+----------------+------------------+-------------+
```
## Performance Considerations
* Due to limited support for filter push-down, enable acceleration to prevent scanning the entire table on every query.
# FlightSQL
Flight SQL Data Connector Documentation
Connect to any Flight SQL compatible server (e.g. Influx 3.0, CnosDB, other Spice runtimes!) as a connector for federated SQL queries.
```yaml
- from: flightsql:my_catalog.good_schemas.cool_dataset
name: cool_dataset
params:
flightsql_endpoint: http://127.0.0.1:50051
flightsql_username: spicy
flightsql_password: ${secrets:my_flightsql_pass}
```
## Configuration
### `from`
The `from` field takes the form `flightsql:dataset` where `dataset` is the fully qualified name of the dataset to read from.
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a dataset with mixed-case characters, wrap each case-sensitive part in double quotes: `flightsql:my_catalog."MySchema"."MyTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
### `params`
| Parameter name | Description |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `flightsql_endpoint` | The Apache Flight endpoint used to connect to the Flight SQL server. |
| `flightsql_username` | Optional. The username to use in the underlying Apache flight Handshake Request to authenticate to the server (see [reference](https://arrow.apache.org/docs/format/Flight.html#authentication)). |
| `flightsql_password` | Optional. The password to use in the underlying Apache flight Handshake Request to authenticate to the server. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the password from a secret store, e.g. `${secrets:my_flightsql_pass}`. |
# FTP
FTP/SFTP Data Connector Documentation
FTP (File Transfer Protocol) and SFTP (SSH File Transfer Protocol) are network protocols used for transferring files between a client and server, with FTP being less secure and SFTP providing encrypted file transfer over SSH.
The FTP/SFTP Data Connector enables federated/accelerated SQL query across [supported file formats](/building-blocks/data-connectors#object-store-file-formats) stored in FTP/SFTP servers.
```yaml
datasets:
- from: sftp://remote-sftp-server.com/path/to/folder/
name: my_dataset
params:
file_format: csv
sftp_port: 22
sftp_user: my-sftp-user
sftp_pass: ${secrets:my_sftp_password}
```
## Configuration
### `from`
The `from` field takes one of two forms: `ftp:///` or `sftp:///` where `` is the host to connect to and `` is the path to the file or directory to read from.
If a folder is provided, all child files will be loaded.
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: sftp://remote-sftp-server.com/path/to/folder/
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
#### FTP
| Parameter Name | Description |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data file format. Required if the format cannot be inferred by from the `from` path. See [Object Store File Formats](/building-blocks/data-connectors#object-store-file-formats). |
| `ftp_port` | Optional, specifies the port of the FTP server. Default is 21. E.g. `ftp_port: 21` |
| `ftp_user` | The username for the FTP server. E.g. `ftp_user: my-ftp-user` |
| `ftp_pass` | The password for the FTP server. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the password from a secret store, e.g. `${secrets:my_ftp_pass}`. |
| `client_timeout` | Optional. Specifies timeout for FTP connection. E.g. `client_timeout: 30s`. When not set, no timeout will be configured for FTP client. |
| `hive_partitioning_enabled` | Optional. Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
#### SFTP
| Parameter Name | Description |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data file format. Required if the format cannot be inferred by from the `from` path. See [Object Store File Formats](/building-blocks/data-connectors#object-store-file-formats). |
| `sftp_port` | Optional, specifies the port of the SFTP server. Default is 22. E.g. `sftp_port: 22` |
| `sftp_user` | The username for the SFTP server. E.g. `sftp_user: my-sftp-user` |
| `sftp_pass` | The password for the SFTP server. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the password from a secret store, e.g. `${secrets:my_sftp_pass}`. |
| `client_timeout` | Optional. Specifies timeout for SFTP connection. E.g. `client_timeout: 30s`. When not set, no timeout will be configured for SFTP client. |
| `hive_partitioning_enabled` | Optional. Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
## Examples
### Connecting to FTP
```yaml
- from: ftp://remote-ftp-server.com/path/to/folder/
name: my_dataset
params:
file_format: csv
ftp_user: my-ftp-user
ftp_pass: ${secrets:my_ftp_password}
hive_partitioning_enabled: false
```
### Connecting to SFTP
```yaml
- from: sftp://remote-sftp-server.com/path/to/folder/
name: my_dataset
params:
file_format: csv
sftp_port: 22
sftp_user: my-sftp-user
sftp_pass: ${secrets:my_sftp_password}
hive_partitioning_enabled: false
```
# GitHub
GitHub Data Connector Documentation
The GitHub Data Connector enables federated SQL queries on various GitHub resources such as files, issues, pull requests, and commits by specifying `github` as the selector in the `from` value for the dataset.
## Common Configuration
## Configuration
### `from`
The `from` field takes the form of `github:github.com///` where `content` could be `files`, `issues`, `pulls`, `commits`, `stargazers`. See [examples](#examples) for more configuration detail.
### `name`
The dataset name. This will be used as the table name within Spice.
### `params`
#### Personal Access Token
| Parameter Name | Description |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `github_token` | Required. GitHub personal access token to use to connect to the GitHub API. [Learn more](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). |
#### GitHub App Installation
GitHub Apps provide a secure and scalable way to integrate with GitHub's API. [Learn more](https://docs.github.com/en/apps).
| Parameter Name | Description |
| ------------------------ | ------------------------------------------------------------------------------ |
| `github_client_id` | Required. Specifies the client ID for GitHub App Installation auth mode. |
| `github_private_key` | Required. Specifies the private key for GitHub App Installation auth mode. |
| `github_installation_id` | Required. Specifies the installation ID for GitHub App Installation auth mode. |
{% hint style="warning" %}
**Limitations**
With GitHub App Installation authentication, the connector's functionality depends on the permissions and scope of the GitHub App. Ensure that the app is installed on the repositories and configured with content, commits, issues and pull permissions to allow the corresponding datasets to work.
{% endhint %}
#### Common Parameters
| Parameter Name | Description |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `github_query_mode` | Optional. Specifies whether the connector should use the GitHub [search API](https://docs.github.com/en/graphql/reference/queries#search) for improved filter performance. Defaults to `auto`, possible values of `auto` or `search`. |
| `owner` | Required. Specifies the owner of the GitHub repository. |
| `repo` | Required. Specifies the name of the GitHub repository. |
## Filter Push Down
GitHub queries support a `github_query_mode` parameter, which can be set to either `auto` or `search` for the following types:
* **Issues**: Defaults to `auto`. Query filters are only pushed down to the GitHub API in `search` mode.
* **Pull Requests**: Defaults to `auto`. Query filters are only pushed down to the GitHub API in `search` mode.
Commits only supports `auto` mode. Query with filter push down is only enabled for the `committed_date` column. `commited_date` supports exact matches, or greater/less than matches for dates provided in [ISO8601](https://www.iso.org/iso-8601-date-and-time-format.html) format, like `WHERE committed_date > '2024-09-24'`.
When set to `search`, Issues and Pull Requests will use the GitHub [Search API](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests) for improved filter performance when querying against the columns:
* `author` and `state`; supports exact matches, or NOT matches. For example, `WHERE author = 'peasee'` or `WHERE author <> 'peasee'`.
* `body` and `title`; supports exact matches, or LIKE matches. For example, `WHERE body LIKE '%duckdb%'`.
* `updated_at`, `created_at`, `merged_at` and `closed_at`; supports exact matches, or greater/less than matches with dates provided in [ISO8601](https://www.iso.org/iso-8601-date-and-time-format.html) format. For example, `WHERE created_at > '2024-09-24'`.
All other filters are supported when `github_query_mode` is set to `search`, but cannot be pushed down to the GitHub API for improved performance.
{% hint style="warning" %}
**Limitations**
* GitHub has a limitation in the Search API where it may return more stale data than the standard API used in the default query mode.
* GitHub has a limitation in the Search API where it only returns a maximum of 1000 results for a query. Use [append mode acceleration](https://github.com/spicehq/docs/blob/trunk/features/data-acceleration/data-refresh.md) to retrieve more results over time. See the [append example](#append-example) for pull requests.
{% endhint %}
## Examples
### Querying GitHub Files
{% hint style="warning" %}
**Limitations**
* `content` column is fetched only when acceleration is enabled.
* Querying GitHub files does not support filter push down, which may result in long query times when acceleration is disabled.
* Setting `github_query_mode` to `search` is not supported.
{% endhint %}
* `ref` - Required. Specifies the GitHub branch or tag to fetch files from.
* `include` - Optional. Specifies a pattern to include specific files. Supports glob patterns. If not specified, all files are included by default.
```yaml
datasets:
- from: github:github.com///files/[
name: spiceai.files
params:
github_token: ${secrets:GITHUB_TOKEN}
include: '**/*.json; **/*.yaml'
acceleration:
enabled: true
```
#### Schema
| Column Name | Data Type | Is Nullable |
| ------------- | --------- | ----------- |
| name | Utf8 | YES |
| path | Utf8 | YES |
| size | Int64 | YES |
| sha | Utf8 | YES |
| mode | Utf8 | YES |
| url | Utf8 | YES |
| download\_url | Utf8 | YES |
| content | Utf8 | YES |
#### Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/files/v0.17.2-beta
name: spiceai.files
params:
github_token: ${secrets:GITHUB_TOKEN}
include: '**/*.txt' # include txt files only
acceleration:
enabled: true
```
```console
sql> select * from spiceai.files
+-------------+-------------+------+------------------------------------------+--------+-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+-------------+
| name | path | size | sha | mode | url | download_url | content |
+-------------+-------------+------+------------------------------------------+--------+-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+-------------+
| version.txt | version.txt | 12 | ee80f747038c30e776eecb2c2ae155dec9a68187 | 100644 | https://api.github.com/repos/spiceai/spiceai/git/blobs/ee80f747038c30e776eecb2c2ae155dec9a68187 | https://raw.githubusercontent.com/spiceai/spiceai/v0.17.2-beta/version.txt | 0.17.2-beta |
| | | | | | | | |
+-------------+-------------+------+------------------------------------------+--------+-------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------+-------------+
Time: 0.005067 seconds. 1 rows.
```
### Querying GitHub Issues
{% hint style="warning" %}
**Limitations**
* Querying with filters using date columns requires the use of [ISO8601 formatted dates](https://www.iso.org/iso-8601-date-and-time-format.html). For example, `WHERE created_at > '2024-09-24'`.
{% endhint %}
```yaml
datasets:
- from: github:github.com///issues
name: spiceai.issues
params:
github_token: ${secrets:GITHUB_TOKEN}
acceleration:
enabled: true
```
#### Schema
| Column Name | Data Type | Is Nullable |
| ---------------- | ------------ | ----------- |
| assignees | List(Utf8) | YES |
| author | Utf8 | YES |
| body | Utf8 | YES |
| closed\_at | Timestamp | YES |
| comments | List(Struct) | YES |
| created\_at | Timestamp | YES |
| id | Utf8 | YES |
| labels | List(Utf8) | YES |
| milestone\_id | Utf8 | YES |
| milestone\_title | Utf8 | YES |
| comments\_count | Int64 | YES |
| number | Int64 | YES |
| state | Utf8 | YES |
| title | Utf8 | YES |
| updated\_at | Timestamp | YES |
| url | Utf8 | YES |
#### Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/issues
name: spiceai.issues
params:
github_token: ${secrets:GITHUB_TOKEN}
```
```console
sql> select title, state, labels from spiceai.issues where title like '%duckdb%'
+-----------------------------------------------------------------------------------------------------------+--------+----------------------+
| title | state | labels |
+-----------------------------------------------------------------------------------------------------------+--------+----------------------+
| Limitation documentation duckdb accelerator about nested struct and decimal256 | CLOSED | [kind/documentation] |
| Inconsistent duckdb connector params: `params.open` and `params.duckdb_file` | CLOSED | [kind/bug] |
| federation across multiple duckdb acceleration tables. | CLOSED | [] |
| Integration tests to cover "On Conflict" behaviors for duckdb accelerator | CLOSED | [kind/task] |
| Permission denied issue while using duckdb data connector with spice using HELM for Kubernetes deployment | CLOSED | [kind/bug] |
+-----------------------------------------------------------------------------------------------------------+--------+----------------------+
Time: 0.011877542 seconds. 5 rows.
```
### Querying GitHub Pull Requests
{% hint style="warning" %}
**Limitations**
* Querying with filters using date columns requires the use of [ISO8601 formatted dates](https://www.iso.org/iso-8601-date-and-time-format.html). For example, `WHERE created_at > '2024-09-24'`.
{% endhint %}
```yaml
datasets:
- from: github:github.com///pulls
name: spiceai.pulls
params:
github_token: ${secrets:GITHUB_TOKEN}
```
#### Schema
| Column Name | Data Type | Is Nullable |
| --------------- | ---------- | ----------- |
| additions | Int64 | YES |
| assignees | List(Utf8) | YES |
| author | Utf8 | YES |
| body | Utf8 | YES |
| changed\_files | Int64 | YES |
| closed\_at | Timestamp | YES |
| comments\_count | Int64 | YES |
| commits\_count | Int64 | YES |
| created\_at | Timestamp | YES |
| deletions | Int64 | YES |
| hashes | List(Utf8) | YES |
| id | Utf8 | YES |
| labels | List(Utf8) | YES |
| merged\_at | Timestamp | YES |
| number | Int64 | YES |
| reviews\_count | Int64 | YES |
| state | Utf8 | YES |
| title | Utf8 | YES |
| url | Utf8 | YES |
#### Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/pulls
name: spiceai.pulls
params:
github_token: ${secrets:GITHUB_TOKEN}
acceleration:
enabled: true
```
```console
sql> select title, url, state from spiceai.pulls where title like '%GitHub connector%'
+---------------------------------------------------------------------+----------------------------------------------+--------+
| title | url | state |
+---------------------------------------------------------------------+----------------------------------------------+--------+
| GitHub connector: convert `labels` and `hashes` to primitive arrays | https://github.com/spiceai/spiceai/pull/2452 | MERGED |
+---------------------------------------------------------------------+----------------------------------------------+--------+
Time: 0.034996667 seconds. 1 rows.
```
#### Append Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/pulls
name: spiceai.pulls
params:
github_token: ${secrets:GITHUB_TOKEN}
github_query_mode: search
time_column: created_at
acceleration:
enabled: true
refresh_mode: append
refresh_check_interval: 6h # check for new results every 6 hours
refresh_data_window: 90d # at initial load, load the last 90 days of pulls
```
### Querying GitHub Commits
{% hint style="warning" %}
**Limitations**
* Querying with filters using date columns requires the use of [ISO8601 formatted dates](https://www.iso.org/iso-8601-date-and-time-format.html). For example, `WHERE committed_date > '2024-09-24'`.
* Setting `github_query_mode` to `search` is not supported.
{% endhint %}
```yaml
datasets:
- from: github:github.com///commits
name: spiceai.commits
params:
github_token: ${secrets:GITHUB_TOKEN}
```
#### Schema
| Column Name | Data Type | Is Nullable |
| ------------------- | --------- | ----------- |
| additions | Int64 | YES |
| author\_email | Utf8 | YES |
| author\_name | Utf8 | YES |
| committed\_date | Timestamp | YES |
| deletions | Int64 | YES |
| id | Utf8 | YES |
| message | Utf8 | YES |
| message\_body | Utf8 | YES |
| message\_head\_line | Utf8 | YES |
| sha | Utf8 | YES |
#### Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/commits
name: spiceai.commits
params:
github_token: ${secrets:GITHUB_TOKEN}
acceleration:
enabled: true
```
```console
sql> select sha, message_head_line from spiceai.commits limit 10
+------------------------------------------+------------------------------------------------------------------------+
| sha | message_head_line |
+------------------------------------------+------------------------------------------------------------------------+
| 2a9fab7905737e1af182e17f40aecc5c4b5dd236 | wait 2 seconds for the status to turn ready in refreshing status tes… |
| b9c210a818abeaf14d2493fde5227781f47faed8 | Update README.md - Remove bigquery from tablet of connectors (#1434) |
| d61e1af61ebf826f83703b8dd939f19e8b2ba426 | Add databricks_use_ssl parameter (#1406) |
| f1ec55c5986e3e5d57eff94197182ffebbae1045 | wording and logs change reflected on readme (#1435) |
| bfc74185584d1e048ef66c72ce3572a0b652bfd9 | Update acknowledgements (#1433) |
| 0d870f1791d456e7924b4ecbbda5f3b762db1e32 | Update helm version and use v0.13.0-alpha (#1436) |
| 12f930cbad69833077bd97ea43599a75cff985fc | Enable push-down federation by default (#1429) |
| 6e4521090aaf39664bd61d245581d34398ce77db | Add functional tests for federation push-down (#1428) |
| fa3279b7d9fcaa5e8baaa2425f69b556bb30e309 | Add LRU cache support for http-based sql queries (#1410) |
| a3f93dde9d1312bfbf14f7ae3b75bdc468289212 | Add guides and examples about error handling (#1427) |
+------------------------------------------+------------------------------------------------------------------------+
Time: 0.0065395 seconds. 10 rows.
```
### Querying GitHub stars (Stargazers)
{% hint style="warning" %}
**Limitations**
* Querying with filters using date columns requires the use of [ISO8601 formatted dates](https://www.iso.org/iso-8601-date-and-time-format.html). For example, `WHERE starred_at > '2024-09-24'`.
* Setting `github_query_mode` to `search` is not supported.
{% endhint %}
```yaml
datasets:
- from: github:github.com///stargazers
name: spiceai.stargazers
params:
github_token: ${secrets:GITHUB_TOKEN}
```
#### Schema
| Column Name | Data Type | Is Nullable |
| ----------- | --------- | ----------- |
| starred\_at | Timestamp | YES |
| login | Utf8 | YES |
| email | Utf8 | YES |
| name | Utf8 | YES |
| company | Utf8 | YES |
| x\_username | Utf8 | YES |
| location | Utf8 | YES |
| avatar\_url | Utf8 | YES |
| bio | Utf8 | YES |
#### Example
```yaml
datasets:
- from: github:github.com/spiceai/spiceai/stargazers
name: spiceai.stargazers
params:
github_token: ${secrets:GITHUB_TOKEN}
acceleration:
enabled: true
```
```console
sql> select starred_at, login from spiceai.stargazers order by starred_at DESC limit 10
+----------------------+----------------------+
| starred_at | login |
+----------------------+----------------------+
| 2024-09-15T13:22:09Z | cisen |
| 2024-09-14T18:04:22Z | tyan-boot |
| 2024-09-13T10:38:01Z | yofriadi |
| 2024-09-13T10:01:33Z | FourSpaces |
| 2024-09-13T04:02:11Z | d4x1 |
| 2024-09-11T18:10:28Z | stephenakearns-insta |
| 2024-09-09T22:17:42Z | Lrs121 |
| 2024-09-09T19:56:26Z | jonathanfinley |
| 2024-09-09T07:02:10Z | leookun |
| 2024-09-09T03:04:27Z | royswale |
+----------------------+----------------------+
Time: 0.0088075 seconds. 10 rows.
```
# GraphQL
GraphQL Data Connector Documentation
The [GraphQL](https://graphql.org/) Data Connector enables federated SQL queries on any GraphQL endpoint by specifying `graphql` as the selector in the `from` value for the dataset.
```yaml
datasets:
- from: graphql:your-graphql-endpoint
name: my_dataset
params:
json_pointer: /data/some/nodes
graphql_query: |
{
some {
nodes {
field1
field2
}
}
}
```
{% hint style="warning" %}
**Limitations**
* The GraphQL data connector does not support variables in the query.
* Filter pushdown, with the exclusion of `LIMIT`, is not currently supported. Using a `LIMIT` will reduce the amount of data requested from the GraphQL server.
{% endhint %}
## Configuration
### `from`
The `from` field takes the form of `graphql:your-graphql-endpoint`.
### `name`
The dataset name. This will be used as the table name within Spice.
### `params`
The GraphQL data connector can be configured by providing the following `params`. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the password from a secret store, e.g. `${secrets:my_graphql_auth_token}`.
| Parameter Name | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `unnest_depth` | Depth level to automatically unnest objects to. By default, disabled if unspecified or `0`. |
| `graphql_auth_token` | The authentication token to use to connect to the GraphQL server. Uses bearer authentication. |
| `graphql_auth_user` | The username to use for basic auth. E.g. `graphql_auth_user: my_user` |
| `graphql_auth_pass` | The password to use for basic auth. E.g. `graphql_auth_pass: ${secrets:my_graphql_auth_pass}` |
| `graphql_query` | The username to use for basic auth. See [examples](#examples) for a sample GraphQL query |
| `json_pointer` | The [JSON pointer](https://datatracker.ietf.org/doc/html/rfc6901) into the response body. When `graphql_query` is [paginated](#pagination), the `json_pointer` can be inferred. |
#### GraphQL Query Example
```yaml
query: |
{
some {
nodes {
field1
field2
}
}
}
```
### Examples
Example using the GitHub GraphQL API and Bearer Auth. The following will use `json_pointer` to retrieve all of the nodes in starredRepositories:
```yaml
from: graphql:https://api.github.com/graphql
name: stars
params:
graphql_auth_token: ${env:GITHUB_TOKEN}
graphql_auth_user: ${env:GRAPHQL_USER} ...
graphql_auth_pass: ${env:GRAPHQL_PASS}
json_pointer: /data/viewer/starredRepositories/nodes
graphql_query: |
{
viewer {
starredRepositories {
nodes {
name
stargazerCount
languages (first: 10) {
nodes {
name
}
}
}
}
}
}
```
## Pagination
The GraphQL Data Connector supports automatic pagination of the response for queries using [cursor pagination](https://graphql.org/learn/pagination/).
The `graphql_query` must include the `pageInfo` field as per [spec](https://relay.dev/graphql/connections.htm#sec-undefined.PageInfo). The connector will parse the `graphql_query`, and when `pageInfo` is present, will retrieve data until pagination completes.
The query must have the correct pagination arguments in the associated paginated field.
### Example
**Forward Pagination:**
```graphql
{
something_paginated(first: 100) {
nodes {
foo
bar
}
pageInfo {
endCursor
hasNextPage
}
}
}
```
**Backward Pagination:**
```graphql
{
something_paginated(last: 100) {
nodes {
foo
bar
}
pageInfo {
startCursor
hasPreviousPage
}
}
}
```
## Working with JSON Data
Tips for working with JSON data. For more information see [Datafusion Docs](https://datafusion.apache.org/user-guide/sql/scalar_functions.html#array-functions).
### Accessing objects fields
You can access the fields of the object using the square bracket notation. Arrays are indexed from 1.
Example for the stargazers query from [pagination section](#pagination):
```bash
sql> select node['login'] as login, node['name'] as name from stargazers limit 5;
+--------------+----------------------+
| login | name |
+--------------+----------------------+
| simsieg | Simon Siegert |
| davidmathers | David Mathers |
| ahmedtadde | Ahmed Tadde |
| lordhamlet | Shih-Fen Cheng |
| thinmy | Thinmy Patrick Alves |
+--------------+----------------------+
```
### Piping array into rows
You can use Datafusion `unnest` function to pipe values from array into rows. We'll be using [countries GraphQL api](https://countries.trevorblades.com) as an example.
```yaml
from: graphql:https://countries.trevorblades.com
name: countries
params:
json_pointer: /data/continents
graphql_query: |
{
continents {
name
countries {
name
capital
}
}
}
description: countries
acceleration:
enabled: true
refresh_mode: full
refresh_check_interval: 30m
```
Example query:
```bash
sql> select continent, country['name'] as country, country['capital'] as capital
from (select name as continent, unnest(countries) as country from countries)
where continent = 'North America' limit 5;
+---------------+---------------------+--------------+
| continent | country | capital |
+---------------+---------------------+--------------+
| North America | Antigua and Barbuda | Saint John's |
| North America | Anguilla | The Valley |
| North America | Aruba | Oranjestad |
| North America | Barbados | Bridgetown |
| North America | Saint Barthélemy | Gustavia |
+---------------+---------------------+--------------+
```
### Unnesting object properties
You can also use the `unnest_depth` parameter to control automatic unnesting of objects from GraphQL responses.
This examples uses the GitHub stargazers endpoint:
```yaml
from: graphql:https://api.github.com/graphql
name: stargazers
params:
graphql_auth_token: ${env:GITHUB_TOKEN}
unnest_depth: 2
json_pointer: /data/repository/stargazers/edges
graphql_query: |
{
repository(name: "spiceai", owner: "spiceai") {
id
name
stargazers(first: 100) {
edges {
node {
id
name
login
}
}
pageInfo {
hasNextPage
endCursor
}
}
}
}
```
If `unnest_depth` is set to 0, or unspecified, object unnesting is disabled. When enabled, unnesting automatically moves nested fields to the parent level.
Without unnesting, stargazers data looks like this in a query:
```bash
sql> select node from stargazers limit 1;
+------------------------------------------------------------+
| node |
+------------------------------------------------------------+
| {id: MDQ6VXNlcjcwNzIw, login: ashtom, name: Thomas Dohmke} |
+------------------------------------------------------------+
```
With unnesting, these properties are automatically placed into their own columns:
```bash
sql> select node from stargazers limit 1;
+------------------+--------+---------------+
| id | login | name |
+------------------+--------+---------------+
| MDQ6VXNlcjcwNzIw | ashtom | Thomas Dohmke |
+------------------+--------+---------------+
```
#### Unnesting Duplicate Columns
By default, the Spice Runtime will error when a duplicate column is detected during unnesting.
For example, this example `spicepod.yml` query would fail due to `name` fields:
```yaml
from: graphql:https://my-graphql-api.com
name: stargazers
params:
unnest_depth: 2
json_pointer: /data/users
graphql_query: |
query {
users {
name
emergency_contact {
name
}
}
}
```
This example would fail with a runtime error:
```bash
WARN runtime: GraphQL Data Connector Error: Invalid object access. Column 'name' already exists in the object.
```
Avoid this error by [using aliases in the query](https://www.apollographql.com/docs/kotlin/advanced/using-aliases/) where possible. In the example above, a duplicate error was introduced from `emergency_contact { name }`.
The example below uses a GraphQL alias to rename `emergency_contact.name` as `emergencyContactName`.
```yaml
from: graphql:https://my-graphql-api.com
name: stargazers
params:
unnest_depth: 2
json_pointer: /data/people
graphql_query: |
query {
users {
name
emergency_contact {
emergencyContactName: name
}
}
}
```
# HTTPS
HTTP(s) Data Connector Documentation
The HTTP(s) Data Connector enables federated SQL query across [supported file formats](/building-blocks/data-connectors#object-store-file-formats) stored at an HTTP(s) endpoint.
```yaml
datasets:
- from: http://my-http-api.com/report.csv
name: local_report
params:
http_password: ${env:MY_HTTP_PASS}
```
## Configuration
### `from`
The `from` field must contain a valid URI to the location of a [supported file](/building-blocks/data-connectors#object-store-file-formats). For example, `http://static_username@my-http-api/report.csv`.
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: http://static_username@my-http-api.com/report.csv
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
The connector supports Basic HTTP authentication via `param` values.
| Parameter Name | Description |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `http_port` | Optional. Port to create HTTP(s) connection over. Default: 80 and 443 for HTTP and HTTPS respectively. |
| `http_username` | Optional. Username to provide connection for HTTP basic authentication. Default: None. |
| `http_password` | Optional. Password to provide connection for HTTP basic authentication. Default: None. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the password from a secret store, e.g. `${secrets:my_http_pass}`. |
| `client_timeout` | Optional. Specifies timeout for HTTP operations. Default value is `30s` E.g. `client_timeout: 60s` |
## Examples
### Basic example
```yaml
datasets:
- from: https://github.com/LAION-AI/audio-dataset/raw/7fd6ae3cfd7cde619f6bed817da7aa2202a5bc28/metadata/freesound/parquet/freesound_parquet.parquet
name: laion_freesound
```
### Using Basic Authentication
```yaml
datasets:
- from: http://static_username@my-http-api.com/report.csv
name: local_report
params:
http_password: ${env:MY_HTTP_PASS}
```
# LocalPod
Localpod Data Connector Documentation
The Localpod Data Connector enables setting up a parent/child relationship between datasets in the current Spicepod. This can be used for configuring multiple/tiered accelerations for a single dataset, and ensuring that the data is only downloaded once from the remote source. For example, you can use the `localpod` connector to create a child dataset that is accelerated in-memory, while the parent dataset is accelerated to a file.
The dataset created by the `localpod` connector will logically have the same data as the parent dataset.
## Synchronized Refreshes
The `localpod` connector supports synchronized refreshes, which ensures that the child dataset is refreshed from the same data as the parent dataset. Synchronized refreshes require that both the parent and child datasets are accelerated with `refresh_mode: full` (which is the default).
When synchronization is enabled, the following logs will be emitted:
```bash
2024-10-28T15:45:24.220665Z INFO runtime::datafusion: Localpod dataset test_local synchronizing refreshes with parent table test
```
### Examples
```yaml
datasets:
- from: postgres:cleaned_sales_data
name: test
params: ...
acceleration:
enabled: true # This dataset will be accelerated into a DuckDB file
engine: duckdb
mode: file
refresh_check_interval: 10s
- from: localpod:test
name: test_local
acceleration:
enabled: true # This dataset accelerates the parent `test` dataset into in-memory Arrow records and is synchronized with the parent
```
# Memory
Memory Data Connector Documentation
The Memory Data Connector enables configuring an in-memory dataset for tables used, or produced by the Spice runtime. Only certain tables, with predefined schemas, can be defined by the connector. These are:
* `store`: Defines a table that LLMs, with memory tooling, can store data in. Requires `mode: read_write`.
### Examples
```yaml
datasets:
- from: memory:store
name: llm_memory
mode: read_write
columns:
- name: value
embeddings: # Easily make your LLM learnings searchable.
- from: all-MiniLM-L6-v2
embeddings:
- name: all-MiniLM-L6-v2
from: huggingface:huggingface.co/sentence-transformers/all-MiniLM-L6-v2
```
# MSSQL
Microsoft SQL Server Data Connector
[Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server) is a relational database management system developed by Microsoft.
The Microsoft SQL Server Data Connector enables federated/accelerated SQL queries on data stored in MSSQL databases.
{% hint style="warning" %}
**Limitations**
1. The connector supports SQL Server authentication (SQL Login and Password) only.
2. Spatial types (`geography`) are not supported, and columns with these types will be ignored.
{% endhint %}
```yaml
datasets:
- from: mssql:path.to.my_dataset
name: my_dataset
params:
mssql_connection_string: ${secrets:mssql_connection_string}
```
## Configuration
### `from`
The `from` field takes the form `mssql:database.schema.table` where `database.schema.table` is the fully-qualified table name in the SQL server.
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a table, schema, or database with mixed-case characters, wrap each case-sensitive part in double quotes: `mssql:my_database."MySchema"."MyTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: mssql:path.to.my_dataset
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
The data connector supports the following `params`. Use the [secret replacement syntax](https://github.com/spicehq/docs/blob/trunk/building-blocks/secret-stores/index.md) to load the secret from a secret store, e.g. `${secrets:my_mssql_conn_string}`.
| Parameter Name | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mssql_connection_string` | The ADO connection string to use to connect to the server. This can be used instead of providing individual connection parameters. |
| `mssql_host` | The hostname or IP address of the Microsoft SQL Server instance. |
| `mssql_port` | (Optional) The port of the Microsoft SQL Server instance. Default value is 1433. |
| `mssql_database` | (Optional) The name of the database to connect to. The default database (`master`) will be used if not specified. |
| `mssql_username` | The username for the SQL Server authentication. |
| `mssql_password` | The password for the SQL Server authentication. |
| `mssql_encrypt` | ](Optional) Specifies whether encryption is required for the connection.
true: (default) This mode requires an SSL connection. If a secure connection cannot be established, server will not connect.false: This mode will not attempt to use an SSL connection, even if the server supports it. Only the login procedure is encrypted.
|
| `mssql_trust_server_certificate` | (Optional) Specifies whether the server certificate should be trusted without validation when encryption is enabled.
true: The server certificate will not be validated and it is accepted as-is.false: (default) Server certificate will be validated against system's certificate storage.
|
### Example
```yaml
datasets:
- from: mssql:SalesLT.Customer
name: customer
params:
mssql_host: mssql-host.database.windows.net
mssql_database: my_catalog
mssql_username: my_user
mssql_password: ${secrets:mssql_pass}
mssql_encrypt: true
mssql_trust_server_certificate: true
```
# MySQL
MySQL Data Connector Documentation
MySQL is an open-source relational database management system that uses structured query language (SQL) for managing and manipulating databases.
The MySQL Data Connector enables federated/accelerated SQL queries on data stored in MySQL databases.
```yaml
datasets:
- from: mysql:mytable
name: my_dataset
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_db: my_database
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
mysql_pool_min: 10
mysql_pool_max: 100
```
## Configuration
### `from`
The `from` field takes the form `mysql:database_name.table_name` where `database_name` is the fully-qualified table name in the SQL server.
If the `database_name` is omitted in the `from` field, the connector will use the database specified in the `mysql_db` parameter. If the `mysql_db` parameter is not provided, it will default to the user's default database.
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a table or database with mixed-case characters, wrap each case-sensitive part in double quotes: `mysql:my_database."MixedCaseTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
These two examples are identical:
```yaml
datasets:
- from: mysql:mytable
name: my_dataset
params:
mysql_db: my_database
...
```
```yaml
datasets:
- from: mysql:my_database.mytable
name: my_dataset
params: ...
```
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: mysql:path.to.my_dataset
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
The dataset name cannot be a reserved keyword or any of the following keywords that are reserved by MySQL:
* `PARTITION`
### `params`
The MySQL data connector can be configured by providing the following `params`. Use the secret replacement syntax to load the secret from a secret store, e.g. `${secrets:my_mysql_conn_string}`.
| Parameter Name | Description |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mysql_connection_string` | The connection string to use to connect to the MySQL server. This can be used instead of providing individual connection parameters. |
| `mysql_host` | The hostname of the MySQL server. |
| `mysql_tcp_port` | The port of the MySQL server. |
| `mysql_db` | The name of the database to connect to. |
| `mysql_user` | The MySQL username. |
| `mysql_pass` | The password to connect with. |
| `mysql_sslmode` | Optional. Specifies the SSL/TLS behavior for the connection, supported values: `required` (default) - requires an SSL connection, `preferred` - tries SSL but connects insecurely if not supported, `disabled` - does not use SSL. |
| `mysql_sslrootcert` | Optional parameter specifying the path to a custom PEM certificate that the connector will trust. |
| `mysql_time_zone` | Optional. Specifies connection time zone. Default is `UTC`. Accepts fixed offsets (e.g., `+02:00`), IANA time zone names (e.g., `America/Los_Angeles`) if supported by the MySQL server, `system` (MySQL server host's OS time zone), or `local_system` (local runtime OS time zone). |
| `mysql_pool_min` | The minimum number of connections to keep open in the pool, lazily created when requested. Default: `10` |
| `mysql_pool_max` | The maximum number of connections to allow in the pool. Default: `100` |
### `metrics`
The MySQL data connector supports the following optional component metrics:
| Metric Name | Type | Description |
| ------------------------------------ | ------- | ------------------------------------------------------------------------------------------------------------------ |
| `connection_count` | Gauge | Gauge of active connections to the database server |
| `connections_in_pool` | Gauge | Gauge of active connections that are idling in the pool |
| `active_wait_requests` | Gauge | Gauge of requests that are waiting for a connection to be returned to the pool |
| `create_failed` | Counter | Counter of connections that failed to be created |
| `discarded_superfluous_connection` | Counter | Counter of connections that were closed because there were already enough idle connections in the pool |
| `discarded_unestablished_connection` | Counter | Counter of connections that were closed because they could not be established |
| `dirty_connection_return` | Counter | Counter of connections that were returned to the pool but were dirty (ie. open transactions, pending queries, etc) |
| `discarded_expired_connection` | Counter | Counter of connections that were discarded because they were expired by the pool constraints (i.e. TTL expired) |
| `resetting_connection` | Counter | Counter of connections that were reset |
| `discarded_error_during_cleanup` | Counter | Counter of connections that were discarded because they returned an error during cleanup |
| `connection_returned_to_pool` | Counter | Counter of connections that were returned to the pool |
These metrics are not enabled by default, enable them by setting the `metrics` parameter:
```yaml
datasets:
- from: mysql:mytable
name: my_dataset
metrics:
- name: connection_count
- name: connections_in_pool
- name: active_wait_requests
- name: create_failed
- name: discarded_superfluous_connection
- name: discarded_unestablished_connection
- name: dirty_connection_return
- name: discarded_expired_connection
- name: resetting_connection
- name: discarded_error_during_cleanup
- name: connection_returned_to_pool
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
```
## Types
The table below shows the MySQL data types supported, along with the type mapping to Apache Arrow types in Spice.
| MySQL Type | Arrow Type |
| ------------ | ------------------------------ |
| `TINYINT` | `Int8` |
| `SMALLINT` | `Int16` |
| `INT` | `Int32` |
| `MEDIUMINT` | `Int32` |
| `BIGINT` | `Int64` |
| `DECIMAL` | `Decimal128` / `Decimal256` |
| `FLOAT` | `Float32` |
| `DOUBLE` | `Float64` |
| `DATETIME` | `Timestamp(Microsecond, None)` |
| `TIMESTAMP` | `Timestamp(Microsecond, None)` |
| `YEAR` | `Int16` |
| `TIME` | `Time64(Nanosecond)` |
| `DATE` | `Date32` |
| `CHAR` | `Utf8` |
| `BINARY` | `Binary` |
| `VARCHAR` | `Utf8` |
| `VARBINARY` | `Binary` |
| `TINYBLOB` | `Binary` |
| `TINYTEXT` | `Utf8` |
| `BLOB` | `Binary` |
| `TEXT` | `Utf8` |
| `MEDIUMBLOB` | `Binary` |
| `MEDIUMTEXT` | `Utf8` |
| `LONGBLOB` | `LargeBinary` |
| `LONGTEXT` | `LargeUtf8` |
| `SET` | `Utf8` |
| `ENUM` | `Dictionary(UInt16, Utf8)` |
| `BIT` | `UInt64` |
{% hint style="info" %}
The MySQL `TIMESTAMP` value is [retrieved as a UTC time value](https://dev.mysql.com/doc/refman/8.4/en/datetime.html) by default. Use the `mysql_time_zone` configuration parameter to specify the desired time zone for interpreting `TIMESTAMP` values during data retrieval.
{% endhint %}
## Examples
### Connecting using username and password
```yaml
datasets:
- from: mysql:path.to.my_dataset
name: my_dataset
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_db: my_database
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
```
### Connecting using SSL
```yaml
datasets:
- from: mysql:path.to.my_dataset
name: my_dataset
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_db: my_database
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
mysql_sslmode: preferred
mysql_sslrootcert: ./custom_cert.pem
```
### Connecting using a Connection String
```yaml
datasets:
- from: mysql:path.to.my_dataset
name: my_dataset
params:
mysql_connection_string: mysql://${secrets:my_user}:${secrets:my_password}@localhost:3306/my_db
```
### Connecting to the default database
```yaml
datasets:
- from: mysql:mytable
name: my_dataset
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
```
### With custom connection pool settings
```yaml
datasets:
- from: mysql:path.to.my_dataset
name: my_dataset
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_db: my_database
mysql_user: my_user
mysql_pass: ${secrets:mysql_pass}
mysql_pool_min: 5
mysql_pool_max: 10
```
# ODBC
# Postgres
PostgreSQL Data Connector Documentation
PostgreSQL is an advanced open-source relational database management system known for its robustness, extensibility, and support for SQL compliance.
The PostgreSQL Server Data Connector enables federated/accelerated SQL queries on data stored in PostgreSQL databases.
```yaml
datasets:
- from: postgres:my_table
name: my_dataset
params:
pg_host: localhost
pg_port: 5432
pg_db: my_database
pg_user: my_user
pg_pass: ${secrets:my_pg_pass}
```
## Configuration
### `from`
The `from` field takes the form `postgres:my_table` where `my_table` is the table identifier in the PostgreSQL server to read from.
The fully-qualified table name (`database.schema.table`) can also be used in the `from` field.
```yaml
datasets:
- from: postgres:my_database.my_schema.my_table
name: my_dataset
params: ...
```
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a table or schema with mixed-case characters, wrap each case-sensitive part in double quotes: `postgres:my_schema."MixedCaseTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: postgres:my_database.my_schema.my_table
name: cool_dataset
params: ...
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
The connection to PostgreSQL can be configured by providing the following `params`:
| Parameter Name | Description |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pg_host` | The hostname of the PostgreSQL server. |
| `pg_port` | The port of the PostgreSQL server. |
| `pg_db` | The name of the database to connect to. |
| `pg_user` | The username to connect with. |
| `pg_pass` | The password to connect with. Use the secret replacement syntax to load the password from a secret store, e.g. `${secrets:my_pg_pass}`. |
| `pg_sslmode` | Optional. Specifies the SSL/TLS behavior for the connection, supported values: `verify-full` (default) - requires SSL, valid root certificate, and matching server host name; `verify-ca` - requires TLS and valid root certificate; `require` - requires TLS; `prefer` - tries TLS but connects insecurely if not supported; `disable` - does not use TLS. |
| `pg_sslrootcert` | Optional parameter specifying the path to a custom PEM certificate that the connector will trust. |
| `connection_pool_size` | Optional. The maximum number of connections to keep open in the connection pool. Default is 10. |
## Types
The table below shows the PostgreSQL data types supported, along with the type mapping to Apache Arrow types in Spice.
| PostgreSQL Type | Arrow Type |
| --------------- | -------------------------------- |
| `int2` | `Int16` |
| `int4` | `Int32` |
| `int8` | `Int64` |
| `money` | `Int64` |
| `float4` | `Float32` |
| `float8` | `Float64` |
| `numeric` | `Decimal128` |
| `text` | `Utf8` |
| `varchar` | `Utf8` |
| `bpchar` | `Utf8` |
| `uuid` | `Utf8` |
| `bytea` | `Binary` |
| `bool` | `Boolean` |
| `json` | `LargeUtf8` |
| `timestamp` | `Timestamp(Nanosecond, None)` |
| `timestampz` | `Timestamp(Nanosecond, TimeZone` |
| `date` | `Date32` |
| `time` | `Time64(Nanosecond)` |
| `interval` | `Interval(MonthDayNano)` |
| `point` | `FixedSizeList(Float64[2])` |
| `int2[]` | `List(Int16)` |
| `int4[]` | `List(Int32)` |
| `int8[]` | `List(Int64)` |
| `float4[]` | `List(Float32)` |
| `float8[]` | `List(Float64)` |
| `text[]` | `List(Utf8)` |
| `bool[]` | `List(Boolean)` |
| `bytea[]` | `List(Binary)` |
| `geometry` | `Binary` |
| `geography` | `Binary` |
| `enum` | `Dictionary(Int8, Utf8)` |
| Composite Types | `Struct` |
{% hint style="info" %}
The Postgres federated queries may result in unexpected result types due to the difference in DataFusion and Postgres size increase rules. Explicitly specify the expected output type of aggregation functions when writing queries involving Postgres tables in Spice. For example, rewrite `SUM(int_col)` into `CAST (SUM(int_col) as BIGINT)`.
{% endhint %}
## Examples
### Connecting using Username/Password
```yaml
datasets:
- from: postgres:my_database.my_schema.my_table
name: my_dataset
params:
pg_host: localhost
pg_port: 5432
pg_db: my_database
pg_user: my_user
pg_pass: ${secrets:my_pg_pass}
```
### Connect using SSL
```yaml
datasets:
- from: postgres:my_database.my_schema.my_table
name: my_dataset
params:
pg_host: localhost
pg_port: 5432
pg_db: my_database
pg_user: my_user
pg_pass: ${secrets:my_pg_pass}
pg_sslmode: verify-ca
pg_sslrootcert: ./custom_cert.pem
```
### Separate dataset/accelerator secrets
Specify different secrets for a PostgreSQL source and acceleration:
```yaml
datasets:
- from: postgres:my_schema.my_table
name: my_dataset
params:
pg_host: localhost
pg_port: 5432
pg_db: my_database
pg_user: my_user
pg_pass: ${secrets:pg1_pass}
acceleration:
engine: postgres
params:
pg_host: localhost
pg_port: 5433
pg_db: acceleration
pg_user: two_user_two_furious
pg_pass: ${secrets:pg2_pass}
```
# S3
S3 Data Connector Documentation
The S3 Data Connector enables federated SQL querying on files stored in S3 or S3-compatible systems (e.g., MinIO, Cloudflare R2).
If a folder path is specified as the dataset source, all files within the folder will be loaded.
File formats are specified using the `file_format` parameter, as described in [Object Store File Formats](/building-blocks/data-connectors#object-store-file-formats).
```yaml
datasets:
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: taxi_trips
params:
file_format: parquet
```
## Configuration
### `from`
S3-compatible URI to a folder or file, in the format `s3:///`
Example: `from: s3://my-bucket/path/to/file.parquet`
### `name`
The dataset name. This will be used as the table name within Spice.
Example:
```yaml
datasets:
- from: s3://s3-bucket-name/taxi_sample.csv
name: cool_dataset
params:
file_format: csv
```
```sql
SELECT COUNT(*) FROM cool_dataset;
```
```shell
+----------+
| count(*) |
+----------+
| 6001215 |
+----------+
```
### `params`
| Parameter Name | Description |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data format. Required if it cannot be inferred from the object URI. Options: `parquet`, `csv`, `json`. Refer to [Object Store File Formats](/building-blocks/data-connectors#object-store-file-formats) for details. |
| `s3_endpoint` | S3 endpoint URL (e.g., for MinIO). Default is the region endpoint. E.g. `s3_endpoint: https://my.minio.server` |
| `s3_region` | S3 bucket region. Default: `us-east-1`. |
| `client_timeout` | Timeout for S3 operations. Default: `30s`. |
| `hive_partitioning_enabled` | Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
| `s3_auth` | Authentication type. Options: `public`, `key` and `iam_role`. Defaults to `public` if `s3_key` and `s3_secret` are not provided, otherwise defaults to `key`. |
| `s3_key` | Access key (e.g. `AWS_ACCESS_KEY_ID` for AWS) |
| `s3_secret` | Secret key (e.g. `AWS_SECRET_ACCESS_KEY` for AWS) |
| `allow_http` | Allow insecure HTTP connections to `s3_endpoint`. Defaults to `false` |
For CSV-specific parameters, see [CSV Parameters](https://github.com/spicehq/docs/blob/trunk/reference/file-format.md#csv).
## Authentication
No authentication is required for public endpoints. For private buckets, set s3\_auth to key or iam\_role. For environments with assigned IAM roles, set `s3_auth` to `iam_role`. If using iam\_role, the [AWS IAM role](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) of the running instance is used.
Minimum IAM policy for S3 access:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": "arn:aws:s3:::company-bucketname-datasets"
},
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": "arn:aws:s3:::company-bucketname-datasets/*"
}
]
}
```
## Types
Refer to [Object Store Data Types](https://github.com/spicehq/docs/blob/trunk/reference/datatypes/object_store.md) for data type mapping from object store files to arrow data type.
## Examples
### Public bucket Example
Create a dataset named `taxi_trips` from a public S3 folder.
```yaml
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: taxi_trips
params:
file_format: parquet
```
### MinIO Example
Create a dataset named `cool_dataset` from a Parquet file stored in MinIO.
```yaml
- from: s3://s3-bucket-name/path/to/parquet/cool_dataset.parquet
name: cool_dataset
params:
s3_endpoint: http://my.minio.server
s3_region: 'us-east-1' # Best practice for MinIO
allow_http: true
```
### Hive Partitioning Example
Hive partitioning is a data organization technique that improves query performance by storing data in a hierarchical directory structure based on partition column values. This allows for efficient data retrieval by skipping unnecessary data scans.
For example, a dataset partitioned by year, month, and day might have a directory structure like:
```plaintext
s3://bucket/dataset/year=2024/month=03/day=15/data_file.parquet
s3://bucket/dataset/year=2024/month=03/day=16/data_file.parquet
```
Spice can automatically infer these partition columns from the directory structure when `hive_partitioning_enabled` is set to `true`.
```yaml
version: v1
kind: Spicepod
name: hive_data
datasets:
- from: s3://spiceai-public-datasets/hive_partitioned_data/
name: hive_data_infer
params:
file_format: parquet
hive_partitioning_enabled: true
```
## Limitations
{% hint style="warning" %}
**Performance Considerations**
When using the S3 Data connector without acceleration, data is loaded into memory during query execution. Ensure sufficient memory is available, including overhead for queries and the runtime, especially with concurrent queries.
Memory limitations can be mitigated by storing acceleration data on disk, which is supported by [`duckdb`](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/duckdb.md) and [`sqlite`](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/sqlite.md) accelerators by specifying `mode: file`.
Each query retrieves data from the S3 source, which might result in significant network requests and bandwidth consumption. This can affect network performance and incur costs related to data transfer from S3.
{% endhint %}
# SharePoint
SharePoint Data Connector Documentation
The SharePoint Data Connector enables federated SQL queries on documents stored in SharePoint.
```yaml
datasets:
- from: sharepoint:drive:Documents/path:/top_secrets/
name: important_documents
params:
sharepoint_client_id: ${secrets:SPICE_SHAREPOINT_CLIENT_ID}
sharepoint_tenant_id: ${secrets:SPICE_SHAREPOINT_TENANT_ID}
sharepoint_client_secret: ${secrets:SPICE_SHAREPOINT_CLIENT_SECRET}
```
#### Example
```sql
SELECT * FROM important_documents limit 1
```
Returns
````json
[
{
"created_by_id": "cbccd193-f9f1-4603-b01d-ff6f3e6f2108",
"created_by_name": "Jack Eadie",
"created_at": "2024-09-09T04:57:00",
"c_tag": "\"c:{BD4D130F-2C95-4E59-9F93-85BD0A9E1B19},1\"",
"e_tag": "\"{BD4D130F-2C95-4E59-9F93-85BD0A9E1B19},1\"",
"id": "01YRH3MPAPCNG33FJMLFHJ7E4FXUFJ4GYZ",
"last_modified_by_id": "cbccd193-f9f1-4603-b01d-ff6f3e6f2108",
"last_modified_by_name": "Jack Eadie",
"last_modified_at": "2024-09-09T04:57:00",
"name": "ngx_google_perftools_module.md",
"size": 959,
"web_url": "https://spiceai.sharepoint.com/Shared%20Documents/md/ngx_google_perftools_module.md",
"content": "# Module ngx_google_perftools_module\n\nThe `ngx_google_perftools_module` module (0.6.29) enables profiling of nginx worker processes using [Google Performance Tools](https://github.com/gperftools/gperftools). The module is intended for nginx developers.\n\nThis module is not built by default, it should be enabled with the `--with-google_perftools_module` configuration parameter.\n\n> **Note:** This module requires the [gperftools](https://github.com/gperftools/gperftools) library.\n\n## Example Configuration\n\n```nginx\ngoogle_perftools_profiles /path/to/profile;\n```\n\nProfiles will be stored as `/path/to/profile.`.\n\n## Directives\n\n### google_perftools_profiles\n\n- **Syntax:** `google_perftools_profiles file;`\n- **Default:** —\n- **Context:** `main`\n\nSets a file name that keeps profiling information of nginx worker process. The ID of the worker process is always a part of the file name and is appended to the end of the file name, after a dot.\n"
}
]
````
{% hint style="warning" %}
**Limitations** The sharepoint connector does not yet support creating a dataset from a single file (e.g. an Excel spreadsheet). Datasets must be created from a folder of documents (see [Document Support](/building-blocks/data-connectors#document-support)).
{% endhint %}
## Configuration
### Parameters
| Name | Required? | Description |
| -------------------------- | --------- | -------------------------------------------------------------------------------------------- |
| `sharepoint_client_id` | **Yes** | The client ID of the Azure AD (Entra) application |
| `sharepoint_tenant_id` | **Yes** | The tenant ID of the Azure AD (Entra) application. |
| `sharepoint_client_secret` | Optional | For service principal authentication. The client secret of the Azure AD (Entra) application. |
{% hint style="info" %}
Only one of `sharepoint_client_secret` or `sharepoint_bearer_token` is allowed.
{% endhint %}
### `from` formats
The `from` field in a SharePoint dataset takes the following format:
```yaml
from: 'sharepoint::/:'
```
#### Drives
`drive_type` in a SharePoint Connector `from` field supports the following types:
| Drive Type | Description | Example |
| ---------- | --------------------------- | ----------------------------------------------------- |
| `drive` | The SharePoint drive's name | `from: sharepoint:drive:Documents/...` |
| `driveId` | The SharePoint drive's ID | `from: sharepoint:driveId:b!Mh8opUGD80ec7zGXgX9r/...` |
| `site` | A SharePoint site's name | `from: sharepoint:site:MySite/...` |
| `siteId` | A SharePoint site's ID | `from: sharepoint:siteId:b!Mh8opUGD80ec7zGXgX9r/...` |
| `group` | A SharePoint group's name | `from: sharepoint:group:MyGroup/...` |
| `groupId` | A SharePoint group's ID | `from: sharepoint:groupId:b!Mh8opUGD80ec7zGXgX9r/...` |
| `me` | A user's OneDrive | `from: sharepoint:me/...` |
{% hint style="info" %}
For the `me` drive type the user is identified based on `sharepoint_client_code` and cannot be used with `sharepoint_client_secret`
{% endhint %}
For a name-based `drive_id`, the connector will attempt to resolve the name to an ID at startup.
#### Subpaths
Within a drive, the SharePoint connector can load documents from:
| Description | Example |
| -------------------------------- | ---------------------------------------------------------------------- |
| The root of the drive | `from: sharepoint:me/root` |
| A specific path within the drive | `from: sharepoint:drive:Documents/path:/top_secrets` |
| A specific folder ID | `from: sharepoint:group:MyGroup/id:01QM2NJSNHBISUGQ52P5AJQ3CBNOXDMVNT` |
## Authentication
### Creating an Enterprise Application
To use the SharePoint connector with service principal authentication, you will need to create an Azure AD application and grant it the necessary permissions. This will also support OAuth2 authentication for users within the tenant (i.e. `sharepoint_bearer_token`).
1. Create a new Azure AD application in the [Azure portal](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/Overview).
2. Under the application's `API permissions`, add the following permissions: `Sites.Read.All`, `Files.Read.All`, `User.Read`, `GroupMember.Read.All`
* For service principal authentication, Application permissions are required.
* For user authentication, only delegated permissions are required.
3. Add `sharepoint_client_id` (from the `Application (Client) ID` field) and `sharepoint_tenant_id` to the connector configuration.
4. Under the application's `Certificates & secrets`, create a new client secret. Use this for the `sharepoint_client_secret` parameter.
# Snowflake
Snowflake Data Connector Documentation
The Snowflake Data Connector enables federated SQL queries across datasets in the [Snowflake Cloud Data Warehouse](https://www.snowflake.com/).
```yaml
datasets:
- from: snowflake:DATABASE.SCHEMA.TABLE
name: table
params:
snowflake_warehouse: COMPUTE_WH
snowflake_role: accountadmin
```
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase by Spice. Snowflake normalizes unquoted identifiers to uppercase, so unquoted identifiers in the `from` field should be UPPERCASED (e.g. `snowflake:MY_DATABASE.MY_SCHEMA.MY_TABLE`). To reference a table created with mixed-case in Snowflake, wrap it in double quotes: `snowflake:MY_DATABASE.MY_SCHEMA."mixedCaseTable"`. See [Snowflake identifier resolution](https://docs.snowflake.com/en/sql-reference/identifiers-syntax#label-identifier-casing) and [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
## Configuration
### `from`
A Snowflake fully qualified table name (database.schema.table). For instance `snowflake:SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.LINEITEM` or `snowflake:TAXI_DATA."2024".TAXI_TRIPS`
### `name`
The dataset name. This will be used as the table name within Spice.
### `params`
| Parameter Name | Description |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `snowflake_warehouse` | Optional, specifies the [Snowflake Warehouse](https://docs.snowflake.com/en/user-guide/warehouses-tasks) to use |
| `snowflake_role` | Optional, specifies the role to use for accessing Snowflake data |
| `snowflake_account` | Required, specifies the Snowflake account-identifier |
| `snowflake_username` | Required, specifies the Snowflake username to use for accessing Snowflake data |
| `snowflake_password` | Optional, specifies the Snowflake password to use for accessing Snowflake data |
| `snowflake_private_key_path` | Optional, specifies the path to Snowflake private key |
| `snowflake_private_key_passphrase` | Optional, specifies the Snowflake private key passphrase |
## Auth
The connector supports password-based and [key-pair](https://docs.snowflake.com/en/user-guide/key-pair-auth) authentication. Login requires the account identifier ('orgname-accountname' format) - use [Finding the organization and account name for an account](https://docs.snowflake.com/en/user-guide/admin-account-identifier#finding-the-organization-and-account-name-for-an-account) instructions.
## Example
```yaml
datasets:
- from: snowflake:SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.LINEITEM
name: lineitem
params:
snowflake_warehouse: COMPUTE_WH
snowflake_role: accountadmin
```
{% hint style="warning" %}
**Limitations**
1. Account identifier does not support the [Legacy account locator in a region format](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-legacy-account-locator-in-a-region). Use [Snowflake preferred name in organization format](https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-1-preferred-account-name-in-your-organization).
2. The connector supports password-based and [key-pair](https://docs.snowflake.com/en/user-guide/key-pair-auth) authentication.
{% endhint %}
# Spark
Apache Spark Data Connector Documentation
Apache Spark as a connector for federated SQL query against a Spark Cluster using [Spark Connect](https://spark.apache.org/docs/latest/spark-connect-overview.html)
```yaml
datasets:
- from: spark:spiceai.datasets.my_awesome_table
name: my_table
params:
spark_remote: sc://my-spark-endpoint
```
{% hint style="info" %}
Unquoted identifiers are normalized to lowercase. To reference a table with mixed-case characters, wrap each case-sensitive part in double quotes: `spark:my_catalog."MySchema"."MyTable"`. See [Identifier Case Sensitivity](/building-blocks/data-connectors#identifier-case-sensitivity-and-quoting).
{% endhint %}
## Configuration
* `spark_remote`: A [spark remote](https://spark.apache.org/docs/latest/spark-connect-overview.html#set-sparkremote-environment-variable) connection URI. Refer to [spark connect client connection string](https://github.com/apache/spark/blob/master/connector/connect/docs/client-connection-string.md) for parameters in URI.
## Limitations
* Correlated scalar subqueries are only supported in filters, aggregations, projections, and UPDATE/MERGE/DELETE commands. [Spark Docs](https://spark.apache.org/docs/latest/sql-error-conditions-unsupported-subquery-expression-category-error-class.html#unsupported_correlated_scalar_subquery)
* The Spark connector does not yet support streaming query results from Spark.
# SpiceAI
Spice.ai Data Connector Documentation
The [Spice.ai](https://spice.ai/) Data Connector enables federated SQL query across datasets in the [Spice.ai Cloud Platform](https://docs.spice.ai/building-blocks/datasets). Access to these datasets requires a free [Spice.ai account](https://spice.ai/login).
## Configuration
### Parameters
#### `from`
The Spice.ai Cloud Platform dataset URI. To query a dataset in a public Spice.ai App, use the format `spice.ai///datasets/`.
## Example
```yaml
- from: spice.ai/spiceai/quickstart/datasets/taxi_trips
name: taxi_trips
```
```yaml
- from: spice.ai/spiceai/tpch/datasets/customer
name: tpch.customer
```
## Full Configuration Example
```yaml
- from: spice.ai/spiceai/tpch/datasets/customer
name: tpch.customer
params:
spiceai_api_key: ${secrets:spiceai_api_key}
acceleration:
enabled: true
```
# Model Providers
Overview of supported model providers for ML and LLMs in Spice.
# Anthropic
Instructions for using language models hosted on Anthropic with Spice.
To use a language model hosted on Anthropic, specify `anthropic` in the `from` field.
To use a specific model, include its model ID in the `from` field (see example below). If not specified, the default model is `claude-3-5-sonnet-latest`.
The following parameters are specific to Anthropic models:
| Parameter | Description | Default |
| ------------------- | -------------------------------- | ------------------------------ |
| `anthropic_api_key` | The Anthropic API key. | - |
| `endpoint` | The Anthropic API base endpoint. | `https://api.anthropic.com/v1` |
Example `spicepod.yml` configuration:
```yaml
models:
- from: anthropic:claude-sonnet-4-5
name: claude_4_5_sonnet
params:
anthropic_api_key: ${ secrets:SPICE_ANTHROPIC_API_KEY }
```
See [Anthropic Model Names](https://platform.claude.com/docs/en/about-claude/models/overview) for a list of supported model names.
# Azure
Instructions for using Azure OpenAI models
To use a language model hosted on Azure OpenAI, specify the `azure` path in the `from` field and the following parameters from the [Azure OpenAI Model Deployment](https://ai.azure.com/resource/deployments) page:
| Param | Description | Default |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- |
| `azure_api_key` | The Azure OpenAI API key from the models deployment page. | - |
| `azure_api_version` | The API version used for the Azure OpenAI service. | - |
| `azure_deployment_name` | The name of the model deployment. | Model name |
| `endpoint` | The Azure OpenAI resource endpoint, e.g., `https://resource-name.openai.azure.com`. | - |
| `azure_entra_token` | The Azure Entra token for authentication. | - |
| `responses_api` | `enabled` or `disabled`. Whether to enable invoking this model from the `/v1/responses` HTTP endpoint | `disabled` |
| `azure_openai_responses_tools` | Comma-separated list of OpenAI-hosted tools exposed via the Responses API for this model. These hosted tools are **not** available from the `/v1/chat/completions` HTTP endpoint. Supported tools: `code_interpreter`, `web_search`. | - |
Only one of `azure_api_key` or `azure_entra_token` can be provided for model configuration.
Example:
```yaml
models:
- from: azure:gpt-4o-mini
name: gpt-4o-mini
params:
endpoint: ${ secrets:SPICE_AZURE_AI_ENDPOINT }
azure_api_version: 2024-08-01-preview
azure_deployment_name: gpt-4o-mini
azure_api_key: ${ secrets:SPICE_AZURE_API_KEY }
# Responses API configuration
responses_api: enabled
azure_openai_responses_tools: web_search
```
Refer to the [Azure OpenAI Service models](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models) for more details on available models and configurations.
Follow the [Azure OpenAI Models Cookbook](https://github.com/spiceai/cookbook/tree/trunk/azure_openai) to try Azure OpenAI models for vector-based search and chat functionalities with structured (taxi trips) and unstructured (GitHub files) data.
# Hugging Face
Instructions for using machine learning models hosted on HuggingFace with Spice.
To use a model hosted on HuggingFace, specify the `huggingface.co` path in the `from` field and, when needed, the files to include.
## Configuration
### `from`
The `from` key takes the form of `huggingface:model_path`. Below shows 2 common example of `from` key configuration.
* `huggingface:username/modelname`: Implies the latest version of `modelname` hosted by `username`.
* `huggingface:huggingface.co/username/modelname:revision`: Specifies a particular `revision` of `modelname` by `username`, including the optional domain.
The `from` key follows the following regex format.
```regex
\A(huggingface:)(huggingface\.co\/)?(?[\w\-]+)\/(?[\w\-]+)(:(?[\w\d\-\.]+))?\z
```
The `from` key consists of five components:
1. **Prefix:** The value must start with `huggingface:`.
2. **Domain (Optional):** Optionally includes `huggingface.co/` immediately after the prefix. Currently no other Huggingface compatible services are supported.
3. **Organization/User:** The HuggingFace organization (`org`).
4. **Model Name:** After a `/`, the model name (`model`).
5. **Revision (Optional):** A colon (`:`) followed by the git-like revision identifier (`revision`).
### `name`
The model name. This will be used as the model ID within Spice and Spice's endpoints (i.e. `https://data.spiceai.io/v1/models`). This can be set to the same value as the model ID in the `from` field.
### `params`
| Param | Description | Default |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `hf_token` | The Huggingface access token. | - |
| `model_type` | The architecture to load the model as. Supported values: `mistral`, `gemma`, `mixtral`, `llama`, `phi2`, `phi3`, `qwen2`, `gemma2`, `starcoder2`, `phi3.5moe`, `deepseekv2`, `deepseekv3` | - |
| `tools` | Which \[tools] should be made available to the model. Set to `auto` to use all available tools. | - |
| `system_prompt` | An additional system prompt used for all chat completions to this model. | - |
### `files`
The specific file path for Huggingface model. For example, GGUF model formats require a specific file path, other varieties (e.g. `.safetensors`) are inferred.
#### Example
```yaml
models:
- from: huggingface:huggingface.co/lmstudio-community/Qwen2.5-Coder-3B-Instruct-GGUF
name: sloth-gguf
files:
- path: Qwen2.5-Coder-3B-Instruct-Q3_K_L.gguf
```
## Access Tokens
Access tokens can be provided for Huggingface models in two ways:
1. In the Huggingface token cache (i.e. `~/.cache/huggingface/token`). Default.
2. Via [model params](#params).
```yaml
models:
- name: llama_3.2_1B
from: huggingface:huggingface.co/meta-llama/Llama-3.2-1B
params:
hf_token: ${ secrets:HF_TOKEN }
```
## Examples
### Load a ML model to predict taxi trips outcomes
```yaml
models:
- from: huggingface:huggingface.co/spiceai/darts:latest
name: hf_model
files:
- path: model.onnx
datasets:
- taxi_trips
```
### Load a LLM model to generate text
```yaml
models:
- from: huggingface:huggingface.co/microsoft/Phi-3.5-mini-instruct
name: phi
```
### Load a private model
```yaml
models:
- name: llama_3.2_1B
from: huggingface:huggingface.co/meta-llama/Llama-3.2-1B
params:
hf_token: ${ secrets:HF_TOKEN }
```
For more details on authentication, see [access tokens](#access-tokens).
{% hint style="warning" %}
**Limitations**
* The throughput, concurrency & latency of a locally hosted model will vary based on the underlying hardware and model size. Spice supports [Apple metal](https://github.com/spicehq/docs/blob/trunk/installation.md#metal-support) and [CUDA](https://github.com/spicehq/docs/blob/trunk/installation.md#cuda-support) for accelerated inference.
* ML models currently only support ONNX file format.
{% endhint %}
# OpenAI
Instructions for using language models hosted on OpenAI or compatible services with Spice.
To use a language model hosted on OpenAI (or compatible), specify the `openai` path in the `from` field.
For a specific model, include it as the model ID in the `from` field (see example below). The default model is `gpt-4o-mini`.
```yaml
models:
- from: openai:gpt-4o-mini
name: openai_model
params:
openai_api_key: ${ secrets:OPENAI_API_KEY } # Required for official OpenAI models
tools: auto # Optional. Connect the model to datasets via SQL query/vector search tools
system_prompt: 'You are a helpful assistant.' # Optional.
# Optional parameters
endpoint: https://api.openai.com/v1 # Override to use a compatible provider (i.e. NVidia NIM)
openai_org_id: ${ secrets:OPENAI_ORG_ID }
openai_project_id: ${ secrets:OPENAI_PROJECT_ID }
# Override default chat completion request parameters
openai_temperature: 0.1
openai_response_format: { 'type': 'json_object' }
# OpenAI Responses API configuration
responses_api: enabled
openai_responses_tools: web_search, code_interpreter
```
## Configuration
### `from`
The `from` field takes the form `openai:model_id` where `model_id` is the model ID of the OpenAI model, valid model IDs are found in the `{endpoint}/v1/models` API response.
Example:
```bash
curl -H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models
```
```json
{
"object": "list",
"data": [
{
"id": "gpt-4o-mini",
"object": "model",
"created": 1727389042,
"owned_by": "system"
},
...
}
```
### `name`
The model name. This will be used as the model ID within Spice and Spice's endpoints (i.e. `http://localhost:8090/v1/models`). This can be set to the same value as the model ID in the `from` field.
### `params`
| Param | Description | Default |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
| `endpoint` | The OpenAI API base endpoint. Can be overridden to use a compatible provider (i.e. Nvidia NIM). | `https://api.openai.com/v1` |
| `tools` | Which [tools](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/tools.md) should be made available to the model. Set to `auto` to use all available tools. | - |
| `system_prompt` | An additional system prompt used for all chat completions to this model. | - |
| `openai_api_key` | The OpenAI API key. | - |
| `openai_org_id` | The OpenAI organization ID. | - |
| `openai_project_id` | The OpenAI project ID. | - |
| `openai_temperature` | Set the default temperature to use on chat completions. | - |
| `openai_response_format` | An object specifying the format that the model must output, see [structured outputs](https://platform.openai.com/docs/guides/structured-outputs). | - |
| `openai_reasoning_effort` | For reasoning models, like `o1`, this parameter specifies the reasoning effort used for the model. | - |
| `openai_usage_tier` | The [OpenAI usage tier](https://platform.openai.com/settings/organization/limits) for the account. This parameter sets the maximum number of concurrent requests based on OpenAI's published limits per tier. Valid values are `free`, `tier1`, `tier2`, `tier3`, `tier4`, or `tier5`. | `tier1` |
| `responses_api` | `enabled` or `disabled`. Whether to enable invoking this model from the `/v1/responses` HTTP endpoint using [OpenAI's Responses API](https://platform.openai.com/docs/api-reference/responses). When using OpenAI-compatible providers, ensure the provider supports OpenAI's Responses API. | `disabled` |
| `openai_responses_tools` | Comma-separated list of OpenAI-hosted tools exposed via the Responses API for this model. These hosted tools are **not** available from the `/v1/chat/completions` HTTP endpoint. Supported tools: `code_interpreter`, `web_search`. | - |
See [Large Language Models](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/index.md) for additional configuration options.
* [Tools](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/tools.md)
* [Memory](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/memory.md)
* [Evals](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/evals.md)
* [Parameter overrides](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/parameter_overrides.md)
## Supported OpenAI Compatible Providers
Spice supports several OpenAI compatible providers. Specify the appropriate endpoint in the params section.
### Azure OpenAI
Follow [Azure AI Models](/building-blocks/model-providers/azure) instructions.
### Groq
Groq provides OpenAI compatible endpoints. Use the following configuration:
```yaml
models:
- from: openai:llama3-groq-70b-8192-tool-use-preview
name: groq-llama
params:
endpoint: https://api.groq.com/openai/v1
openai_api_key: ${ secrets:SPICE_GROQ_API_KEY }
```
### NVidia NIM
NVidia NIM models are OpenAI compatible endpoints. Use the following configuration:
```yaml
models:
- from: openai:my_nim_model_id
name: my_nim_model
params:
endpoint: https://my_nim_host.com/v1
openai_api_key: ${ secrets:SPICE_NIM_API_KEY }
```
### Parasail
Parasail also offers OpenAI compatible endpoints. Use the following configuration:
```yaml
models:
- from: openai:parasail-model-id
name: parasail_model
params:
endpoint: https://api.parasail.com/v1
openai_api_key: ${ secrets:SPICE_PARASAIL_API_KEY }
```
Refer to the respective provider documentation for more details on available models and configurations.
# Perplexity
Instructions for using language models hosted on Perplexity with Spice.
To use a language model hosted on Perplexity, specify `perplexity` in the `from` field.
To use a specific model, include its model ID in the `from` field (see example below). If not specified, the default model is `sonar`.
```yaml
models:
- name: webs
from: perplexity:sonar
params:
perplexity_auth_token: ${ secrets:SPICE_PERPLEXITY_AUTH_TOKEN }
```
The following parameters are specific to Perplexity models:
| Parameter | Description | Default |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `perplexity_auth_token` | The Perplexity API authentication token. | - |
| `perplexity_*` | Additional, perplexity specific parameters to use on all requests. See [Perplexity API Reference](https://docs.perplexity.ai/api-reference/chat-completions) | - |
**Note:** Like other models in Spice, Perplexity can set default overrides for OpenAI parameters. See [Parameter Overrides](https://github.com/spicehq/docs/blob/trunk/features/large-language-models/parameter_overrides.md).
### Example Configuration
```yaml
models:
- name: webs
from: perplexity:sonar
params:
perplexity_auth_token: ${ secrets:SPICE_PERPLEXITY_AUTH_TOKEN }
perplexity_search_domain_filter:
- docs.spiceai.org
- huggingface.co
perplexity_temperature: 0.3141595
```
# SpiceAI
Instructions for using models hosted on the Spice Cloud Platform with Spice.
To use a model hosted on the [Spice Cloud Platform](https://docs.spice.ai/building-blocks/spice-models), specify the `spice.ai` path in the `from` field.
Example:
```yaml
models:
- from: spice.ai/taxi_tech_co/taxi_drives/models/drive_stats
name: drive_stats
datasets:
- drive_stats_inferencing
```
Specific model versions can be referenced using a version label or Training Run ID.
```yaml
models:
- from: spice.ai/taxi_tech_co/taxi_drives/models/drive_stats:latest # Label
name: drive_stats_a
datasets:
- drive_stats_inferencing
- from: spice.ai/taxi_tech_co/taxi_drives/models/drive_stats:60cb80a2-d59b-45c4-9b68-0946303bdcaf # Training Run ID
name: drive_stats_b
datasets:
- drive_stats_inferencing
```
## `from` Format
The from key must conform to the following regex format:
```regex
\A(?:spice\.ai\/)?(?[\w\-]+)\/(?[\w\-]+)(?:\/models)?\/(?[\w\-]+):(?[\w\d\-\.]+)\z
```
Examples:
* `spice.ai/lukekim/smart/models/drive_stats:latest`: Refers to the latest version of the drive\_stats model in the smart application by the user or organization lukekim.
* `spice.ai/lukekim/smart/drive_stats:60cb80a2-d59b-45c4-9b68-0946303bdcaf`: Specifies a model with a unique training run ID.
### Specification
1. **Prefix (Optional):** The value must start with `spice.ai/`.
2. **Organization/User:** The name of the organization or user (`org`) hosting the model.
3. **Application Name**: The name of the application (`app`) which the model belongs to.
4. **Model Name:** The name of the model (`model`).
5. **Version (Optional):** A colon (`:`) followed by the version identifier (`version`), which could be a semantic version, `latest` for the most recent version, or a specific training run ID.
# XAI
Instructions for using xAI models
To use a language model hosted on xAI, specify `xai` path in the `from` field and the associated `xai_api_key` parameter:
| Param | Description | Default |
| ------------- | ---------------- | ------- |
| `xai_api_key` | The xAI API key. | - |
Example:
```yaml
models:
- from: xai:grok4
name: xai
params:
xai_api_key: ${secrets:SPICE_GROK_API_KEY}
```
Refer to the [xAI models documentation](https://docs.x.ai/docs/models) for more details on available models and configurations.
{% hint style="info" %}
Although the xAI [documentation](https://docs.x.ai/docs/guides/structured-outputs) shows that xAI models can return structured outputs, this is not true.
{% endhint %}
# SDKs
Client libraries for querying Spice.ai Cloud and a local Spice runtime
Client SDKs connect to a Spice app over [Apache Arrow Flight](https://github.com/spicehq/docs/tree/trunk/cloud/api/sql-query/apache-arrow-flight-api.md) and return results as Apache Arrow record batches, which convert cheaply into the dataframe type of each language. They are the recommended way to query Spice from application code.
Every SDK connects to either **Spice.ai Cloud** or a **local Spice runtime**. Authentication against Cloud uses an app [API key](https://github.com/spicehq/docs/tree/trunk/cloud/portal/apps/api-keys.md).
| Language | Package | Latest | Documentation |
| ------------------- | ------------------------------- | -------- | ------------------------------------- |
| Python | `spicepy` | `v3.1.0` | [Python SDK](/sdks/sdks/python-sdk) |
| TypeScript, Node.js | `@spiceai/spice` | `3.1.0` | [Node.js SDK](/sdks/sdks/node.js-sdk) |
| Go | `github.com/spiceai/gospice/v8` | `v8.0.1` | [Go SDK](/sdks/sdks/go) |
| Rust | `spiceai` | `3.0.0` | [Rust SDK](/sdks/sdks/rust-sdk) |
| Java | `ai.spice:spiceai` | `0.6.0` | [Java SDK](/sdks/sdks/java-sdk) |
| C#, .NET | `SpiceAI` | `0.3.0` | [.NET SDK](/sdks/sdks/dotnet-sdk) |
## Default endpoints
Each SDK ships defaults for both Cloud and a local runtime. The exact values and how to override them are on each SDK's page, but the hosts are common:
| Target | Arrow Flight | HTTP |
| -------------- | ------------------------------ | ------------------------- |
| Spice.ai Cloud | `flight.spiceai.io` port `443` | `https://data.spiceai.io` |
| Local runtime | `localhost` port `50051` | `http://localhost:8090` |
{% hint style="warning" %}
Most SDKs default to the **local** runtime for Flight. Connecting to Spice.ai Cloud generally requires setting the Cloud endpoint explicitly, or calling the SDK's Cloud helper — supplying an API key alone is not always enough. Each page states what its own defaults are.
{% endhint %}
## Choosing between an SDK and the APIs directly
An SDK is preferable for application code: it handles the Flight handshake, authentication headers, and retries. Query directly against the [HTTP API](https://github.com/spicehq/docs/tree/trunk/cloud/api/sql-query/http-api.md) or [Arrow Flight API](https://github.com/spicehq/docs/tree/trunk/cloud/api/sql-query/apache-arrow-flight-api.md) when a language has no SDK, or for one-off requests from a shell.
# Python SDK
The [Python SDK](https://github.com/spiceai/spicepy) `spicepy` queries [Spice.ai](https://spice.ai) from Python. It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to stream results, returning Apache Arrow records that convert directly into pandas dataframes.
### Requirements
* Python 3.10 or later
The following packages are installed automatically:
* `pyarrow`
* `pandas`
* `certifi`
* `requests`
### Installation
Install from the [GitHub repository](https://github.com/spiceai/spicepy), pinned to a release tag:
```bash
pip install git+https://github.com/spiceai/spicepy@v3.1.0
```
{% hint style="danger" %}
Do **not** run `pip install spicepy`. The `spicepy` name on PyPI belongs to an unrelated third-party project, not to this SDK. Install from the GitHub URL above.
{% endhint %}
To use [parameterized queries](#parameterized-queries), two additional packages are required:
```bash
pip install adbc-driver-flightsql adbc-driver-manager
```
### Usage
Create a `Client`, then call `query()`:
```python
from spicepy import Client
client = Client(
api_key='API_KEY',
flight_url='grpc+tls://flight.spiceai.io',
)
data = client.query('SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;', timeout=5*60)
pd = data.read_pandas()
```
`Client` has the following arguments, all optional:
* **api\_key** (string): App API key, used to authenticate with Spice.ai Cloud. Falls back to the `SPICE_API_KEY` environment variable.
* **flight\_url** (string): Arrow Flight endpoint (default: `grpc://localhost:50051`). Use `grpc+tls://` for TLS and `grpc://` for plaintext.
* **http\_url** (string): HTTP endpoint, used for dataset refreshes (default: `https://data.spiceai.io`).
* **tls\_root\_cert** (Path or string): Path to the TLS certificate to use for the secure connection (omit for automatic detection).
* **user\_agent** (string): Overrides the reported user agent.
{% hint style="warning" %}
The `SPICE_API_KEY` environment variable authenticates HTTP requests only — it is not applied to Arrow Flight. Pass `api_key` to the constructor when querying Spice.ai Cloud.
{% endhint %}
Once a `Client` is obtained, queries can be made using the `query()` function, which returns a `pyarrow.flight.FlightStreamReader`. It has the following arguments:
* **query** (string, required): The SQL query.
* **timeout** (int, optional): The timeout in seconds.
If no timeout is specified, it will default to a 10 min timeout then cancel the query, and a `TimeoutError` exception will be raised.
Call `read_pandas()` to read the whole result into a dataframe, or read it incrementally — see [Streaming](/sdks/sdks/python-sdk/streaming).
### Usage with local Spice runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally. `flight_url` already defaults to the local runtime:
```python
from spicepy import Client
client = Client(http_url='http://localhost:8090')
data = client.query('SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;', timeout=5*60)
pd = data.read_pandas()
```
{% hint style="info" %}
`flight_url` defaults to the local runtime, but `http_url` defaults to Spice.ai Cloud. When working entirely locally, set `http_url` as above so dataset refreshes are sent to the local runtime.
{% endhint %}
### Parameterized queries
`query_with_params(sql, params)` binds positional `$1`, `$2` placeholders and returns a `pyarrow.RecordBatchReader`. It requires the two ADBC packages listed under [Installation](#installation).
```python
reader = client.query_with_params(
'SELECT trip_distance, fare_amount FROM taxi_trips WHERE trip_distance > $1 LIMIT 10',
[5.0],
)
for batch in reader:
print(batch.to_pandas())
```
Parameter values may be plain Python values, whose Arrow type is inferred, or `(value, pyarrow_type)` tuples to set the type explicitly. Pass `[]` for a query with no parameters; `None` raises `ValueError`.
### Refreshing a dataset
`refresh_dataset(dataset, refresh_opts=None)` triggers a refresh of an accelerated dataset over the HTTP endpoint. `RefreshOpts` accepts `refresh_sql`, `refresh_mode`, and `refresh_jitter_max`.
```python
from spicepy import Client, RefreshOpts
client.refresh_dataset('taxi_trips', RefreshOpts(refresh_mode='full'))
```
### Contributing
Contribute to or file an issue with the `spicepy` library at:
# Streaming
The `spicepy` SDK supports streaming partial results as they become available.
This can be used to enable more efficient pipelining scenarios where processing each row of the result set can happen independently.
`spicepy` enables streaming through the use of the [pyarrow Flight API](https://arrow.apache.org/docs/dev/python/api/flight.html).
The object returned from `spicepy.Client.query()` is a [`pyarrow.flight.FlightStreamReader`](https://arrow.apache.org/docs/dev/python/generated/pyarrow.flight.FlightStreamReader.html#pyarrow.flight.FlightStreamReader).
```python
>>> from spicepy import Client
>>> import os
>>> client = Client(api_key=os.environ["API_KEY"], flight_url="grpc+tls://flight.spiceai.io")
>>> rdr = client.query("SELECT * FROM taxi_trips")
```
Calling `read_pandas()` on the `FlightStreamReader` will wait for the stream to return all of the data before returning a pandas DataFrame.
To operate on partial results while the data is streaming, we will take advantage of the [`read_chunk()`](https://arrow.apache.org/docs/dev/python/generated/pyarrow.flight.FlightStreamReader.html#pyarrow.flight.FlightStreamReader.read_chunk) method on `FlightStreamReader`. This returns a `FlightStreamChunk`, which has a `data` attribute that is a [`RecordBatch`](https://arrow.apache.org/docs/dev/python/generated/pyarrow.RecordBatch.html#pyarrow.RecordBatch). Once we have the RecordBatch, we can call `to_pandas()` on it to return the partial data as a pandas DataFrame. When the stream has ended, calling `read_chunk()` will raise a `StopIteration` exception that we can catch.
In this example, we retrieve all 10,000 suppliers from the TPCH Suppliers table. This query retrieves all suppliers in a single call:
```python
import os
from spicepy import Client
client = Client(api_key=os.environ["API_KEY"], flight_url="grpc+tls://flight.spiceai.io")
query = """
SELECT s_suppkey, s_name
FROM tpch.supplier
"""
reader = client.query(query)
suppliers = reader.read_pandas()
```
This call will return a pandas [`DataFrame`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) with all 10,000 suppliers, and is a synchronous call that waits for all data to arrive before returning.
Alternatively, to process chunks of data as they arrive instead of waiting for all data to arrive, `FlightStreamReader` supports reading chunks of data as they become available with `read_chunk()`. Using the same query example above, but processing data chunk by chunk:
```python
reader = client.query(query)
has_more = True
while has_more:
try:
flight_batch = reader.read_chunk()
record_batch = flight_batch.data
processChunk(record_batch.to_pandas())
except StopIteration:
has_more = False
```
{% hint style="info" %}
`query_with_params()` returns a `pyarrow.RecordBatchReader`, but the full result is read before the reader is returned, so it does not deliver partial results as they arrive. Use `query()` for streaming.
{% endhint %}
# Node.js SDK
The Node.js SDK [spice.js](https://www.npmjs.com/package/@spiceai/spice) is the easiest way to use and query [Spice.ai](https://spice.ai) with Node.js.
It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to efficiently stream data to the client and [Apache Arrow](https://arrow.apache.org/) Records as data frames which are then easily converted to JavaScript objects/arrays or JSON.
### Requirements
* [Node.js 20+](https://nodejs.org/)
### Installation
{% tabs %}
{% tab title="npm" %}
```sh
npm install @spiceai/spice@latest --save
```
{% endtab %}
{% tab title="yarn" %}
```sh
yarn add @spiceai/spice
```
{% endtab %}
{% endtabs %}
### Usage
Import `SpiceClient` and instantiate a new instance with your configuration.
You can then submit queries using the `sql` method.
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient({
apiKey: 'API_KEY',
httpUrl: 'https://data.spiceai.io',
flightUrl: 'flight.spiceai.io:443',
});
const table = await spiceClient.sql('SHOW TABLES;');
console.table(table.toArray());
```
Or using the shorthand with just an API key:
```javascript
const spiceClient = new SpiceClient('API_KEY');
```
`SpiceClient` accepts a config object or a string API key:
* `apiKey` (string, optional): API key to authenticate with the endpoint.
* `flightUrl` (string, optional): Host and port of the Flight endpoint, without a scheme (default: `127.0.0.1:50051`).
* `httpUrl` (string, optional): URL of the HTTP endpoint, including the scheme (default: `http://127.0.0.1:8090`).
* `flightTlsEnabled` (boolean, optional): Use TLS for Flight. Defaults to `false` for a localhost address and `true` otherwise.
* `userAgent` (string, optional): Prepended to the reported user agent.
* `customHeaders` (object, optional): Additional headers to send with each request.
* `flightOnly` (boolean, optional): Use only the Flight transport (default: `false`).
* `httpOnly` (boolean, optional): Use only the HTTP transport (default: `false`). Setting both `flightOnly` and `httpOnly` throws.
* `logging` (boolean, optional): Enable or disable logging output (default: `true`).
{% hint style="info" %}
Supplying only an `apiKey` — with neither `httpUrl` nor `flightUrl` — selects the Spice.ai Cloud endpoints (`https://data.spiceai.io` and `flight.spiceai.io:443`). Setting either URL explicitly keeps the local defaults for the other.
{% endhint %}
#### **`sqlJson(query: string)` - Execute SQL queries with JSON results**
The `sqlJson()` method executes SQL queries and returns results in a JSON format with schema information.
```js
const result = await spiceClient.sqlJson('SELECT name, age FROM users LIMIT 5');
console.log(`Returned ${result.row_count} rows`);
console.log('Schema:', result.schema);
console.log('Data:', result.data);
console.log(`Query took ${result.execution_time_ms}ms`);
// Access individual rows
result.data.forEach((row) => {
console.log(`${row.name} is ${row.age} years old`);
});
```
The response includes:
* `row_count`: Number of rows returned
* `schema`: Schema information with field names and types
* `data`: Array of row objects
* `execution_time_ms`: Query execution time in milliseconds
#### Other methods
* `nsql(query, options?)` — generate and run SQL from a natural language question. See the [Text-to-SQL API](https://github.com/spicehq/docs/tree/trunk/cloud/api/nsql.md).
* `search(query, options?)` — run a search query against the app's datasets.
* `refreshAcceleration(dataset, options?)` — trigger a refresh of an accelerated dataset.
* `isSpiceReady()` and `isSpiceHealthy()` — check the runtime's readiness and health.
See the [API reference](/sdks/sdks/node.js-sdk/api-reference) for full signatures.
### Usage with local Spice runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally.
```javascript
import { SpiceClient } from '@spiceai/spice';
const main = async () => {
// uses connection to local runtime by default
const spiceClient = new SpiceClient();
// or use custom connection params:
// const spiceClient = new SpiceClient({
// httpUrl: 'http://my_spice_http_host',
// flightUrl: 'my_spice_flight_host',
// });
const table = await spiceClient.sql(
'SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;'
);
console.table(table.toArray());
};
main();
```
Check [Spice OSS documentation](https://docs.spiceai.org/clients) to learn more.
### Connection retry
From [version 1.0.1](https://github.com/spiceai/spice.js/releases/tag/v1.0.1) the `SpiceClient` implements connection retry mechanism (3 attempts by default). The number of attempts can be configured via `setMaxRetries`:
```
const spiceClient = new SpiceClient('API_KEY');
spiceClient.setMaxRetries(5); // Setting to 0 will disable retries
```
Retries are performed for connection and system internal errors. It is the SDK user's responsibility to properly handle other errors, for example `RESOURCE_EXHAUSTED (HTTP 429)`.
### Contributing
Contribute to or file an issue with the `spice.js` library at: .
# API Reference
## SpiceClient(params)
The top-level object that connects to Spice.ai.
* `params.apiKey` (string, optional): API key to authenticate with the endpoint.
* `params.httpUrl` (string, optional): URL of the HTTP endpoint, including the scheme (default: `http://127.0.0.1:8090`).
* `params.flightUrl` (string, optional): Host and port of the Flight endpoint, without a scheme (default: `127.0.0.1:50051`, using local Spice Runtime).
* `params.flightTlsEnabled` (boolean, optional): Use TLS for Flight. Defaults to `false` for a localhost address and `true` otherwise.
* `params.userAgent` (string, optional): Prepended to the reported user agent.
* `params.customHeaders` (object, optional): Additional headers to send with each request.
* `params.flightOnly` (boolean, optional): Use only the Flight transport (default: `false`).
* `params.httpOnly` (boolean, optional): Use only the HTTP transport (default: `false`). Setting both `flightOnly` and `httpOnly` throws.
* `params.logging` (boolean, optional): Enable or disable logging output (default: `true`).
Supplying only an `apiKey` — with neither URL set — selects the Spice.ai Cloud endpoints for both.
Default connection to local Spice Runtime:
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient();
```
Connect to Spice.ai Cloud Platform:
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient({
apiKey: 'API_KEY',
httpUrl: 'https://data.spiceai.io',
flightUrl: 'flight.spiceai.io:443'
});
```
Or using shorthand:
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient('API_KEY');
```
### SpiceClient Methods
#### `sql(query, options?, onData?, headers?)` — Execute SQL queries
The recommended method for executing SQL queries. Returns an Apache Arrow [Table](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html).
* `query` (string, required): The SQL query to execute.
* `options` (object, optional): Query options including `parameters` for parameterized queries. A callback may be passed in this position instead of `onData`.
* `onData` (callback, optional): Callback for handling [streaming](/sdks/sdks/node.js-sdk/streaming) data.
* `headers` (object, optional): Custom headers to include with the request.
```javascript
// Standard query
const table = await spiceClient.sql("SELECT * FROM tpch.lineitem LIMIT 10");
table.toArray().forEach((row) => {
console.log(JSON.stringify(row));
});
// Parameterized query
const table = await spiceClient.sql(
'SELECT * FROM taxi_trips WHERE passenger_count = $1 AND trip_distance > $2 LIMIT 10',
{ parameters: [2, 5.0] }
);
```
{% hint style="info" %}
Rows returned by `toArray()` are plain objects when the result contains a decimal, timestamp, list, or struct column, and Arrow row proxies otherwise. Use plain property access or `JSON.stringify(row)` rather than calling `row.toJSON()`, which is not present on the plain-object form.
{% endhint %}
Get all elements for a column by calling [`getChild(name: string)`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html#getChild):
```javascript
const table = await spiceClient.sql(
'SELECT sum(l_extendedprice) as sum_extendedprice FROM tpch.lineitem'
);
let sumExtendedPrice = table.getChild("sum_extendedprice");
console.log(sumExtendedPrice?.toJSON());
```
#### `sqlJson(query, headers?)` — Execute SQL queries with JSON results
Returns results in JSON format with schema information.
* `query` (string, required): The SQL query to execute.
* `headers` (object, optional): Custom headers to include with the request.
```javascript
const result = await spiceClient.sqlJson('SELECT name, age FROM users LIMIT 5');
console.log(`Returned ${result.row_count} rows`);
console.log('Schema:', result.schema);
console.log('Data:', result.data);
console.log(`Query took ${result.execution_time_ms}ms`);
```
The response includes:
* `row_count`: Number of rows returned
* `schema`: Schema information with field names and types
* `data`: Array of row objects
* `execution_time_ms`: Query execution time in milliseconds
#### `nsql(query, options?)` — Natural language to SQL
Converts natural language queries into SQL and executes them.
* `query` (string, required): The natural language query.
* `options` (object, optional):
* `datasets` (array, optional): Dataset names to sample from when building the model context. A sampling hint only — it does not restrict which tables the query can target.
* `model` (string, optional): Model to use for SQL generation. When omitted, the single compatible model configured in the app is used; if none or more than one is configured, the request fails.
* `sample_data_enabled` (boolean, optional): Include sample data in context (default: `false`).
See the [Text-to-SQL API](https://github.com/spicehq/docs/tree/trunk/cloud/api/nsql.md) for the full contract.
```javascript
const result = await spiceClient.nsql('Show me the top 5 customers by total sales');
console.log('Generated SQL:', result.sql);
console.log('Results:', result.data);
console.log(`Returned ${result.row_count} rows`);
```
#### `refreshAcceleration(dataset, options?)` — Trigger dataset refresh
Triggers an on-demand refresh for an accelerated dataset.
* `dataset` (string, required): Name of the dataset to refresh.
* `options` (object, optional):
* `refresh_mode` (string): `'full'`, `'append'`, `'changes'`, or `'disabled'`.
* `refresh_sql` (string): Custom SQL query for the refresh.
* `refresh_jitter_max` (string): Maximum jitter time for refresh scheduling.
```javascript
await spiceClient.refreshAcceleration('my_dataset');
// With options
await spiceClient.refreshAcceleration('my_dataset', {
refresh_mode: 'full',
refresh_sql: 'SELECT * FROM source WHERE updated_at > NOW() - INTERVAL 1 DAY',
});
```
#### `isSpiceHealthy()` — Check runtime health
Checks if the Spice runtime is healthy. This endpoint is **unauthenticated**.
```javascript
const isHealthy = await spiceClient.isSpiceHealthy();
```
#### `isSpiceReady()` — Check runtime readiness
Checks if the Spice runtime is ready to accept queries. This endpoint is **authenticated** if an API key is configured.
```javascript
const isReady = await spiceClient.isSpiceReady();
```
#### `search(query, options?)` — Search datasets
Runs a search query against the app's datasets.
```javascript
const results = await spiceClient.search('quarterly revenue trends');
```
#### `query(sql, onData?, headers?)` — Deprecated query method
Deprecated and scheduled for removal in a future version. Use `sql()` instead.
```javascript
const table = await spiceClient.query("SELECT * FROM tpch.lineitem LIMIT 10");
```
#### `setMaxRetries(retries)` — Configure connection retries
Configures the maximum number of connection retry attempts (default: 3).
```javascript
spiceClient.setMaxRetries(5); // Setting to 0 disables retries
```
# Streaming
The `@spiceai/spice` SDK supports streaming partial results as they become available.
This can be used to enable more efficient pipelining scenarios where processing each row of the result set can happen independently.
The [`SpiceClient.sql`](/sdks/sdks/node.js-sdk/api-reference#spiceclient-methods) method takes an optional `onData` callback that will be passed partial results as they become available.
```typescript
public async sql(
queryText: string,
optionsOrCallback?: SqlQueryOptions | ((data: Table) => void),
onData?: (data: Table) => void,
headers?: { [key: string]: string }
): Promise
```
A callback may be passed either as the second argument or as `onData`.
In this example, we retrieve all 10,000 suppliers from the TPCH Suppliers table. This query retrieves all suppliers in a single call:
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient({ apiKey: process.env.API_KEY! });
const query = `
SELECT s_suppkey, s_name
FROM tpch.supplier
`;
const allSuppliers = await spiceClient.sql(query);
allSuppliers.toArray().forEach((row) => {
processSupplier(row);
});
```
This call will wait for the promise returned by `sql()` to complete, returning all 10,000 suppliers.
Alternatively, data can be processed as it is streamed to the SDK. Provide a callback function to the `onData` parameter, which will be called with every partial set of data streamed to the SDK:
```javascript
import { SpiceClient } from "@spiceai/spice";
const spiceClient = new SpiceClient({ apiKey: process.env.API_KEY! });
const query = `
SELECT s_suppkey, s_name
FROM tpch.supplier
`;
await spiceClient.sql(query, undefined, (partialData) => {
partialData.toArray().forEach((row) => {
processSupplier(row);
});
});
```
{% hint style="info" %}
On the Arrow Flight transport the first message carries the schema and is not passed to `onData`; each subsequent message is delivered as its own `Table`. The promise still resolves with the complete result assembled from every chunk.
{% endhint %}
# Go SDK
Golang SDK for Spice.ai
The [Go SDK](https://github.com/spiceai/gospice) `gospice` is the easiest way to query [Spice.ai](https://spice.ai) from Go.
It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to efficiently stream data to the client and [Apache Arrow](https://arrow.apache.org/) Records as data frames.
GoDocs are available at [pkg.go.dev/github.com/spiceai/gospice/v8](https://pkg.go.dev/github.com/spiceai/gospice/v8).
### Requirements
* [Go 1.24](https://go.dev/doc/go1.24) (or later)
### Installation
Get the **gospice** package.
```bash
go get github.com/spiceai/gospice/v8@latest
```
{% hint style="warning" %}
The module path carries the major version. Requesting `gospice/v7` resolves an older major release, not the latest.
{% endhint %}
### Usage
1\. Import the package.
```go
import gospice "github.com/spiceai/gospice/v8"
```
2\. Create a `SpiceClient` by providing your API key. Get your free API key at [spice.ai](https://spice.ai/).
```go
spice := gospice.NewSpiceClient()
defer func() { _ = spice.Close() }()
```
`Close()` releases the Flight and HTTP connections and must be called.
3\. Initialize the `SpiceClient`.
```go
if err := spice.Init(
gospice.WithApiKey(ApiKey),
gospice.WithSpiceCloudAddress(),
); err != nil {
panic(fmt.Errorf("error initializing SpiceClient: %w", err))
}
```
The options passed to `Init` are package-level functions on `gospice`, not methods on the client:
| Option | Description |
| ------------------------- | ---------------------------------------------------------------------------------- |
| `WithApiKey(key)` | App API key, in `appId\|secret` form. |
| `WithSpiceCloudAddress()` | Connect to Spice.ai Cloud. |
| `WithFlightAddress(addr)` | Arrow Flight address. A `grpc://` prefix selects plaintext; otherwise TLS is used. |
| `WithHttpAddress(addr)` | HTTP address, used for health checks and dataset refreshes. |
| `WithUserAgent(ua)` | Prepends to the reported user agent. |
4\. Execute a query and get back an [Apache Arrow Record Reader](https://pkg.go.dev/github.com/apache/arrow-go/v18/arrow/array#RecordReader).
```go
reader, err := spice.Sql(context.Background(), "SELECT * FROM tpch.lineitem LIMIT 10")
if err != nil {
panic(fmt.Errorf("error querying: %w", err))
}
defer reader.Release()
```
5\. Iterate through the reader to access the records.
```go
for reader.Next() {
record := reader.RecordBatch()
defer record.Release()
fmt.Println(record)
}
```
### Parameterized queries
`SqlWithParams` binds positional `$1`, `$2` placeholders:
```go
reader, err := spice.SqlWithParams(
context.Background(),
"SELECT * FROM tpch.lineitem WHERE l_quantity > $1 LIMIT 10",
10,
)
```
### Usage with local Spice runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally.
```go
spice := gospice.NewSpiceClient()
defer func() { _ = spice.Close() }()
if err := spice.Init(
gospice.WithHttpAddress("http://127.0.0.1:8090"),
); err != nil {
panic(fmt.Errorf("error initializing SpiceClient: %w", err))
}
```
{% hint style="info" %}
`NewSpiceClient()` defaults the Flight address to the local runtime but the HTTP address to Spice.ai Cloud. Pass `WithHttpAddress` as above so health checks and dataset refreshes reach the local runtime.
{% endhint %}
Or using a custom flight address:
```go
spice := gospice.NewSpiceClient()
defer func() { _ = spice.Close() }()
if err := spice.Init(
gospice.WithFlightAddress("grpc://localhost:50052"),
); err != nil {
panic(fmt.Errorf("error initializing SpiceClient: %w", err))
}
```
### Default endpoints
| Target | Arrow Flight | HTTP |
| -------------- | ------------------------ | ------------------------- |
| Spice.ai Cloud | `flight.spiceai.io:443` | `https://data.spiceai.io` |
| Local runtime | `grpc://localhost:50051` | `http://localhost:8090` |
These can also be set with the `SPICE_FLIGHT_URL`, `SPICE_HTTP_URL`, `SPICE_LOCAL_FLIGHT_URL`, and `SPICE_LOCAL_HTTP_URL` environment variables.
### Health checks
`IsSpiceHealthy(ctx)` and `IsSpiceReady(ctx)` check the HTTP endpoint's `/health` and `/v1/ready` routes and return a boolean.
### Example
Run `go run .` to execute a sample query and print the results to the console.
### Connection retry
The `SpiceClient` implements connection retry mechanism (3 attempts by default). The number of attempts can be configured via `SetMaxRetries`:
```go
spice := gospice.NewSpiceClient()
spice.SetMaxRetries(5) // Setting to 0 will disable retries
```
Retries are performed for connection and system internal errors. It is the SDK user's responsibility to properly handle other errors, for example `RESOURCE_EXHAUSTED (HTTP 429)`.
### Contributing
Contribute to or file an issue with the `gospice` library at:
# Rust SDK
Rust SDK for Spice.ai
The Rust SDK crate [`spiceai`](https://crates.io/crates/spiceai) is the easiest way to query [Spice.ai](https://spice.ai) from Rust. It is developed in the [spice-rs](https://github.com/spiceai/spice-rs) repository.
It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to efficiently stream data to the client and [Apache Arrow](https://arrow.apache.org/) Records as data frames.
### Requirements
* A Rust toolchain supporting edition 2021
* [Tokio](https://tokio.rs/) — every client method is `async`
### Installation
Add Spice SDK
```bash
cargo add spiceai
```
Note the crate is named `spiceai`, while the repository is named `spice-rs`. All imports use `spiceai`.
### Usage
1\. Create a client by providing your API key to `ClientBuilder`. Get your free API key at [spice.ai](https://spice.ai/).
```rust
use spiceai::ClientBuilder;
#[tokio::main]
async fn main() {
let client = ClientBuilder::new()
.api_key("API_KEY")
.use_spiceai_cloud()
.build()
.await
.unwrap();
}
```
`ClientBuilder` also accepts `.flight_url()` to set a custom endpoint, plus `.user_agent()`, `.max_retries()` (default 3), and `.cache_control()`. `.flight_url()` and `.use_spiceai_cloud()` are last-writer-wins, in either order.
For a Cloud connection with no other configuration, `spiceai::Client::new("API_KEY").await` is equivalent.
2\. Execute a query and get back a stream of Apache Arrow record batches.
```rust
let mut flight_data_stream = client.query("SELECT * FROM tpch.lineitem LIMIT 10;").await.expect("Error executing query");
```
The binding must be `mut`, because advancing the stream takes a mutable borrow.
3\. Iterate through the stream to access the records. `StreamExt` must be in scope; the crate re-exports it.
```rust
use spiceai::StreamExt;
while let Some(batch) = flight_data_stream.next().await {
match batch {
Ok(batch) => {
/* process batch */
println!("{:?}", batch)
},
Err(e) => {
/* handle error */
},
};
}
```
### Parameterized queries
`query_with_params` binds parameters supplied as an Arrow `RecordBatch` whose fields are named `$1`, `$2`, and so on.
### Usage with local Spice runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally. `ClientBuilder` defaults to the local runtime, so no endpoint is needed:
```rust
use spiceai::ClientBuilder;
#[tokio::main]
async fn main() {
let client = ClientBuilder::new()
.build()
.await
.unwrap();
let data = client.query("SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;").await;
}
```
### Default endpoints
| Target | Arrow Flight |
| -------------- | --------------------------- |
| Spice.ai Cloud | `https://flight.spiceai.io` |
| Local runtime | `http://localhost:50051` |
`ClientBuilder` defaults to the local runtime; `Client::new(api_key)` defaults to Spice.ai Cloud. Override either with `.flight_url()`.
### Contributing
Contribute to or file an issue with the `spiceai` crate at:
# Java SDK
The [`Java SDK`](https://github.com/spiceai/spice-java) is the easiest way to query the Spice Cloud Platform from Java.
It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to efficiently stream data to the client and [Apache Arrow](https://arrow.apache.org/) Records as data frames.
### Supported Java Versions
The library targets **Java 11** and above, and is tested against the following implementations:
* Microsoft OpenJDK 11, 17, 21
* Eclipse Temurin 21, 23, 24
* Oracle JDK 17, 21, 23, 24, 25
### Installation
{% tabs %}
{% tab title="Maven" %}
```xml
ai.spice
spiceai
0.7.0
compile
```
{% endtab %}
{% tab title="Gradle" %}
```groovy
implementation 'ai.spice:spiceai:0.7.0'
```
{% endtab %}
{% endtabs %}
### Usage
1\. Import the package.
```java
import ai.spice.SpiceClient;
```
2\. Create a `SpiceClient` by providing your API key. Get your free API key at [spice.ai](https://spice.ai/).
`SpiceClient` implements `AutoCloseable`, so use it in a try-with-resources block.
```java
try (SpiceClient spice = SpiceClient.builder()
.withApiKey(ApiKey)
.withSpiceCloud()
.build()) {
// query here
}
```
The builder also accepts `withFlightAddress(URI)`, `withHttpAddress(URI)`, `withUserAgent(String)`, `withMaxRetries(int)`, and `withArrowMemoryLimitMB(long)`. The API key must be in `appId|key` form.
For mutual TLS, connection pooling, and query deadlines, see [Mutual TLS](#mutual-tls) and [Performance tuning](#performance-tuning).
3\. Execute a query and get back a [`FlightStream`](https://arrow.apache.org/docs/java/reference/org.apache.arrow.flight.core/org/apache/arrow/flight/FlightStream.html).
```java
FlightStream stream = spice.query("SELECT * FROM tpch.lineitem LIMIT 10");
```
4\. Iterate through the `FlightStream` to access the records.
```java
while (stream.next()) {
try (VectorSchemaRoot batches = stream.getRoot()) {
System.out.println(batches.contentToTSVString());
}
}
```
Check [full example](https://github.com/spiceai/spice-java/blob/trunk/src/main/java/ai/spice/example/ExampleSpiceCloudPlatform.java) to learn more.
### Parameterized queries
`queryWithParams(String sql, Object... params)` binds positional `$1`, `$2` placeholders and returns an `ArrowReader`, which the caller closes.
```java
try (ArrowReader reader = spice.queryWithParams(
"SELECT * FROM tpch.lineitem WHERE l_quantity > $1 LIMIT 10", 10)) {
while (reader.loadNextBatch()) {
System.out.println(reader.getVectorSchemaRoot().contentToTSVString());
}
}
```
Parameterized queries run on Apache Arrow Flight SQL prepared statements over the same connections as `query()`, so they inherit the client's TLS, retry, and keep-alive settings.
Prepared statements are cached and reused, which removes the prepare round trips from every repeated query. The cache holds 64 statements by default; `withPreparedStatementCacheSize(0)` disables caching:
```java
SpiceClient spice = SpiceClient.builder()
.withPreparedStatementCacheSize(128)
.build();
```
Failures surface as a `FlightRuntimeException` wrapped in an `ExecutionException`.
### Usage with local Spice.ai OSS runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally. The builder defaults to the local runtime:
```java
try (SpiceClient spice = SpiceClient.builder()
.build()) {
// query here
}
```
Or using custom flight address:
```java
try (SpiceClient spice = SpiceClient.builder()
.withFlightAddress(new URI("grpc://my_remote_spice_instance:50051"))
.build()) {
// query here
}
```
{% hint style="warning" %}
Always include an explicit port. An `http://` or `https://` address is rewritten to a `grpc+tcp://` or `grpc+tls://` address using the URI's port, so an address without one resolves to an invalid port.
{% endhint %}
Check [Spice OSS documentation](https://docs.spiceai.org/sdks/java) or [Java SDK Sample](https://github.com/spiceai/samples/tree/trunk/client-sdk/spice-java-sdk-sample) to learn more
### Default endpoints
| Target | Arrow Flight | HTTP |
| -------------- | ------------------------------- | ------------------------- |
| Spice.ai Cloud | `https://flight.spiceai.io:443` | `https://data.spiceai.io` |
| Local runtime | `http://localhost:50051` | `http://localhost:8090` |
These can also be set with the `SPICE_FLIGHT_URL` and `SPICE_HTTP_URL` environment variables.
### Mutual TLS
Connect to a Spice deployment that requires mutual TLS, or one presenting a certificate signed by a private certificate authority. All three options take a path to a PEM file:
```java
try (SpiceClient spice = SpiceClient.builder()
.withFlightAddress(new URI("grpc+tls://spice.example-org.com:50051"))
.withTlsClientCertFile("/etc/spice/client.crt")
.withTlsClientKeyFile("/etc/spice/client.key")
.withTlsRootCertFile("/etc/spice/ca.crt")
.build()) {
// query here
}
```
| Option | Purpose |
| ----------------------- | ----------------------------------------------- |
| `withTlsClientCertFile` | Client certificate presented to the server |
| `withTlsClientKeyFile` | Private key for the client certificate |
| `withTlsRootCertFile` | Certificate authority used to verify the server |
The settings apply to Flight queries, parameterized queries, and HTTP operations. `withTlsRootCertFile` can be used on its own to trust a private CA without presenting a client certificate.
### Health and runtime status
Three methods report runtime state. None throws when the runtime is unreachable, so they are safe to poll from a health probe.
```java
boolean healthy = spice.isHealthy(); // runtime is responding
boolean ready = spice.isReady(); // datasets are loaded and queryable
for (ConnectionDetails details : spice.runtimeStatus()) {
System.out.printf("%s (%s): %s%n",
details.getName(), details.getEndpoint(), details.getStatus());
}
```
`isHealthy()` suits a liveness check, and `isReady()` a readiness check — a runtime can respond before its datasets finish loading.
`runtimeStatus()` returns per-connection detail and throws `ExecutionException` on failure. Each `ConnectionDetails` exposes `getName()`, `getEndpoint()`, `getStatus()`, `getRawStatus()`, and `isReady()`. `getStatus()` returns a `ComponentStatus`: `INITIALIZING`, `READY`, `DISABLED`, `ERROR`, `REFRESHING`, `SHUTTING_DOWN`, `NOT_LOADED`, or `UNKNOWN`.
{% hint style="info" %}
`UNKNOWN` means the runtime reported a status this version of the SDK does not recognize. Use `getRawStatus()` to read the value the runtime sent.
{% endhint %}
### Performance tuning
```java
SpiceClient spice = SpiceClient.builder()
.withChannelCount(4)
.withQueryTimeout(Duration.ofSeconds(30))
.withPreparedStatementCacheSize(128)
.build();
```
| Option | Default | Description |
| ------------------------------------- | ------- | ----------------------------------------------------------------------------- |
| `withChannelCount(int)` | `1` | Size of the round-robin gRPC connection pool, for highly concurrent workloads |
| `withQueryTimeout(Duration)` | none | Deadline for query planning, prepare, and bind calls; must be positive |
| `withPreparedStatementCacheSize(int)` | `64` | Prepared statements retained for reuse; `0` disables caching |
`withQueryTimeout` bounds the planning and binding calls, not the streaming of results — a long-running result stream is not cut short by it.
### Connection pooling with HikariCP
Spice can also be reached over JDBC using the Apache Arrow Flight SQL JDBC driver, which allows pooling connections with HikariCP. See the [spice-java README](https://github.com/spiceai/spice-java#readme) for a worked example.
### Refreshing a dataset
`refreshDataset(String dataset)` triggers a refresh of an accelerated dataset, optionally taking a `RefreshOptions`.
### Connection retry
The `SpiceClient` implements connection retry mechanism (3 attempts by default). The number of attempts can be configured with `withMaxRetries`:
```java
SpiceClient client = SpiceClient.builder()
.withMaxRetries(5) // Setting to 0 will disable retries
.build();
```
Retries are performed for connection and system internal errors. It is the SDK user's responsibility to properly handle other errors, for example `RESOURCE_EXHAUSTED (HTTP 429)`.
Attempts use exponential backoff with random jitter, capped at 10 seconds per wait. The jitter keeps a fleet of clients from retrying in lockstep, and the backoff lets the default three attempts span a load-balancer failover or a runtime restart.
{% hint style="info" %}
Because retries now wait between attempts, a query against an unreachable runtime takes longer to report failure. Use `withMaxRetries(0)` where a fast failure matters more than surviving a restart.
{% endhint %}
### Re-authentication
An expired handshake token is detected and renewed automatically: the client re-handshakes once and retries the request, so long-lived `SpiceClient` instances keep working without calling `reset()`.
`reset()` retires existing connections gracefully — in-flight queries and open result streams run to completion while new queries use fresh connections.
### Contributing
Contribute to or file an issue with the `spice-java` library at:
# Dotnet SDK
Dotnet SDK for Spice.ai
The [Dotnet SDK](https://github.com/spiceai/spice-dotnet) `SpiceAI` is the easiest way to query [Spice.ai](https://spice.ai) from Dotnet.
It uses [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) to efficiently stream data to the client and [Apache Arrow](https://arrow.apache.org/) Records as data frames.
### Requirements
The package targets .NET Standard 2.0, .NET 8.0, .NET 9.0, and .NET 10.0. .NET 8.0 or later is recommended.
### Installation
Add Spice SDK
```bash
dotnet add package SpiceAI
```
### Usage
1. Create a `SpiceClient` by providing your API key to `SpiceClientBuilder`. Get your free API key at [Spice.ai](https://spice.ai/).
`SpiceClient` implements `IDisposable`, so declare it with `using`.
```csharp
using Spice;
using var client = new SpiceClientBuilder()
.WithSpiceCloud("API_KEY")
.Build();
```
{% hint style="warning" %}
`WithSpiceCloud` takes the **API key**, not a URL, and applies the Spice.ai Cloud endpoints itself. The key must be in `appId|key` form. To connect to a custom endpoint, use `WithFlightAddress` instead.
{% endhint %}
The builder also accepts `WithApiKey(string)`, `WithFlightAddress(string)`, `WithHttpAddress(string)`, `WithMaxRetries(int)` (default 3), `WithUserAgent(string)`, and `WithTls(bool)`.
2. Execute a query and get back an Apache Arrow [Flight Client Record Batch Stream Reader](https://github.com/apache/arrow/blob/67bbf846d0d47075e1711b3fb4cf8fb05c74bd09/csharp/src/Apache.Arrow.Flight/Client/FlightClientRecordBatchStreamReader.cs#L22).
```csharp
var result = await client.Query("SELECT * FROM tpch.lineitem LIMIT 10;");
```
3. Iterate through the reader to access the records.
```csharp
var enumerator = result.GetAsyncEnumerator();
while (await enumerator.MoveNextAsync())
{
var batch = enumerator.Current;
// Process batch
}
```
### Parameterized queries
`QueryWithParams(string sql, params object?[] parameters)` binds positional `$1`, `$2` placeholders:
```csharp
var data = await client.QueryWithParams(
"SELECT * FROM tpch.lineitem WHERE l_quantity > $1 LIMIT 10;", 10);
var batch = await data!.ReadNextRecordBatchAsync();
```
{% hint style="info" %}
Parameters are positional only. Named placeholders such as `:product_id` are not supported.
{% endhint %}
### Usage with local Spice runtime
Follow the [quickstart guide](https://github.com/spiceai/spiceai?tab=readme-ov-file#%EF%B8%8F-quickstart-local-machine) to install and run spice locally. The builder defaults to the local runtime:
```csharp
using Spice;
using var client = new SpiceClientBuilder()
.Build();
var data = await client.Query("SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;");
```
Or using a custom flight address:
```csharp
using var client = new SpiceClientBuilder()
.WithFlightAddress("http://localhost:50051")
.Build();
```
### Default endpoints
| Target | Arrow Flight | HTTP |
| -------------- | ------------------------------- | ------------------------- |
| Spice.ai Cloud | `https://flight.spiceai.io:443` | `https://data.spiceai.io` |
| Local runtime | `http://localhost:50051` | `http://localhost:8090` |
The Cloud endpoints can be set with `SPICE_FLIGHT_URL` and `SPICE_HTTP_URL`, and the local endpoints with `SPICE_LOCAL_FLIGHT_URL` and `SPICE_LOCAL_HTTP_URL`.
### Refreshing a dataset
`RefreshDatasetAsync(string datasetName)` triggers a refresh of an accelerated dataset.
### Contributing
Contribute to or file an issue with the `spice-dotnet` library at:
# Agentic AI Apps
Using Spice.ai for Agentic AI Applications
Build intelligent autonomous agents that act contextually by grounding AI models in secure, full-knowledge datasets with fast, iterative feedback loops.
Spice.ai helps in building intelligent autonomous agents by leveraging several key features:
### Federated SQL Query
Spice.ai enables federated querying across databases, data warehouses, and lakes. With advanced query push-down optimizations, it ensures efficient retrieval and processing of data across disparate sources, reducing latency and operational complexity. [Learn more about Federated SQL Query](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/federated-queries/index.md). For practical implementation, refer to the [Federated SQL Query recipe](https://github.com/spiceai/cookbook/blob/trunk/federation/README.md).
### Data Acceleration & Materialization with Change Data Capture (CDC)
Spice.ai materializes application-specific datasets close to the point of use, reducing query and thus retrieval times, and infrastructure costs. It supports Change Data Capture (CDC), keeping materialized data sets up-to-date with minimal overhead and enabling real-time, reliable data access. [Learn more about Data Acceleration](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/data-acceleration/index.md). See the [DuckDB Data Accelerator recipe](https://github.com/spiceai/cookbook/blob/trunk/duckdb/accelerator/README.md) for an example.
### AI Gateway
Integrate AI into your applications with Spice.ai’s AI Gateway. It supports hosted models like OpenAI and Anthropic and local models such as OSS Llama and NVIDIA NIM. Fine-tuning and model distillation are simplified, helping faster cycles of development and deployment. [Learn more about AI Gateway](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/large-language-models/index.md). Refer to the [Running Llama3 Locally recipe](https://github.com/spiceai/cookbook/blob/trunk/llama/README.md) for details.
### Search with Vector Similarity Search (VSS)
Spice.ai provides advanced search capabilities, including vector similarity search (VSS), enabling efficient retrieval of unstructured data, embeddings, and AI model outputs. This is critical for applications like RAG and intelligent search systems. [Learn more about Vector Similarity Search](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/search/index.md). For implementation, see the [Searching GitHub Files recipe](https://github.com/spiceai/cookbook/blob/trunk/search_github_files/README.md).
### Semantic Model for AI
Built-in semantic models allow Spice.ai to align AI operations with enterprise data, ensuring that applications are grounded in contextual, full-knowledge datasets. This enhances the accuracy and reliability of AI outputs while reducing risks of irrelevant or untrustworthy results. [Learn more about Semantic Model for AI](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/semantic-model/index.md).
### Monitoring and Observability
Spice.ai includes robust monitoring and observability tools tailored for AI applications. These tools provide end-to-end visibility into data flows and AI workflows, LLM-specific observability to monitor model performance, track usage, and manage drift, and security and compliance auditing for data and model interactions. [Learn more about Monitoring and Observability](https://github.com/spiceai/docs/blob/trunk/spiceaidocs/docs/features/monitoring/index.md).
# Analytics Replica
Attach a columnar analytics replica to an operational database without ETL or migration
The **analytics replica pattern** attaches a dedicated columnar node to an operational database and lets it absorb the analytical query load. The primary keeps serving transactions. Analytical questions — which customers churned today, which orders are stuck, how revenue moved this hour — run against a replica that stays seconds behind.
It is the shortest path from an operational database to analytical and AI workloads, because nothing has to be migrated and no pipeline has to be built.
### The problem
Analytical queries and transactional queries want opposite things from a database. Transactions want narrow row lookups and short-lived locks. Analytics wants wide scans over large ranges. Running both on one primary means the reporting query and the checkout path compete for the same buffer pool and CPU.
The established workarounds each trade one problem for another:
| Approach | What it costs |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
| **ETL pipeline** | Data arrives minutes or hours late, and the pipeline itself becomes infrastructure to build, schedule, monitor, and repair |
| **Read replica** | Removes load from the primary, but runs the same row-oriented engine — analytical scans are just as slow |
| **HTAP database** | Requires migrating off the current database, and recouples transactional and analytical failure domains |
| **Data warehouse** | Strong for analytics, but reached through a pipeline, so it inherits the latency and the operational burden |
### How it works
An analytics replica connects to the operational database, loads an initial snapshot, then uses change data capture (CDC) to apply committed changes continuously from the source's native changelog — the PostgreSQL write-ahead log, a MongoDB change stream, a DynamoDB stream. There is no batch interval; a committed change is queryable within seconds.
The replica stores data in a columnar format on its own storage and compute, so scans are fast and the load never reaches the primary. Queries run through the same federated SQL interface as the rest of the platform, which means a replicated table can be joined against a data lake, a warehouse, or another operational system in a single query.
```yaml
datasets:
- from: postgres:public.orders
name: orders
params:
pg_host: postgres.example-org.com
pg_user: spice
pg_pass: ${secrets:pg_pass}
pg_db: myapp
acceleration:
enabled: true
engine: cayenne
mode: file
refresh_mode: changes
primary_key: id
on_conflict:
id: upsert
```
See [Database Replication and CDC](/features/database-replication-and-cdc) for the supported sources, source prerequisites, and full configuration reference.
### Why it differs from a read replica
A read replica and an analytics replica solve different halves of the problem:
* A **read replica** moves load off the primary but keeps the row-oriented storage and execution engine, so a query scanning millions of rows is no faster than it was.
* An **analytics replica** changes the storage layout and the execution engine. Data is columnar, scans read only the columns a query touches, and segment statistics skip data that cannot match.
Both keep the primary healthy. Only one makes the analytical query fast.
### Incremental adoption
The pattern is adopted table by table. Replicate one table, point one dashboard or one agent at it, and leave everything else untouched. There is no cutover, no schema migration, and no change to how the application writes.
That property matters most when the destination is an AI agent. Grounding an agent in production data usually stalls on the pipeline needed to expose that data safely. Replicating the handful of tables the agent needs is a smaller commitment than building a warehouse feed, and the agent queries data that is seconds old rather than a day stale.
### Governing access
Because queries reach the replica rather than the operational database, access is governed at the replica:
* Agents and applications authenticate to Spice instead of holding database credentials, so the primary's credentials never leave the infrastructure that owns them.
* [Policy](https://github.com/spicehq/docs/tree/trunk/enterprise/features/policy.md) rules apply row-level filters and column masking at query time, based on the identity of the caller.
* Queries are recorded, so what an agent read is auditable after the fact.
### Related
* [Database Replication and CDC](/features/database-replication-and-cdc) — configuration, supported sources, and prerequisites
* [Federated SQL Query](/features/federated-sql-query) — joining a replica against other systems
* [Database CDN](/use-cases/database-cdn) — colocating a hot working set with an application
* [Agentic AI Apps](/use-cases/agentic-ai-apps) — grounding agents in operational data
# Database CDN
Use Spice as a CDN for Databases
### Enhancing data application performance
Colocate a local working set of hot data with data applications and frontends to serve more concurrent requests and users with faster page loads and data updates.
[Try the CQRS Cookbook](https://github.com/spiceai/cookbook/tree/trunk/cqrs#readme)
### Increasing application resilience
Maintain local replicas of data with the application to significantly enhance application resilience and availability.
### Improving dashboard, analytics, and BI performance
Create a materialization layer for visualization tools like Power BI, Tableau, or Superset to achieve faster, more responsive dashboards without incurring massive compute costs.
[Watch the Apache Superset demo](https://github.com/spiceai/cookbook/blob/trunk/sales-bi/README.md)
# Data Lakehouse
Create a Data Lakehouse using Spice
### Accessing data across multiple, disparate data sources
Perform [federated SQL queries](https://github.com/spiceai/docs/blob/trunk/features/federated-queries) across databases, data warehouses, and data lakes using [Data Connectors](https://github.com/spiceai/docs/blob/trunk/components/data-connectors).
### Migrating from legacy data systems
Spice provides a drop-in solution that offers a single, unified endpoint to multiple data systems without requiring changes to the application.
# Enterprise Search
Use Spice for Enterprise Search and Retrieval
### Vector similarity search across disparate and legacy data systems
Enterprises face the challenge of accessing data from various disparate and legacy systems to provide AI with comprehensive context. Speed is crucial for this process to be effective.
Spice offers a fast knowledge index into both structured and unstructured data, enabling efficient vector similarity search across multiple data sources. This ensures that AI applications have access to the necessary data for accurate and timely responses.
# Enterprise RAG
Use Spice for Retrieval-Augmented-Generation (RAG)
Use Spice to access data across various data sources for Retrieval-Augmented-Generation (RAG).
Spice enables developers to combine structured data via SQL queries and unstructured data through built-in vector similarity search. This combined data can then be fed to large language models (LLMs) through a native AI gateway, enhancing the models' ability to generate accurate and contextually relevant responses.
# Privacy Policy
Spice AI, Inc.
Last updated: September 13, 2023
**Spice AI, Inc.**, a Delaware corporation, including its affiliates ("**Company**”, “**we**”, “**us**”, or “**our**”) is committed to maintaining robust privacy protections for you, whether personally or on behalf of an entity (“**you**” or “**user**”).\
\
**Our Privacy Policy (“Privacy Policy” or this “Agreement”) is designed to help you understand how we collect, use, and safeguard the information you provide to us and to assist you in making informed decisions when using our Service. For purposes of this Agreement, “Site” refers to our website, blogs, social media accounts, or apps in which Company’s service can be accessed in** which users can view the website and subscribe to and use our software, products, and/or services (each a “**Service**,” and collectively the “**Services**”).
Your privacy is important to us. So, please take the time to carefully read this Privacy Policy. If you have any questions on how we collect, use, protect, or otherwise handle your Personal Information (as further described below), please contact us.
Unless otherwise defined in this Privacy Policy, terms used in this Privacy Policy have the same meaning as in our Terms of Service. By using the Services, you consent to our collection, storage, use and disclosure of your Personal Information under the terms set forth herein.
### I. INFORMATION WE COLLECT
\
We collect “**Non-Personal Information**” and “**Personal Information**.” Non-Personal Information includes information that cannot be used to personally identify you, such as anonymous usage data, general demographic information we may collect, referring/exit pages and URLs, platform types, preferences you submit, and preferences that are generated based on the data you submit and number of clicks. Personal Information includes your email, phone number, address, and any other information you submit to us through the Site. We collect information related to you for the purpose of carrying out the Services. This information includes but is not limited to: name, address, email address, telephone number, bank account number, credit or debit card number, or other financial information.
We collect “**Commercial Information**” and “**Internet or other electronic network activity information**” such as records of personal property, products or services purchased, obtained, or considered, or other purchasing or consuming histories, or tendencies, browsing history, search history, or information regarding a consumer’s interaction with an Internet website, application, or advertisement.
1. *Information collected via Technology*. In an effort to improve the quality of the Services, we track information provided to us by your browser or by our software application when you view or use the Services, such as the website you came from (known as the “referring URL”), the type of browser you use, the device from which you connected to the Services, the time and date of access, and other information that does not personally identify you. We track this information using cookies, or small text files which include an anonymous unique identifier. Cookies are sent to a user’s browser from our servers and are stored on the user’s computer hard drive. Sending a cookie to a user’s browser enables us to collect Non-Personal information about that user and keep a record of the user’s preferences when utilizing our Services, both on an individual and aggregate basis. Company may use both persistent and session cookies; persistent cookies remain on your computer after you close your session and until you delete them, while session cookies expire when you close your browser.
2. *Information you provide us by registering for an account*. In addition to the information provided automatically by your browser when you visit the Site, to become a customer to the Service, you will need to create a personal profile. Creating a profile and registering with the Service may require you to provide your name, phone number, email address, and other personal information and creating a username and a password. By registering, you are authorizing us to collect, store, and use your email address in accordance with this Privacy Policy. \\
### II. HOW WE USE AND SHARE INFORMATION
#### Personal Information:
Except as otherwise stated in this Privacy Policy, we do not sell, trade, rent, or otherwise share for marketing purposes your Personal Information with third parties without your consent. We do share Personal Information with vendors who are performing services for Company, such for third party integrations. Those vendors use your Personal Information only at our direction and in accordance with our Privacy Policy. In general, the Personal Information you provide to us is used to help us communicate with you. For example, we use Personal Information to contact users in response to questions, solicit feedback from users, provide technical support, and inform users about promotional offers.\
\
We may send you offers and information related to our Services from which you may opt out via the unsubscribe function in the email. We may also send you important account related information, from which you cannot opt out.
#### Non-Personal Information:
In general, we use Non-Personal Information to help us improve the Services and customize the user experience. We also aggregate Non-Personal Information in order to track trends and analyze use patterns on the Site. This Privacy Policy does not limit in any way our use or disclosure of Non-Personal Information and we reserve the right to use and disclose such Non-Personal Information to our partners, advertisers and other third parties at our discretion. In the event we undergo a business transaction such as a merger, acquisition by another company, or sale of all or a portion of our assets, your Personal Information may be among the assets transferred. You acknowledge and consent that such transfers may occur and are permitted by this Privacy Policy, and that any acquirer of our assets may continue to process your Personal Information as set forth in this Privacy Policy. If our information practices change at any time in the future, we will post the policy changes to the Site so that you may opt out of the new information practices. We suggest that you check the Site periodically if you are concerned about how your information is used. \\
### III. HOW WE PROTECT INFORMATION
We implement security measures designed to protect your information from unauthorized access. Your account is protected by your account password and we urge you to take steps to keep your personal information safe by not disclosing your password and by logging out of your account after each use. However, these measures do not guarantee that your information will not be accessed, disclosed, altered or destroyed by breach of such firewalls and secure server software. By using our Services, you acknowledge that you understand and agree to assume these risks. \\
### IV. YOUR RIGHTS REGARDING THE USE OF YOUR PERSONAL INFORMATION
You have the right at any time to prevent us from contacting you for marketing purposes. When we send a promotional communication to a user, the user can opt out of further promotional communications by following the unsubscribe instructions provided in each promotional e-mail. Please note that notwithstanding the promotional preferences you indicate by either unsubscribing or opting out in the Settings section of the Site, we may continue to send you administrative emails including, for example, periodic updates to our Privacy Policy, or other important account information. \\
### V. CUSTOMER DATA
”Customer Data” means all information processed or stored on the Service by Customer or on Customer’s behalf, as well as any information derived from such information. Customer Data includes, without limitation: (1) information provided on the Service; (2) information provided to Company by other users or by other third parties; and (3) personally identifiable information from such customers, users, or other third parties.\
\
(a) *Access, Use, & Legal Compulsion*. Unless it receives Customer’s prior written consent, Company:
* shall not access, process, or otherwise use Customer Data other than as necessary to facilitate the Services;
* shall not give any of its employees access to Customer Data except to the extent that such individual needs access to facilitate performance under this Agreement and is subject to a reasonable written nondisclosure agreement with Company protecting such data, with terms reasonably consistent with those of this Section; and
* shall not give any third-party access to Customer Data. Notwithstanding the foregoing, Company may disclose Customer Data as required by applicable law or by proper legal or governmental authority. Company shall give Customer prompt notice of any such legal or governmental demand and reasonably cooperate with Customer in any effort to seek a protective order or otherwise to contest such required disclosure, at Customer’s expense.
\
(b) *Customer’s Rights*. Unless otherwise stated in this Agreement, Customer possesses and retains all right, title, and interest in and to Customer Data, and Company’s use and possession thereof is solely on Customer’s behalf.
### VI. LINKS TO OTHER WEBSITES
As part of the Services, we may provide links to or compatibility with other websites or applications. However, we are not responsible for the privacy practices employed by those websites or the information or content they contain. This Privacy Policy applies solely to information collected by us through the Site and the Service. Therefore, this Privacy Policy does not apply to your use of a third-party website accessed by selecting a link on our Site or via our Services. To the extent that you access or use the Service through or on another website or application, then the privacy policy of that other website or application will apply to your access or use of that site or application. We encourage our users to read the privacy statements of other websites before proceeding to use them. \\
### VII. CHANGES TO OUR PRIVACY POLICY
Company reserves the right to change this policy and our Terms of Service at any time. We will notify you of significant changes to our Privacy Policy by sending a notice to the primary email address specified in your account or by placing a prominent notice on our site. Significant changes will go into effect 30 days following such notification. Non-material changes or clarifications will take effect immediately. You should periodically check the Site and this privacy page for updates. \\
### VIII. CALIFORNIA PRIVACY RIGHTS
If you are a California resident, you may have additional privacy rights. For a complete discussion of such rights, please see our California Rights and Disclosures.
### IX. CONTACT US
\
If you have any questions regarding this Privacy Policy or the practices of this Site, please contact us by sending an email to .
\[End of Privacy Policy.]
## **Spice AI, Inc.**
### **Privacy Policy**
\*\*\*\*
### **California Rights and Disclosures**
Spice AI, Inc., including its affiliates (“**Spice AI, Inc.**”, “**we**”, “**us**”, or “**our**”) is committed to maintaining robust privacy protections for you, whether personally or on behalf of an entity (“**you**”, “**customer**”, “**user**”, or “**your**”).
In addition to the rights listed in our Privacy Policy applicable to all users, in limited circumstances, California residents may have the following rights over certain information collected, used, and shared pursuant to our Privacy Policy.\
\
**For purposes hereof, “Site” refers to our website, blogs, social media accounts, or apps in which Spice AI, Inc.’s service can be accessed in** which users can view the website and subscribe to and use our software, products, and services (“**Service**,” collectively the “**Services**”).
Please take the time to carefully read this Privacy Policy as your privacy is important to us. If you have any questions on how we collect, use, protect, or otherwise handle your Personal Information (as further described below), please contact us using the contact information below.
Unless otherwise defined in this Privacy Policy, terms used in this Privacy Policy have the same meaning as in our Terms of Service. By using the Services, you consent to our collection, storage, use and disclosure of your Personal Information under the terms set forth herein.
### I. INFORMATION WE COLLECT, SELL, AND SHARE
We collect and share information as described in Section I and Section II of our Privacy Policy. As applicable to the categories of information defined in the California Consumer Privacy Act of 2018:
* In the preceding twelve (12) months, we have collected the following types of information: We collect identifiers such as name, alias, postal address, unique personal identifier, online identifier, IP address, email address, account name, social security number, driver’s license number, passport number, or other similar identifiers, customer records information such as name, signature, social security number, physical characteristics or description, address, telephone number, passport number, driver’s license or state identification card number, insurance policy number, education, employment, employment history, bank account number, credit or debit card number, other financial information, medical information, health insurance information, commercial information such as records of personal property, products or services purchased, obtained, or considered, or other purchasing or consuming histories or tendencies, Internet or other electronic network activity information such as browsing history, search history, and information regarding a consumer’s interaction with an Internet website, application, or advertisement, geolocation data, professional or employment-related information, education information, inferences to a consumer’s preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes.
* In the preceding twelve (12) months, we have sold the following types of information to third parties: N/A.
* In the preceding twelve (12) months, we have used the following types of information for business purposes (potentially including auditing interactions with consumers, security measures, debugging/repair, performing services, internal research for technology development, and quality and safety maintenance and verification): name, address, email address, height, weight, facial recognition, other biometric data, race, gender, education information, and medial or health insurance information.
### II. THIRD-PARTY SHARING
California Civil Code Section 1798.83 permits our customers who are California residents to request certain information regarding our disclosure of personal information to third parties for their direct marketing purposes. We do not share our customer's personal information with unaffiliated third parties for their own direct marketing purposes.
### III. USERS UNDER THE AGE OF 18
\*\*\*\*
California Business and Professions Code Section 22581 permits you, if you are a California resident under the age of 18, to view, correct, or remove information provided by you or publicly posted by you, by accessing your account or another product or service as applicable and editing/removing your personal information. You will need your password to access your personal account. You may also send us an email or written request asking us to remove certain posted content using the addresses in the Contact Information section of this document.
We will be happy to review, update, or remove information and/or content as appropriate. Residual copies of information and/or content that have been removed from your account and/or our platform may remain in our backup systems for approximately one month. We may still retain your information to resolve disputes, enforce our user agreements, or comply with legal requirements; in this case, your personal information will be blocked from use for any other purpose.
### IV. DO NOT TRACK SETTINGS
\*\*\*\*
California Business and Professions Code Section 22575(b) permits our customers who are California residents to be informed as to how we respond to web browser "Do Not Track" settings. As Do Not Track is a standard that is currently being developed, we do not take actions to respond to Do Not Track settings, and instead we adhere to the standards set out in this Privacy Policy. If you would like to find out more about Do Not Track, you may find the following link useful: [http://www.allaboutdnt.com/](http://www.allaboutdnt.com).
### V. OTHER RIGHTS
\*\*\*\*
* *Right to access personal information*. You may be entitled to receive the specific pieces of your personal information we have collected in the 12 months preceding your request.
* *Right to data portability*. You may be entitled to receive a copy of your electronic personal information in a readily usable format.
* *Right to know*. You may be entitled to receive information regarding the purposes for which we collected and shared personal information, the categories of personal information that we sold and the categories of third parties to whom the personal information was sold, as well as the categories of personal information that we disclosed for a business purpose in the 12 months preceding your request.
* *Right to delete*. You may be entitled to request that we delete the personal information that we have collected from you. We will use commercially reasonable efforts to honor your request, in compliance with applicable laws. Please note, however, that we may need to keep such information, such as for our legitimate business purposes or as required to comply with applicable law.
* *Right to opt-out*. You may be entitled to direct us to stop disclosing your personal information to share with third parties for monetary or other valuable consideration by visiting our “Do Not Sell My Personal Information” opt-out page in the event that we ever endeavor to share or sell information in the future.
### VI. HOW TO EXERCISE THESE RIGHTS
\*\*\*\*
Any requests to exercise User rights can be directed to the Owner through the contact details below. These requests can be exercised free of charge and will be addressed by the Owner as early as possible and always within one month. For all requests, please clearly state that the request is related to your “California Privacy Rights” and provide your name, street address, city, state, zip code, and an e-mail address or phone number where you may be contacted.
Before we can respond to your request, California law requires us to verify your identity using your personal information. If we (or third parties we engage to assist us) are not able to verify your request, we will contact you for more information. If we are unable to verify your identity after a good faith attempt, we may deny the request and, if so, will explain the basis for the denial.
You may designate someone to submit requests and act on your behalf as an authorized agent. To do so, you must provide us with written permission to allow your authorized agent to act on your behalf and, where appropriate, accept information about you. Either way, you and/or your authorized agent must provide sufficient information to us that would allow us to verify your identity.
### VII. CONTACT INFORMATION
\*\*\*\*
Spice AI, Inc.\
113 Cherry St\
PMB 26297\
Seattle, Washington 98104-2205
\*\*\*\*
\[End of California Rights and Disclosures.]
# Website Terms of Use
Last Updated: November 3, 2023
### Acceptance of the Terms of Use
These Terms of Use are entered into by and between you and Spice AI, Inc., a Delaware corporation ("Company," "we," or "us"). The following terms and conditions, (the "Terms of Use"), govern your access to and use of [spice.ai](http://www.spice.ai), [spice.xyz](https://spice.xyz), and [spicerack.org](https://spicerack.org), including any content, functionality, and services offered on or through spice.ai, spice.xyz, and spicerack.org, or any related subdomain (the "Website"), whether as a guest or a registered user.
Please read the Terms of Use carefully before you start to use the Website. **By using the Website, you accept and agree to be bound and abide by these Terms of Use and our** [**Privacy Policy**](/legal/privacy)**, incorporated herein by reference.** If you do not want to agree to these Terms of Use or the Privacy Policy, you must not access or use the Website.
This Website is offered and available to users who are 18 years of age or older. By using this Website, you represent and warrant that you are of legal age to form a binding contract with the Company and meet all of the foregoing eligibility requirements. If you do not meet all of these requirements, you must not access or use the Website.
### Changes to the Terms of Use
We may revise and update these Terms of Use from time to time at our sole discretion. All changes are effective immediately when we post them, and apply to all access to and use of the Website thereafter. However, any changes to the dispute resolution provisions set out in Governing Law and Jurisdiction will not apply to any disputes for which the parties have actual notice on or before the date the change is posted on the Website.
Your continued use of the Website following the posting of revised Terms of Use means that you accept and agree to the changes. You are expected to check this page from time to time so you are aware of any changes, as they are binding on you.
### Accessing the Website and Account Security
We reserve the right to withdraw or amend this Website, and any service or material we provide on the Website, in our sole discretion without notice. We will not be liable if for any reason all or any part of the Website is unavailable at any time or for any period. From time to time, we may restrict access to some parts of the Website, or the entire Website, to users, including registered users.
You are responsible for both:
* Making all arrangements necessary for you to have access to the Website.
* Ensuring that all persons who access the Website through your internet connection are aware of these Terms of Use and comply with them.
To access the Website or some of the resources it offers, you may be asked to provide certain registration details or other information. It is a condition of your use of the Website that all the information you provide on the Website is correct, current, and complete. You agree that all information you provide to register with this Website or otherwise, including, but not limited to, through the use of any interactive features on the Website, is governed by our [Privacy Policy](https://www.securedgenetworks.com/privacy-policy), and you consent to all actions we take with respect to your information consistent with our Privacy Policy.
If you choose, or are provided with, a user name, password, or any other piece of information as part of our security procedures, you must treat such information as confidential, and you must not disclose it to any other person or entity. You also acknowledge that your account is personal to you and agree not to provide any other person with access to this Website or portions of it using your user name, password, or other security information. You agree to notify us immediately of any unauthorized access to or use of your user name or password or any other breach of security. You also agree to ensure that you exit from your account at the end of each session. You should use particular caution when accessing your account from a public or shared computer so that others are not able to view or record your password or other personal information.
We have the right to disable any user name, password, or other identifier, whether chosen by you or provided by us, at any time in our sole discretion for any or no reason, including if, in our opinion, you have violated any provision of these Terms of Use.
### Intellectual Property Rights
The Website and its entire contents, features, and functionality (including but not limited to all information, software, text, displays, images, video, and audio, and the design, selection, and arrangement thereof) are owned by the Company, its licensors, or other providers of such material and are protected by United States and international copyright, trademark, patent, trade secret, and other intellectual property or proprietary rights laws.
These Terms of Use permit you to use the Website for your personal, non-commercial use only. You must not reproduce, distribute, modify, create derivative works of, publicly display, publicly perform, republish, download, store, or transmit any of the material on our Website, except as follows:
* Your computer may temporarily store copies of such materials in RAM incidental to your accessing and viewing those materials.
* You may store files that are automatically cached by your Web browser for display enhancement purposes.
* You may print or download one copy of a reasonable number of pages of the Website for your own personal, non-commercial use and not for further reproduction, publication, or distribution.
* If we provide desktop, mobile, or other applications for download, you may download a single copy to your computer or mobile device solely for your own personal, non-commercial use, provided you agree to be bound by our end user license agreement for such applications.
You must not:
* Modify copies of any materials from this site.
* Use any illustrations, photographs, video or audio sequences, or any graphics separately from the accompanying text.
* Delete or alter any copyright, trademark, or other proprietary rights notices from copies of materials from this site.
You must not access or use for any commercial purposes any part of the Website or any services or materials available through the Website.
If you wish to make any use of material on the Website other than that set out in this section, please address your request to: .
If you print, copy, modify, download, or otherwise use or provide any other person with access to any part of the Website in breach of the Terms of Use, your right to use the Website will stop immediately and you must, at our option, return or destroy any copies of the materials you have made. No right, title, or interest in or to the Website or any content on the Website is transferred to you, and all rights not expressly granted are reserved by the Company. Any use of the Website not expressly permitted by these Terms of Use is a breach of these Terms of Use and may violate copyright, trademark, and other laws.
### Trademarks
The Company name, the Company logo, and all related names, logos, product and service names, designs, and slogans are trademarks of the Company or its affiliates or licensors. You must not use such marks without the prior written permission of the Company. All other names, logos, product and service names, designs, and slogans on this Website are the trademarks of their respective owners.
### Prohibited Uses
You may use the Website only for lawful purposes and in accordance with these Terms of Use. You agree not to use the Website:
* In any way that violates any applicable federal, state, local, or international law or regulation (including, without limitation, any laws regarding the export of data or software to and from the US or other countries).
* For the purpose of exploiting, harming, or attempting to exploit or harm minors in any way by exposing them to inappropriate content, asking for personally identifiable information, or otherwise.
* To transmit, or procure the sending of, any advertising or promotional material including any "junk mail," "chain letter," "spam," or any other similar solicitation.
* To impersonate or attempt to impersonate the Company, a Company employee, another user, or any other person or entity (including, without limitation, by using email addresses or screen names associated with any of the foregoing).
* To engage in any other conduct that restricts or inhibits anyone's use or enjoyment of the Website, or which, as determined by us, may harm the Company or users of the Website, or expose them to liability.
Additionally, you agree not to:
* Use the Website in any manner that could disable, overburden, damage, or impair the site or interfere with any other party's use of the Website, including their ability to engage in real time activities through the Website.
* Use any robot, spider, or other automatic device, process, or means to access the Website for any purpose, including monitoring or copying any of the material on the Website.
* Use any manual process to monitor or copy any of the material on the Website, or for any other purpose not expressly authorized in these Terms of Use, without our prior written consent.
* Use any device, software, or routine that interferes with the proper working of the Website.
* Introduce any viruses, Trojan horses, worms, logic bombs, or other material that is malicious or technologically harmful.
* Attempt to gain unauthorized access to, interfere with, damage, or disrupt any parts of the Website, the server on which the Website is stored, or any server, computer, or database connected to the Website.
* Attack the Website via a denial-of-service attack or a distributed denial-of-service attack.
* Otherwise attempt to interfere with the proper working of the Website.
### User Contributions
The Website may contain message boards, chat rooms, personal web pages or profiles, forums, bulletin boards, and other interactive features (collectively, "Interactive Services") that allow users to post, submit, publish, display, or transmit to other users or other persons (hereinafter, "post") content or materials (collectively, "User Contributions") on or through the Website.
All User Contributions must comply with the Content Standards set out in these Terms of Use.
Any User Contribution you post to the site will be considered non-confidential and non-proprietary. By providing any User Contribution on the Website, you grant us and our affiliates and service providers, and each of their and our respective licensees, successors, and assigns the right to use, reproduce, modify, perform, display, distribute, and otherwise disclose to third parties any such material for any purpose/according to your account settings.
You represent and warrant that:
* You own or control all rights in and to the User Contributions and have the right to grant the license granted above to us and our affiliates and service providers, and each of their and our respective licensees, successors, and assigns.
* All of your User Contributions do and will comply with these Terms of Use.
You understand and acknowledge that you are responsible for any User Contributions you submit or contribute, and you, not the Company, have full responsibility for such content, including its legality, reliability, accuracy, and appropriateness.
We are not responsible or liable to any third party for the content or accuracy of any User Contributions posted by you or any other user of the Website.
### Monitoring and Enforcement; Termination
We have the right to:
* Remove or refuse to post any User Contributions for any or no reason in our sole discretion.
* Take any action with respect to any User Contribution that we deem necessary or appropriate in our sole discretion, including if we believe that such User Contribution violates the Terms of Use, including the Content Standards, infringes any intellectual property right or other right of any person or entity, threatens the personal safety of users of the Website or the public, or could create liability for the Company.
* Disclose your identity or other information about you to any third party who claims that material posted by you violates their rights, including their intellectual property rights or their right to privacy.
* Take appropriate legal action, including without limitation, referral to law enforcement, for any illegal or unauthorized use of the Website.
* Terminate or suspend your access to all or part of the Website for any or no reason, including without limitation, any violation of these Terms of Use.
Without limiting the foregoing, we have the right to cooperate fully with any law enforcement authorities or court order requesting or directing us to disclose the identity or other information of anyone posting any materials on or through the Website. YOU WAIVE AND HOLD HARMLESS THE COMPANY AND ITS AFFILIATES, LICENSEES, AND SERVICE PROVIDERS FROM ANY CLAIMS RESULTING FROM ANY ACTION TAKEN BY ANY OF THE FOREGOING PARTIES DURING, OR TAKEN AS A CONSEQUENCE OF, INVESTIGATIONS BY EITHER SUCH PARTIES OR LAW ENFORCEMENT AUTHORITIES.
However, we cannot review all material before it is posted on the Website, and cannot ensure prompt removal of objectionable material after it has been posted. Accordingly, we assume no liability for any action or inaction regarding transmissions, communications, or content provided by any user or third party. We have no liability or responsibility to anyone for performance or nonperformance of the activities described in this section.
### Content Standards
These content standards apply to any and all User Contributions and use of Interactive Services. User Contributions must in their entirety comply with all applicable federal, state, local, and international laws and regulations. Without limiting the foregoing, User Contributions must not:
* Contain any material that is defamatory, obscene, indecent, abusive, offensive, harassing, violent, hateful, inflammatory, or otherwise objectionable.
* Promote sexually explicit or pornographic material, violence, or discrimination based on race, sex, religion, nationality, disability, sexual orientation, or age.
* Infringe any patent, trademark, trade secret, copyright, or other intellectual property or other rights of any other person.
* Violate the legal rights (including the rights of publicity and privacy) of others or contain any material that could give rise to any civil or criminal liability under applicable laws or regulations or that otherwise may be in conflict with these Terms of Use and our [Privacy Policy](/legal/privacy).
* Be likely to deceive any person.
* Promote any illegal activity, or advocate, promote, or assist any unlawful act.
* Cause annoyance, inconvenience, or needless anxiety or be likely to upset, embarrass, alarm, or annoy any other person.
* Impersonate any person, or misrepresent your identity or affiliation with any person or organization.
* Involve commercial activities or sales, such as contests, sweepstakes, and other sales promotions, barter, or advertising.
* Give the impression that they emanate from or are endorsed by us or any other person or entity, if this is not the case.
### Copyright Infringement
If you believe that any User Contributions violate your copyright, please send us a notice of copyright infringement. It is the policy of the Company to terminate the user accounts of repeat infringers.
### Reliance on Information Posted
The information presented on or through the Website is made available solely for general information purposes. We do not warrant the accuracy, completeness, or usefulness of this information. Any reliance you place on such information is strictly at your own risk. We disclaim all liability and responsibility arising from any reliance placed on such materials by you or any other visitor to the Website, or by anyone who may be informed of any of its contents.
This Website may include content provided by third parties, including materials provided by other users, bloggers, and third-party licensors, syndicators, aggregators, and/or reporting services. All statements and/or opinions expressed in these materials, and all articles and responses to questions and other content, other than the content provided by the Company, are solely the opinions and the responsibility of the person or entity providing those materials. These materials do not necessarily reflect the opinion of the Company. We are not responsible, or liable to you or any third party, for the content or accuracy of any materials provided by any third parties.
### Changes to the Website
We may update the content on this Website from time to time, but its content is not necessarily complete or up-to-date. Any of the material on the Website may be out of date at any given time, and we are under no obligation to update such material.
### Information About You and Your Visits to the Website
All information we collect on this Website is subject to our [Privacy Policy](/legal/privacy). By using the Website, you consent to all actions taken by us with respect to your information in compliance with the Privacy Policy.
### Online Purchases and Other Terms and Conditions
All purchases through our site or other transactions for the sale of goods or services formed through the Website, or resulting from visits made by you, are governed by the applicable terms and conditions you will agree to during such transaction, which are hereby incorporated into these Terms of Use.
Additional terms and conditions may also apply to specific portions, services, or features of the Website. All such additional terms and conditions are hereby incorporated by this reference into these Terms of Use.
### Linking to the Website and Social Media Features
You may link to our homepage, provided you do so in a way that is fair and legal and does not damage our reputation or take advantage of it, but you must not establish a link in such a way as to suggest any form of association, approval, or endorsement on our part without our express written consent.
This Website may provide certain social media features that enable you to:
* Link from your own or certain third-party websites to certain content on this Website.
* Send emails or other communications with certain content, or links to certain content, on this Website.
* Cause limited portions of content on this Website to be displayed or appear to be displayed on your own or certain third-party websites.
You may use these features solely as they are provided by us, and solely with respect to the content they are displayed with. Subject to the foregoing, you must not:
* Establish a link from any website that is not owned by you.
* Cause the Website or portions of it to be displayed on, or appear to be displayed by, any other site, for example, framing, deep linking, or in-line linking.
* Link to any part of the Website other than the homepage.
* Otherwise take any action with respect to the materials on this Website that is inconsistent with any other provision of these Terms of Use.
You agree to cooperate with us in causing any unauthorized framing or linking immediately to stop. We reserve the right to withdraw linking permission without notice.
We may disable all or any social media features and any links at any time without notice in our discretion.
### Links from the Website
If the Website contains links to other sites and resources provided by third parties, these links are provided for your convenience only. This includes links contained in advertisements, including banner advertisements and sponsored links. We have no control over the contents of those sites or resources, and accept no responsibility for them or for any loss or damage that may arise from your use of them. If you decide to access any of the third-party websites linked to this Website, you do so entirely at your own risk and subject to the terms and conditions of use for such websites.
### Disclaimer of Warranties
You understand that we cannot and do not guarantee or warrant that files available for downloading from the internet or the Website will be free of viruses or other destructive code. You are responsible for implementing sufficient procedures and checkpoints to satisfy your particular requirements for anti-virus protection and accuracy of data input and output, and for maintaining a means external to our site for any reconstruction of any lost data. TO THE FULLEST EXTENT PROVIDED BY LAW, WE WILL NOT BE LIABLE FOR ANY LOSS OR DAMAGE CAUSED BY A DISTRIBUTED DENIAL-OF-SERVICE ATTACK, VIRUSES, OR OTHER TECHNOLOGICALLY HARMFUL MATERIAL THAT MAY INFECT YOUR COMPUTER EQUIPMENT, COMPUTER PROGRAMS, DATA, OR OTHER PROPRIETARY MATERIAL DUE TO YOUR USE OF THE WEBSITE OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE OR TO YOUR DOWNLOADING OF ANY MATERIAL POSTED ON IT, OR ON ANY WEBSITE LINKED TO IT.
YOUR USE OF THE WEBSITE, ITS CONTENT, AND ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE IS AT YOUR OWN RISK. THE WEBSITE, ITS CONTENT, AND ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE ARE PROVIDED ON AN "AS IS" AND "AS AVAILABLE" BASIS, WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. NEITHER THE COMPANY NOR ANY PERSON ASSOCIATED WITH THE COMPANY MAKES ANY WARRANTY OR REPRESENTATION WITH RESPECT TO THE COMPLETENESS, SECURITY, RELIABILITY, QUALITY, ACCURACY, OR AVAILABILITY OF THE WEBSITE. WITHOUT LIMITING THE FOREGOING, NEITHER THE COMPANY NOR ANYONE ASSOCIATED WITH THE COMPANY REPRESENTS OR WARRANTS THAT THE WEBSITE, ITS CONTENT, OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE WILL BE ACCURATE, RELIABLE, ERROR-FREE, OR UNINTERRUPTED, THAT DEFECTS WILL BE CORRECTED, THAT OUR SITE OR THE SERVER THAT MAKES IT AVAILABLE ARE FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS, OR THAT THE WEBSITE OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE WILL OTHERWISE MEET YOUR NEEDS OR EXPECTATIONS.
TO THE FULLEST EXTENT PROVIDED BY LAW, THE COMPANY HEREBY DISCLAIMS ALL WARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED, STATUTORY, OR OTHERWISE, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, AND FITNESS FOR PARTICULAR PURPOSE.
THE FOREGOING DOES NOT AFFECT ANY WARRANTIES THAT CANNOT BE EXCLUDED OR LIMITED UNDER APPLICABLE LAW.
### Limitation on Liability
TO THE FULLEST EXTENT PROVIDED BY LAW, IN NO EVENT WILL THE COMPANY, ITS AFFILIATES, OR THEIR LICENSORS, SERVICE PROVIDERS, EMPLOYEES, AGENTS, OFFICERS, OR DIRECTORS BE LIABLE FOR DAMAGES OF ANY KIND, UNDER ANY LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH YOUR USE, OR INABILITY TO USE, THE WEBSITE, ANY WEBSITES LINKED TO IT, ANY CONTENT ON THE WEBSITE OR SUCH OTHER WEBSITES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES, INCLUDING BUT NOT LIMITED TO, PERSONAL INJURY, PAIN AND SUFFERING, EMOTIONAL DISTRESS, LOSS OF REVENUE, LOSS OF PROFITS, LOSS OF BUSINESS OR ANTICIPATED SAVINGS, LOSS OF USE, LOSS OF GOODWILL, LOSS OF DATA, AND WHETHER CAUSED BY TORT (INCLUDING NEGLIGENCE), BREACH OF CONTRACT, OR OTHERWISE, EVEN IF FORESEEABLE.
THE FOREGOING DOES NOT AFFECT ANY LIABILITY THAT CANNOT BE EXCLUDED OR LIMITED UNDER APPLICABLE LAW.
### Indemnification
You agree to defend, indemnify, and hold harmless the Company, its affiliates, licensors, and service providers, and its and their respective officers, directors, employees, contractors, agents, licensors, suppliers, successors, and assigns from and against any claims, liabilities, damages, judgments, awards, losses, costs, expenses, or fees (including reasonable attorneys' fees) arising out of or relating to your violation of these Terms of Use or your use of the Website, or any use of the Website's content, services, and products other than as expressly authorized in these Terms of Use, or your use of any information obtained from the Website.
### Governing Law and Jurisdiction
All matters relating to the Website and these Terms of Use, and any dispute or claim arising therefrom or related thereto (in each case, including non-contractual disputes or claims), shall be governed by and construed in accordance with the internal laws of the State of Delaware without giving effect to any choice or conflict of law provision or rule (whether of the State of Delaware or any other jurisdiction).
Any legal suit, action, or proceeding arising out of, or related to, these Terms of Use or the Website shall be instituted exclusively in the federal or state courts located in the County of King in the State of Washington. You waive any and all objections to the exercise of jurisdiction over you by such courts and to venue in such courts.
### Arbitration
At Company's sole discretion, it may require you to submit any disputes arising from these Terms of Use or use of the Website, including disputes arising from or concerning their interpretation, violation, invalidity, non-performance, or termination, to final and binding arbitration under the Rules of Arbitration of the American Arbitration Association applying Delaware law.
Limitation on Time to File Claims
ANY CAUSE OF ACTION OR CLAIM YOU MAY HAVE ARISING OUT OF OR RELATING TO THESE TERMS OF USE OR THE WEBSITE MUST BE COMMENCED WITHIN ONE (1) YEAR AFTER THE CAUSE OF ACTION ACCRUES; OTHERWISE, SUCH CAUSE OF ACTION OR CLAIM IS PERMANENTLY BARRED.
### Waiver and Severability
No waiver by the Company of any term or condition set out in these Terms of Use shall be deemed a further or continuing waiver of such term or condition or a waiver of any other term or condition, and any failure of the Company to assert a right or provision under these Terms of Use shall not constitute a waiver of such right or provision.
If any provision of these Terms of Use is held by a court or other tribunal of competent jurisdiction to be invalid, illegal, or unenforceable for any reason, such provision shall be eliminated or limited to the minimum extent such that the remaining provisions of the Terms of Use will continue in full force and effect.
### Entire Agreement
The Terms of Use, our Privacy Policy, and any additional terms and conditions you may agree to in the use of our Service, constitute the sole and entire agreement between you and Spice AI regarding the Website and supersede all prior and contemporaneous understandings, agreements, representations, and warranties, both written and oral, regarding the Website.
### Your Comments and Concerns
All feedback, comments, requests for technical support, and other communications relating to the Website should be directed to: .
# Terms of Service
Last Updated: September 13, 2023
These are the terms of service (the “**Terms of Service**”) under which Spice AI agrees to grant Customer access to and use of Spice AI’s online software service offerings and, if applicable, certain professional services. By indicating Customer’s acceptance of these Terms of Service, by executing a Sales Order that references these Terms of Service, or by using Spice AI’s software or services, Customer agrees to be bound by these Terms of Service, including the [Website Terms of Use](/legal/terms) and the [End User License Agreement (“**EULA**”)](/legal/eula), which are hereby incorporated herein.
### Background
Spice AI provides artificial intelligence and data solutions, by providing its customers with access to Spice AI’s online software applications and tools and, if applicable, by providing related professional services. Customer wishes to acquire a subscription-based license to the computer software applications and tools and, if applicable, to purchase professional services, all as specified in one or more “Sales Orders” under and subject to these Terms of Service. Therefore, for good and valuable consideration, the receipt and sufficiency of which they each acknowledge, Spice AI and Customer agree to be bound by these Terms of Service.
### Section 1: Definitions
For the purposes of these Terms of Service, the following initially capitalized words are ascribed the following meanings:
“**Affiliate**” means any person, partnership, joint venture, corporation or other form of venture or enterprise, domestic or foreign, including subsidiaries, which directly or indirectly Control, are Controlled by, or are under common Control with a party.
“**Community Edition**” means a limited version of the Subscription Services that a Customer may access and use without the payment of Subscription Fees.
“**Confidential Information**” has the meaning ascribed to it in Section 6.1.
“**Control**” means the possession, directly or indirectly, of the power to direct or cause the direction of the management and operating policies of the entity in respect of which the determination is being made, through the ownership of more than fifty percent (50%) of its voting or equity securities.
“**Customer**” means the entity identified as such in the applicable Sales Order.
“**Customer Data**” means any data or other information owned or controlled by Customer which is provided by Customer to Spice AI in connection with the Services.
“**Customer Materials**” means any software, documentation, Customer Data, hardware, tools, trademarks, service marks or brands, or any other materials, information or intellectual property owned, leased, licensed or used by Customer, and that Customer delivers to Spice AI or to which Customer provides Spice AI with access, or that Customer requires Spice AI to use for purposes of the Services.
“**Documentation**” means the software user and administrator manuals provided to Customer by Spice AI regarding use of the Spice AI Technology, including additional, updated or revised documentation, if any.
“**Intellectual Property Rights**” means all trade secrets, patents and patent applications, trademarks (whether registered or unregistered and including any goodwill acquired in such trade marks), service marks, trade names, copyrights, moral rights, database rights, design rights, rights in know-how, rights in Confidential Information, rights in inventions (whether patentable or not) and all other intellectual property and proprietary rights (whether registered or unregistered, any application for the foregoing, and all rights to enforce the foregoing), and all other equivalent or similar rights which may subsist anywhere in the world.
"**Output**” means information generated and returned by the Services based on data and information that you enter into the Services.
“**Personal Information**” means personally identifiable information of an individual person that is required to be protected by applicable law.
“**Professional Services**” means the professional services (typically consulting and configuration) to be performed by Spice AI that are specified in the applicable Sales Order. Professional Services are not required for use of the Subscription Services. Terms governing Professional Services are specified in the ‘Additional Terms of Service – Professional Services Terms’ referenced in Section 12.
“**Sales Order**” means any mutually agreed, written sales order, executed on behalf of Spice AI and Customer (including its exhibits and addenda), describing the Subscription Services, Support Program, Professional Services (if applicable), fees, and any special terms for using the Subscription Services that Customer has ordered. If Customer subscribes via Spice AI’s online checkout, then the checkout form submitted by Customer and accepted by Spice AI constitutes the applicable Sales Order for purposes of these Terms of Service. Each Sales Order becomes effective on its Sales Order Effective Date, and is made part of these Terms of Service as described in Section 1.2.
“**Sales Order Effective Date**” means, with respect to each Sales Order, (i) the effective date specified in the Sale Order or, if none, then the date on which the last of Spice AI and Customer executes the Sales Order, (ii) in the case of an online checkout form, the date on which Spice AI enters the completed online checkout form into its system, or (iii) if Customer purchases Services through a Channel Partner, upon Spice AI’s acceptance of the Sales Order following its submission by the Channel Partner.
“**Services**” means the Subscription Services and the Professional Services.
“**Site**” means a location designated by Spice AI which may include a subpage within Spice AI’s website () or other location or method of providing access to Customer to the Spice AI Content or the Spice AI Technology.
“**Spice AI Methodology**” means Spice AI’s processes, methods, techniques and know-how, relating to the Spice AI Technology or to time-series data management solutions.
“**Spice AI Software**” means the Spice AI computer software applications, tools and other programs specified in a Sales Order.
“**Spice AI Technology**” means the Site, the Spice AI Software and its Documentation, together with other computer software programs, networks and equipment that Spice AI uses to make the foregoing available to its customers as an on-line subscription offering.
“**Spice AI Technology Subscription Services**” means the Spice AI Technology service offerings to which Customer subscribes, as specified in the applicable Sales Order.
“**Subscription Services**” means the Spice AI Technology Subscription Services.
“**Subscription Term**” has the meaning ascribed to it in Section 8.
“**Term**” means the Initial Term and any Renewal Terms, as more fully described in Section 8.
“**Third-Party Service Provider**” means a third party that provides services to Customer, and that has agreed in writing with Customer to requirements to protect Intellectual Property Rights and Confidential Information of Customer, its suppliers and licensors that are substantially no less protective of Intellectual Property Rights and Confidential Information than those contained in these Terms of Service, and that is not a direct competitor of Spice AI.
“**User**” means any individual who is an employee or independent contractor of Customer or its Affiliates or, to the extent providing services to Customer, is an employee of a Third-Party Service Provider, and who is authorized by Customer to use the Spice AI Technology pursuant to the applicable Sales Order.
The following words will be interpreted as designated: (i) “or” connotes any combination of all or any of the items listed; (ii) where “including” is used to refer to an example or begins a list of items, such example or items will not be exclusive; (iii) “specified” requires that an express statement is contained in the relevant document; and (iv) “will” is, unless the context requires otherwise, an expression of command, not merely an expression of future intent or expectation.
**1.2 Construction.** These Terms of Service apply to the provision of all Services. The parties will enter into one or more Sales Orders that contain additional terms and conditions applicable to the provision of certain Services. Upon execution by the parties, each Sales Order will be incorporated into these Terms of Service.
### Section 2. Provision and Use of Services
**2.1 Provision of Spice AI Technology Subscription Services.** Customer may access and use the Spice AI Technology Subscription Services in accordance with these Terms of Service. Access to the Spice AI Technology Subscription Services is available through the Internet over an https connection, via a password-protected computer interface or access code and multi-factor authentication.
**2.2 Community Edition.** Spice AI makes available the Community Edition of the Spice AI Software with limited features and support as described [here](broken://pages/TswKQuxvMpeSWfvqFhxa). Customers may use the Community Edition without payment of Subscription Fees, however, such use of Community Edition shall be subject to all other terms and conditions of this Terms of Service, including the Website Terms of Use and EULA.
**2.3 Authority and Capacity.** If you are entering into these Terms of Service on behalf of an entity, such as the company you work for, then you represent to Spice AI that you have the legal authority to bind the entity Customer to these Terms of Service. If you do not have that authority or if Customer does not agree with the terms of these Terms of Service, then you may not indicate acceptance of these Terms of Service, and neither you nor Customer may use or access any of Spice AI’s service offerings or other services.
**2.4 Customer’s Account.** Customer will designate one of its employees to be the point of contact with Spice AI for the management and support of the Spice AI Technology Subscription Services (“Administrator”), and who will be responsible for establishing and managing Customer’s use of the Spice AI Technology Subscription Services (“Account”), including the creation of usernames and passwords to access Customer’s Account. Customer is solely responsible for maintaining the status of its User base. Customer will safeguard all User authentication credentials in its possession or under its control. Customer is responsible for all activities that occur under the Account. Customer will notify Spice AI immediately if Customer believes an unauthorized third party may be using Customer’s Account or if Customer’s Account information is lost or stolen.
**2.5 Spice AI’s Responsibilities.** During the Subscription Term, Spice AI will use commercially reasonable efforts to make the Spice AI Technology Subscription Services available 24 hours a day, 7 days a week, except for planned downtime (of which Spice AI will give at least 48 hours’ notice and which Spice AI will schedule to the extent practicable during the weekend hours from 6:00 p.m. Friday to 3:00 a.m. Monday Pacific Time). Spice AI may in addition perform unscheduled emergency maintenance for the security or performance of the Spice AI Technology Subscription Services; Spice AI will use commercially reasonable efforts to provide at least 30 minutes prior notification of emergency maintenance that Spice AI believes will cause downtime.
**2.6 Customer’s Responsibilities.** Customer is solely responsible for the accuracy, quality and integrity of the Customer Data that Customer inputs or provides for input into the Spice AI Technology. Customer must comply with the License Conditions in the EULA and Prohibited Uses in the Website Terms of Service. Customer will ensure that its Users, its Affiliates, its Third Party Service Providers, and its and their employees, agents and representatives comply with Customer’s obligations under these Terms of Service, and Customer is responsible for their acts and omissions relating to these Terms of Service as though they were Customer’s own.
### Section 3. Proprietary Rights
**3.1 License by Spice AI.** Subject to the terms and conditions of these Terms of Service, Spice AI hereby grants to Customer a non-exclusive, non-transferable, limited license, without right to sub-license, for the Subscription Term, to access and use the Spice AI Technology Subscription Services in accordance with these Terms of Service, solely for Customer’s internal business operations, and subject to the license metrics and other scope limitations specified in the applicable Sales Order. Spice AI reserves all other rights not expressly granted in these Terms of Service.
**3.2 License by Customer.** Customer hereby grants to Spice AI a non-exclusive, limited, royalty-free license, without right to sub-license, to use the Customer Materials solely as necessary to perform the Services. Customer reserves all other rights not expressly granted in these Terms of Service.
**3.3 Ownership of Intellectual Property Rights.**
**3.3.1 Ownership and Use of Customer Materials.** Customer retains all of its rights, title and interest and Intellectual Property Rights in and to the Customer Materials and Customer Confidential Information. No ownership interest in the Customer Materials or Customer Confidential Information is transferred or conveyed to Spice AI by virtue of these Terms of Service. Spice AI will use Customer Materials and Customer Confidential Information only for purposes of providing the Services, unless otherwise be authorized in writing by Customer.
**3.3.2 Spice AI’s Intellectual Property and Ownership Rights.** As between Customer and Spice AI, Spice AI and Spice AI’s licensors retain and own all right, title and interest and all Intellectual Property Rights in and to the Spice AI Technology, the Spice AI Methodology, Spice AI’s Confidential Information, the Subscription Services, and all enhancements or improvements to, or derivative works of the foregoing created or developed by or on behalf of Spice AI (collectively, “Spice AI Intellectual Property”). Except as otherwise agreed in writing between the parties, nothing in these Terms of Service transfers or conveys to Customer any ownership interest in or to the Spice AI Intellectual Property.
**3.4 Restrictions.** Customer will not: (i) except to the extent, if any, permitted by applicable law or required by Spice AI’s licensors, reverse assemble, reverse engineer, decompile or otherwise attempt to derive source code from any of the Spice AI Technology; (ii) reproduce, modify, or prepare derivative works of any of the Spice AI Technology or Documentation; (iii) distribute or display any of the Spice AI Technology or Documentation other than to Customer’s Users; or (iv) share, rent or lease the Spice AI Technology Subscription Services, or use the Spice AI Technology Subscription Services to operate any timesharing, service bureau or similar business. Some components of the Spice AI Technology may also be governed by applicable open source software licenses located in the software component’s source code. Customer’s license rights with respect to these individual components are defined by the applicable open source software license, and nothing in this Agreement will restrict, limit, or otherwise affect any rights or obligations Customer may have, or conditions to which Customer may be subject, under such open source software licenses.
**3.5 Suggestions.** If Customer provides Spice AI with any suggested improvements or enhancements to the Services or Spice AI Technology (“Suggestions”), then Customer also grants Spice AI a non-exclusive, perpetual, irrevocable, paid-up, royalty-free, worldwide, transferable license, with right to sublicense, to make, have made, sell, offer for sale, use, import, reproduce, distribute, display, perform, and make derivative works of the Suggestions.
### Section 4. Compensation
**4.1 Subscription Plans.** Customer’s subscription plan for the Subscription Services is specified in the applicable Sales Order. Customer may not reduce Customer’s commitment under the Subscription Services subscription plan specified in the Sales Order during the Services Subscription Term. Customer is not entitled to any refund of fees paid or relief from fees due if the volume of Subscription Services Customer actually uses is less than the volume Customer ordered, and Customer may not carry over any of the unused volume to Customer’s next Subscription Term. If Customer wishes to reduce the volume of a Subscription Services subscription plan for a Renewal Term, then Customer must notify Spice AI at least 60 days before the start of the Renewal Term for the applicable Subscription Services; the reduction will be effective at the start of the Renewal Term.
**4.2 Subscription Fees.** Except as expressly stated otherwise in any Sales Order, Spice AI will invoice Customer for Subscription Services in advance, at the rates or for the fees specified in the applicable Sales Order. If Customer subscribes via Spice AI’s online checkout, customer will pay the Subscription Fees by credit card.
**4.3 Payment Terms.** The billing method and payment terms for amounts due under these Terms of Service are as specified in the applicable Sales Order. Except as otherwise specified in these Terms of Service, Spice AI bills and collects in advance for use of the Services. Invoices for Service subscription fees and other amounts are due and payable in US dollars within 30 days of the date of invoice; however, if Customer subscribes via Spice AI’s online checkout, then Customer will pay its fees for Subscription Services by credit card when Customer submits its order. Spice AI may apply finance charges equal to 1.5% per month or the highest rate permitted by applicable usury law, whichever is less, to any amount not paid when due.
**4.4 Taxes.** Customer will be responsible for and will pay to Spice AI any applicable sales, use, or any value added or similar taxes (collectively, “Taxes”) payable with respect to provision of the Services to Customer, or arising out of or in connection with these Terms of Service, other than taxes based upon Spice AI’s personal property ownership or net income. Unless expressly specified otherwise in any Sales Order, all fees, rates and estimates exclude Taxes.
**4.5 Records and Inspection; Audit.**
**4.5.1 Spice AI Records.** Spice AI will maintain reasonably complete and accurate accounting records to substantiate Spice AI’s variable charges and expenses hereunder. Spice AI will retain such records for a period of at least two years from the date of the invoice applicable to such charges and/or expenses. Upon not less than 45 days prior written notice, an independent certified public accountant appointed by Customer may inspect, copy and audit such records at Spice AI’s corporate offices.
**4.5.2 Audit Conditions.** Whenever a party or its designee conducts any audit or inspection under these Terms of Service, such party will comply, and will ensure that its appointed accountants comply with the following (collectively, the “Audit Protection Conditions”): audits and other inspections will be conducted at any time during the audited party’s regular business hours; the auditing party and its auditors will comply with the audited party’s reasonable security requirements in conducting the inspection, and will use commercially reasonable efforts to minimize disruption to the audited party’s business; the auditing party and its auditors will treat all information that they obtain from the inspection as Confidential Information of the audited party, except to the extent necessary for the auditing party to enforce its rights under the Terms of Service. In no circumstances will the auditing party or its auditors have access to any information protected by attorney client privilege or the “work product” doctrine nor, if Spice AI is the audited party, to any information pertaining to Spice AI’s other customers, nor to any information relating to the audited party’s margins. Unless an audit discloses a material non-compliance by the audited party with its obligations under these Terms of Service, the auditing party may exercise such audit rights no more than once during any twelve (12) month period. Unless the audit reveals a discrepancy in favor of the audited party of more than 10% of fees and expenses that should have been paid for the audited period, the auditing party will bear all costs and expenses relating to the audit, including compensation to Spice AI for personnel and materials provided to facilitate the audit, and Spice AI’s then current rates.
### Section 5. Warranties and Disclaimers
**5.1 Warranties.** Spice AI warrants to Customer that:
**5.1.1 Performance Warranty.** During the Subscription Term, the Spice AI Technology made available to Customer for Customer’s use will conform in all material respects to its applicable specifications set forth in the Documentation.
**5.1.2 Viruses and Lock-Outs.** Spice AI will use commercially reasonable efforts, using then-current versions of commercially available anti-virus software and tools, to ensure that the Spice AI Technology, in the form provided by Spice AI to Customer under these Terms of Service, contain no computer virus, Trojan horse, worm or other similar malicious code.
**5.1.3 Infringement.** Spice AI’s provision to Customer of the Subscription Services does not infringe any third party patent existing under the laws of the United States or Canada, or infringe any third party copyright, trademark or service mark, or result from misappropriation by Spice AI of any third party’s trade secrets (collectively, an “Spice AI Infringement”).
**5.2 Performance Remedy.** If any Spice AI Technology fails to conform to the warranty set forth in Section 5.1.1 and Customer provides written notice of the non-conformance to Spice AI within the applicable Subscription Term then, as Customer’s exclusive remedy and Spice AI’s sole obligation: Spice AI will either repair or, at its option, replace the non-conforming Spice AI Technology with conforming Spice AI Technology or, if Spice AI is unable to correct the non-conformance within 30 days of receipt of such written notice from Customer, Customer may terminate the applicable Subscription Services insofar as they apply to the non-conforming Spice AI Technology, and Spice AI will refund to Customer a pro-rata amount of any Subscription Services fees prepaid to Spice AI and applicable to the unutilized portion of the Subscription Term for the terminated Subscription Services.
**5.3 Bugs and Abatement; Scope.** Without limiting the foregoing, Spice AI does not warrant that the Spice AI Technology or Services are completely free from all bugs, errors, or omissions, or will ensure complete security. The warranty in Section 5.1.1 does not apply to any trial or evaluation subscription to the Spice AI Technology. The warranties in these Terms of Service will automatically abate to the extent that the Spice AI Technology have been modified by persons other than Spice AI’s authorized employees or representatives, or other than at Spice AI’s express direction. The warranties in these Terms of Service are for the sole benefit of Customer, and may not be extended to any other person or entity.
**5.4 Disclaimer of Accuracy.** Artificial intelligence and machine learning are rapidly evolving fields of study. Spice AI is constantly working to improve our Services to make them more accurate, reliable, safe and beneficial. Given the probabilistic nature of machine learning, use of our Services may in some situations result in incorrect data that does not accurately reflect real people, places, or facts. You should evaluate the accuracy of any data as appropriate for your use case, including by using human review of the data.
**5.5 Disclaimer Of Implied Warranties.** Spice AI makes no representation or warranty in connection with the Spice AI Technology or Services, except as expressly so set forth in this Section 5 or the Additional Terms of Service. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, EXCEPT AS SPECIFICALLY WARRANTED IN THIS SECTION 5 OR THE ADDITIONAL TERMS OF SERVICE, EACH PARTY DISCLAIMS ALL IMPLIED WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, ANY IMPLIED WARRANTY OF NON-INFRINGEMENT OR IMPLIED OBLIGATION TO INDEMNIFY FOR INFRINGEMENT, ANY IMPLIED WARRANTY ARISING FROM COURSE OF PERFORMANCE, COURSE OF DEALING, OR USAGE OF TRADE, AND ANY STATUTORY REMEDY.
### Section 6. Confidential Information; Personal Information; Data Protection
**6.1 Restrictions on use and Disclosure.** Neither Spice AI nor Customer will disclose to any third party any information provided by the other party pursuant to or in connection with these Terms of Service that the disclosing party identifies as being proprietary or confidential or that, by the nature of the circumstances surrounding the disclosure, ought in good faith to be treated as proprietary or confidential (such information, “Confidential Information”), and will make no use of such Confidential Information except under and in accordance with these Terms of Service. Confidential Information includes: information concerning Inventions, concepts, ideas, techniques, specifications, drawings, diagrams, models, samples, flow charts, computer programs and code and their associated documentation and programmer’s notes, network topography and network configuration and access information, security policies and processes, data, finances and plans, business plans, contracts, marketing plans, system implementation plans, business concepts, business procedures and business operations, pricing, market analysis, research, strategies, projections, forecasts and financial information and all materials related thereto; and Personal Information. Confidential Information also includes information disclosed by the disclosing party with permission from a third party, and combinations of or with publicly known information where the nature of the combination is not publicly known. Customer’s Confidential Information includes Customer Data. Spice AI’s Confidential Information includes any information regarding the Spice AI Technology, Spice AI Methodology, Spice AI service offerings, and Documentation.
**6.2 Exclusions.** Except with respect to Personal Information, Confidential Information does not include information that the receiving party can establish: (i) has entered the public domain without the receiving party’s breach of any obligation owed to the disclosing party; (ii) has been rightfully received by the receiving party from a third party without confidentiality restrictions; (iii) is known to the receiving party without any restriction as to use or disclosure prior to first receipt by the receiving party from the disclosing party; or (iv) has been independently developed by the receiving party without use of or reference to the disclosing party’s Confidential Information.
**6.3 Disclosure Required By Law.** If any applicable law, regulation or judicial or administrative order requires the receiving party to disclose any of the disclosing party’s Confidential Information or Personal Information (a “Disclosure Order”) then, unless otherwise required by the Disclosure Order, the receiving party will promptly notify the disclosing party in writing prior to making any such disclosure, in order to facilitate the disclosing party’s efforts to protect its Confidential Information or Personal Information. Following such notification, the receiving party will cooperate with the disclosing party, at the disclosing party’s reasonable expense, in seeking and obtaining protection for the disclosing party’s Confidential Information or Personal Information.
**6.4 Independent Development.** The terms of confidentiality under these Terms of Service will not limit either party’s right to independently develop or acquire products, software or services without use of or reference to the other party’s Confidential Information.
**6.5 Data Processing and Personnel Approvals.** Except to the extent otherwise specified in the applicable Sales Order, Customer is solely responsible for obtaining and will obtain all necessary consents, licenses and approvals for the processing of any Customer Data as part of the Services.
**6.6 Protection of Customer Data.**
**6.6.1 Regulatory Requirements.** To the extent applicable to the Services required to be provided by Spice AI to Customer under these Terms of Service, Spice AI will comply, and will ensure that its employees and subcontractors comply (to the extent such subcontractors have access to Personal Information), with the requirements of applicable state and federal privacy laws and regulations governing Customer Personal Information in Spice AI’s possession or under its control (collectively, “Regulatory Requirements”). Customer may not provide Spice AI with Personal Information or upload Personal Information to the Spice AI Technology without Spice AI’s express prior written consent in each instance.
**6.6.2 Security Program.** To the extent applicable to the Services provided by Spice AI to Customer under these Terms of Service, Spice AI will implement and maintain commercially reasonable security measures designed to meet the following objectives (collectively, the “Spice AI Security Program”): (i) ensure the security and confidentiality of Customer Data in the custody and under the control of Spice AI; (ii) protect against any anticipated threats or hazards to the security or integrity of such Customer Data; (iii) protect against unauthorized access to or use of such Customer Data; (d) encrypt Customer Data as specified in Section 6.6.3 below; and (iv) ensure that Spice AI’s return or disposal of such Customer Data is performed in a manner consistent with Spice AI’s obligations under items (i)-(iv) above. Customer acknowledges and agrees that is commercially reasonable for Spice AI to rely upon the security processes and measures utilized by Spice AI’s cloud infrastructure providers.
**6.6.3 Encryption.** Spice AI will encrypt Customer Data in Spice AI’s possession or under its control when transmitted by Spice AI, using any then current industry standard encryption technologies. Notwithstanding the foregoing, unless specified otherwise in the applicable Sales Order, Spice AI is not required to encrypt Personal Information which is stored on servers located in Spice AI’s or its service providers’ data centers with a SSAE-16 (or similar) certification.
**6.6.4 Certification; Audits.** Spice AI will provide Customer with copies of its or its service providers’ SSAE-16 or similar certifications applicable to Spice AI’s provision of Subscription Services, if any, upon request by Customer. During Spice AI’s provision of Subscription Services under these Terms of Service, if and to the extent required by applicable law, Customer and its regulators may audit Spice AI’s records related to its access to and storage or processing of Customer Data, upon not less than 45 days prior written notice to Spice AI, solely as necessary to verify Spice AI’s compliance with its obligations under this Section 6.6. With respect to any audit performed under this Section 6.6.4, Customer will, and will require that its auditors will, and will request its regulators to, comply with the Audit Protection Conditions.
**6.6.5 Breach Notification.** Spice AI will notify Customer of unauthorized access to, or use or disclosure of Customer Data within Spice AI’s custody and control within one business day of Spice AI’s confirmation of the same; each party will reasonably cooperate with the other with respect to the investigation and resolution of such unauthorized access, use or disclosure. Upon confirmation of any vulnerability or breach of Spice AI’s security affecting Customer Data in Spice AI’s custody and control, Spice AI will modify its processes and security program as necessary to mitigate the effects of the vulnerability or breach upon such Customer Data. Customer will notify Spice AI of any security compromise affecting its Users’ authentication credentials used to access the Spice AI Technology, and any Customer systems or networks that interoperate with or transmit data to the Spice AI Technology, within one business day of confirmation of the same.
**6.6.6 Spice AI’s Processing of Customer Data.** Spice AI will be liable for any unauthorized access to Customer Data by third parties only to the extent resulting from Spice AI’s failure to adhere to the Spice AI Security Program, or from Spice AI’s gross negligence or willful misconduct. The provisions of this Section 6.6.6 apply notwithstanding any provision of these Terms of Service or any other agreement between Spice AI and Customer (or any Affiliate of Customer) to the contrary.
### Section 7. Beta Feature Testing
**7.1 Beta; Preview.** Customer understands and agrees that certain features of the Services and Products may be provided with early access (“Beta Features” or “Preview Features”) which Customer may choose to “opt in” to for testing and evaluation purposes and, therefore, such Beta Features may include or cause some unknown issues and use cases that Spice AI will use best efforts to resolve in stride. In consideration of receiving the opportunity to access these Beta Features, Customer hereby releases and holds harmless Spice AI and its Affiliates, and their respective members, officers, agents and employees, and all persons participating in the manufacturing or development of the Products and Services (“Released Parties”) from any and all liability, claims, demands or causes of action that Customer may hereafter incur for injuries or damages arising out of use of the Beta Features. Customer specifically releases the Released Parties from any responsibility for their negligence in the planning, conducting or supervising the use of the Beta Features.
### Section 8. Term; Termination of Sales Orders
These Terms of Service will commence on the Effective Date and will continue in effect until terminated in accordance with Section 8.1 or 8.2 below.
**8.1 Termination On Breach.** In the event of a material breach of the Terms of Service by either party, the non-breaching party may terminate the Terms of Service or any Sales Order directly affected by the breach by giving the breaching party written notice of the breach and the non-breaching party’s intention to terminate. If the breach has not been cured within the period ending 30 days after such notice, and if the non-breaching party provides written notice of termination to the breaching party (“Termination Notice”), these Terms of Service or any such Sales Order will terminate within the time period specified in the Termination Notice. Notwithstanding the foregoing, Customer’s failure to pay any overdue fees and expenses within 30 days of Spice AI notifying Customer of the overdue payment, which notice may include informing Customer’s Project Manager or accounts payable representative of the overdue payment in writing (including by email), will constitute a material breach of these Terms of Service. If Customer has not cured a material breach within the applicable cure period, then Spice AI may, in its sole discretion, and without prejudice to its other rights following material breach and failure to cure, until such breach has been cured in full, suspend performance of some or all of Spice AI’s obligations to provide Services or Spice AI Technology under these Terms of Service.
**8.2 Termination for Convenience.** Either party may terminate any Subscription Services effective upon expiration of the then current Subscription Term by providing the other party prior written notice of termination of at least the following length: if Customer is the terminating party, at least 30 days; and if Spice AI is the terminating party, at least 90 days.
**8.3 Subscription Term and Renewal.**
**8.3.1 Term.** Each subscription term for Subscription Services will commence on the Sales Order Effective Date and will continue for the period specified in the Sales Order or, if not so specified, one year (an “Initial Term”). Unless otherwise specified in the Sales Order, the Initial Term will automatically renew for additional terms of the same length of the Initial Term (each, a “Renewal Term”) unless either party declines renewal by notice in writing to that effect delivered to the other party at least 30 days prior to expiration of the then current subscription term. The Initial Term and each Renewal Term are individually referred to in these Terms as the “Subscription Term”.
**8.3.2. Subscription to Upgraded or Additional Services.** If Customer upgrades any of Customer’s Subscription Service subscriptions during a Subscription Term, then the Subscription Term for the upgraded Subscription Service will be coterminous with the current Subscription Term and Spice AI will invoice Customer an amount equal to the difference between the original Subscription Service fee and the upgraded Subscription Service fee. If Customer subscribes to an additional Subscription Service, the Subscription Term for that Subscription Service will begin on the Sales Order Effective Date for that Subscription Service and Customer will be separately invoiced for the applicable activation and Subscription Service fee for the additional Subscription Service.
**8.3.3 Fee Increases.** Fees for Services are subject to increases, which will be effective beginning upon the first day of each Renewal Term. Spice AI will notify Customer of any increase prior to its becoming effective; notice may be in a form of an invoice. If Customer objects to the increase, then Customer may terminate its subscription to the affected Services for convenience by notice provided to Spice AI at any time within 30 days of receipt of the increase notice; any such termination will be effective on the later to occur of the expiration of the 30 day period or the then current Subscription Term. Customer acknowledges that expiration of any discount or incentive programs to which Customer was previously entitled do not constitute fee increases.
**8.4 Fulfillment of Obligations on Termination.** Termination of the Terms of Service or of any Services will not relieve Customer from any obligation to pay fees or reimburse expenses for Services performed prior to termination.
**8.5 Refund of Prepaid Subscription Fees if Customer Terminates for Breach.** If Customer terminates these Terms of Service or any Sales Order for breach in accordance with Section 8.1 above, then Spice AI will refund to Customer a pro-rata amount of any Subscription Services fees prepaid to Spice AI and applicable to the unutilized portion of the Subscription Term for terminated Subscription Services. Customer is not entitled to any refund or relief from payment of any Subscription Services fees paid or payable under these Terms of Service if Customer terminates these Terms of Service or an applicable Sales Order for convenience.
**8.6 Post Termination Obligations.** Following any termination of the Terms of Service or any Sales Order, each party will, within 30 days of such termination, (i) immediately cease use of any Confidential Information of the other communicated for the purposes of these Terms of Service or such Sales Order, and (ii) return or destroy (and certify destruction of) all copies of any Confidential Information of the other party disclosed under the Terms of Service or such Sales Order within 30 days of such termination.
**8.7 Access to and Retention of Customer Data.** Customer may download an export file of Customer Data stored on the Spice AI Technology at any time during the Subscription Term, and during the 30 day period following its termination or expiration. Spice AI will destroy or overwrite Customer Data within a reasonable period of time following termination of the applicable Sales Order, subject to conformance with Spice AI’s backup and data retention policies.
**8.8 Suspension.** Spice AI may suspend Customer’s right to access or use any portion of the Spice AI Technology Subscription Services immediately upon notice to Customer if Spice AI determines that Customer’s, its Affiliates’, or its or their Users’ use of the Spice AI Technology Subscription Services (i) poses a security risk to the Spice AI Technology Subscription Services or any third party, (ii) may adversely impact the Spice AI Technology Subscription Services, or the networks or data of any other Spice AI customer or business partner, (iii) does not comply with applicable law, or (iv) may subject Spice AI or any third party to liability.
**8.9 Survival.** The provisions of Sections 1, 3, 4.5, 6, 8.5-8.7, 8.9, 9, 10 and 14 of these Terms of Service will survive any termination of these Terms of Service.
### Section 9. Indemnification
**9.1 Spice AI’s Infringement Indemnification.**
**9.1.1 Defense and Indemnity.** If any third party makes any claim against Customer that, if true, would constitute an Spice AI Infringement then, upon notification of such claim, Spice AI will, at its sole cost and expense, defend Customer against such claim and any related proceeding brought by such third party against Customer, and indemnify Customer from and against all damages finally awarded against Customer or agreed to be paid by Customer in a written settlement approved in writing by Spice AI, and resulting from the Spice AI Infringement. Spice AI’s obligations under this Section 9.1.1 are conditioned upon Customer’s compliance with the “Indemnification Conditions” (defined below).
“Indemnification Conditions” means the following conditions with which a party must comply in order to be entitled to defense or indemnification under the Terms of Service by the other party: (i) the indemnified party notifies the indemnifying party in writing of any claim that might be the subject of indemnification promptly after any executive officer of the indemnified party or member of the indemnified party’s legal department first knows of the claim, provided, however, that no failure to so notify an indemnifying party will relieve the indemnifying party of its obligations under these Terms of Service except to the extent that such failure materially prejudices defense of the claim, and except to the extent of damages incurred by the indemnifying party as a result of the delay; (ii) the indemnifying party is given primary control over the defense and settlement of the claim (subject to the foregoing, the indemnified party may nonetheless participate in the defense at its sole cost and expense); (iii) the indemnified party makes no admission of liability (except as required by applicable law) nor enters into any settlement without the indemnifying party’s prior written agreement; (iv) the indemnified party provides such assistance in defense of the proceeding as the indemnifying party may reasonably request, at the indemnifying party’s reasonable expense; (v) the indemnified party complies with any court order or reasonable settlement made in connection with the proceeding; and (vi) the indemnified party uses all commercially reasonable efforts to mitigate its losses.
**9.1.2 Spice AI’s Mitigation Rights.** If any Subscription Services supplied by Spice AI become (or in Spice AI’s opinion are likely to become) the subject of any infringement or misappropriation claim, Spice AI may, and if Customer’s use of the Subscription Services is enjoined, Spice AI must, at its sole expense, either: (i) procure for Customer the right to continue using the relevant Subscription Services; (ii) replace or modify the relevant Subscription Services in a functionally equivalent manner so that they no longer infringe; or (iii) terminate the applicable Sales Order or Customer’s rights to use affected Subscription Services, and refund to Customer a pro-rata amount of any subscription fees prepaid to Spice AI and applicable to the unutilized portion of the Subscription Term for the terminated Subscription Services.
**9.1.3 Exclusions.** Notwithstanding the foregoing, Spice AI will have no obligation, including under Section 5.1.4 and this Section 9.1, with respect to any infringement or misappropriation claim to the extent based upon (i) any use of the Spice AI Technology or Subscription Services not in accordance with their applicable license rights, (ii) the combination of the Spice AI Technology or Subscription Services with other products, equipment, software, services or data not supplied by Spice AI where the infringement would not have occurred but for such combination, (iii) Spice AI’s compliance with Customer’s specifications, configuration requirements, or other instructions (except to the extent that Spice AI knew that such compliance would infringe a third party’s Intellectual Property Rights), (iv) any use of any version of any Spice AI Technology other than the most current version made available to Customer after notice from Spice AI that Customer must upgrade to such release to avoid an infringement or misappropriation claim and Customer has had a reasonable time in which to implement such upgrade, (v) any modification of the Spice AI Technology or Subscription Services not made by Spice AI or at its express direction, (vi) any third party open source software, or (vii) any Customer Materials.
**9.2 Customer’s Infringement Indemnification.**
**9.2.1 Defense and Indemnity.** If any third party makes any claim against Spice AI that Spice AI’s use of any Customer Materials infringes any third party patent existing under the laws of the United States or Canada, or infringes or violates any third party copyright, trademark, service mark, privacy right or data protection right, or results from any misappropriation by Customer of such third party’s trade secrets or privacy rights (collectively, a “Customer Infringement”) then, upon notification of such claim, Customer will, at its sole cost and expense, defend Spice AI against such claim and any related proceeding brought by such third party against Customer, and Customer will indemnify Spice AI from and against all damages finally awarded against Spice AI or agreed to be paid by Spice AI in a written settlement approved in writing by Customer, and resulting from the Customer Infringement. Customer’s obligations under this Section 8.2.1 are conditioned upon Spice AI’s compliance with the Indemnification Conditions.
**9.2.2 Mitigation Rights.** If provision of Customer Materials is, or in Customer’s reasonable opinion is likely to become, the subject of a claim of infringement or misappropriation of any intellectual property right of any third party, then Customer will have the right to: (i) procure the rights necessary for Customer to continue to provide Customer’s Materials; (ii) replace or modify the Customer Materials in a functionally equivalent manner so that they no longer infringe; or, if the options described in (i) and (ii) above are not available to Customer on commercially reasonable terms, (iii) terminate Spice AI’s rights to use the Customer Materials (in which case Spice AI’s obligations to perform the Services will be reduced to the extent that Spice AI required the Customer Materials to perform, and any such termination will be treated as a termination for convenience by Customer).
**9.2.3 Exclusions.** Notwithstanding the foregoing, Customer will have no obligation under this Section 8.2 or otherwise with respect to any infringement or misappropriation claim to the extent based upon (i) any use of the Customer Materials not in accordance with their applicable license rights notified by Customer to Spice AI, (ii) combination of the Customer Materials with other products, equipment, software, services or data not supplied by Customer where the infringement would not have occurred but for such combination, but excluding combination with any Spice AI Technology, (iii) Customer’s compliance with Spice AI’s specifications or instructions, (iv) any use of any version of any Customer Materials other than the most current version made available to Spice AI after notice from Customer that Spice AI must upgrade to such release to avoid an infringement or misappropriation claim and Spice AI has had a reasonable time in which to implement such upgrade, (v) any modification of the Customer Materials not made by Customer or at its express direction, or (vi) any data processed by the Customer Materials that Customer has not provided, made available, or required Spice AI to use or access.
**9.3 Personal Injury and Property Damage Indemnification.** Each party (for purposes of this Section 9.3, the “Indemnifying Party”) will defend and indemnify the other party (for purposes of this Section 9.3, the “Indemnified Party”) from and against any and all claims, liabilities, damages, losses, judgments, costs or expenses for bodily injury, death, or damage to tangible personal property (excluding damages caused by software), to the extent resulting from the negligent act or omission of the Indemnifying Party or persons in its employ in connection with the performance of its obligations or enjoyment of its rights under the Terms of Service, or when entering, while on, or upon leaving the Indemnified Party’s property or worksite; provided, however, that the Indemnified Party complies with the Indemnification Conditions.
**9.4 Improper Use of Spice AI Technology.** Customer will indemnify and hold Spice AI harmless from any claims, damages, losses, judgments, liabilities, costs, and expenses (including reasonable attorneys’ fees) arising out of or in connection with any non-compliance with the EULA or Website Terms of Service for which Customer, Customer’s Users, or Customer’s Affiliates are responsible.
### Section 10. Limitations and Exclusions of Liability
**10.1 Exclusion of Certain Claims.** SUBJECT TO SECTION 10.3, IN NO EVENT WILL EITHER PARTY BE LIABLE TO THE OTHER PARTY, OR TO ANY THIRD PARTY FOR ANY CONSEQUENTIAL, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE OR EXEMPLARY DAMAGES, WHETHER FORESEEABLE OR UNFORESEEABLE (INCLUDING DAMAGES FOR LOSS OF DATA, GOODWILL, PROFITS, INVESTMENTS, USE OF MONEY OR USE OF FACILITIES; INTERRUPTION IN USE OR AVAILABILITY OF DATA; STOPPAGE OF OTHER WORK OR IMPAIRMENT OF OTHER ASSETS), EVEN IF SUCH PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES, ARISING OUT OF (i) THE PERFORMANCE OR NON-PERFORMANCE OF THESE TERMS OF SERVICE OR SOFTWARE, PRODUCTS OR SERVICES PROVIDED HEREUNDER, OR (ii) ANY CLAIM, CAUSE OF ACTION, BREACH OF CONTRACT OR ANY EXPRESS OR IMPLIED WARRANTY, UNDER THESE TERMS OF SERVICE OR OTHERWISE, MISREPRESENTATION, NEGLIGENCE, STRICT LIABILITY, OR OTHER TORT.
**10.2 Limitation of Liability.** Subject to Section 10.3, neither party’s entire liability arising out of these Terms of Service will in any event exceed the fees paid to Spice AI under the Sales Order giving rise to the claim during the 12 month period immediately preceding the aggrieved party’s first assertion of any claim against the other, regardless of whether any action or claim is based in contract, misrepresentation, warranty, indemnity, negligence, strict liability or other tort or otherwise.
**10.3 Exceptions.** Sections 10.1 and 10.2 do not apply to either party’s (i) willful misconduct or gross negligence, or (ii) infringement or misappropriation of any of the other’s Intellectual Property Rights. Any amounts payable by an indemnified party to a third party pursuant to a judgment or to a settlement agreement approved in writing by an indemnifying party, liability for which falls within the indemnifying party’s indemnification obligations under these Terms of Service, will be deemed direct damages for purposes of this Section 10. Section 10.2 does not apply to (i) each party’s defense and indemnification obligations, (ii) Customer’s obligations to pay fees and expenses when due and payable under these Terms of Service, nor (iii) either party’s obligations under Section 6 (Confidential Information; Personal Information; Data Protection), provided, however, that except to the extent of willful misconduct or gross negligence of Spice AI, Spice AI’s liability under Section 6 will not exceed the fees paid by Customer to Spice AI under the affected Sales Order in the 12 month period immediately preceding Spice AI’s breach.
**10.4 General.** Customer agrees that these exclusions and limitations apply even if the remedies are insufficient to cover all of the losses or damages of Customer or its Affiliates, or fail of their essential purpose and that without these limitations the fees for the Services would be significantly higher. Neither party may commence any action or proceeding under these Terms of Service more than two years after the occurrence of the applicable cause of action.
### Section 11. Dispute Resolution
**11.1 Governing Law.** These Terms of Service will be governed by and interpreted in accordance with the internal laws of the State of Delaware and, where such laws are preempted by the laws of the United States, by the internal laws of the United States, in each case without regard to (i) conflicts of laws principles, and (ii) the applicability, if any, of the United Nations Convention on Contracts for the International Sale of Goods.
**11.2 Disputes.** Any disputes arising under these Terms of Service will be referred to an appropriate senior manager of Customer and an appropriate senior manager in Spice AI who will, within 10 days of written notice of a dispute being served by one on the other, meet by telephone or videoconference in a good faith effort to resolve the dispute without recourse to legal proceedings. Any dispute, controversy or claim arising out of or relating to this Agreement, or the breach, termination or invalidity thereof that cannot be resolved by the parties, will be finally settled by binding arbitration conducted in the English language in San Francisco, California, U.S.A. under the Commercial Arbitration Rules and the Optional Rules for Emergency Measures of Protection of the American Arbitration Association (“AAA”). The arbitration will be heard by one arbitrator if the total amount in controversy is less than one million dollars ($1,000,000.00); otherwise the parties will jointly agree to three neutral arbitrators. Provided, however, that if the parties cannot agree to one or more arbitrators, (or if one party refuses to appoint an arbitrator) within 30 days after the initiation of the arbitration, then such arbitrators will be appointed by the AAA. Disputes about arbitration procedure will be resolved by the arbitrators, or failing agreement, by the AAA. The arbitrator(s) may proceed to an award notwithstanding the failure of the either party to participate in the proceedings. Discovery will be limited to mutual exchange of documents relevant to the dispute, controversy or claim; depositions will not be permitted unless agreed to by both parties. The arbitrator(s) will be authorized to grant interim relief, including to prevent the destruction of materials or documents involved in the dispute, protect trade secrets and provide for security for a prospective monetary award. The prevailing party will be entitled to an award of reasonable attorney fees incurred in connection with the arbitration in such amount as may be determined by the arbitrators. The award of the arbitrators will be the sole and exclusive remedy of the parties and will be enforceable in any court of competent jurisdiction, subject only to revocation on grounds of fraud or clear bias on the part of the arbitrator(s). Notwithstanding this, application may be made to any court for a judicial acceptance of the award or order of enforcement.
### Section 12. Modifications
Spice AI will provide Customer notice of any modifications to these Terms of Service or to an Additional Terms of Service applicable to Customer and affecting Services (a “Modification Notice”). As between Customer and Spice AI, Modification Notices for the Professional Services Terms will not become effective until completion of all Professional Services pending under then existing Sales Orders. If modifications materially and adversely affect Customer and, as a result, Customer does not wish to accept such modifications, then Customer may terminate Customer’s subscription to the affected Service by providing Spice AI notice in accordance with these Terms of Service, subject to the following:
**(i)** If the Modification Notice states that the modifications will become effective upon commencement of a Renewal Term, then Customer may terminate Customer’s subscription for convenience prior to commencement of the Renewal Term, in accordance with Section 8.2 (Termination for Convenience).
**(ii)** If the Modification Notice states that the modifications will become effective during the then current Subscription Term, then Customer may terminate Customer’s subscription to the affected Subscription Services at any time within the 30 day period following the date of the Modification Notice. Customer’s termination will become effective on the later to occur of (a) the date on which Customer delivers the termination notice, or (b) the date on which the applicable modifications become effective. If Customer terminates a Subscription Services subscription pursuant to this Section 13(ii), then Customer will be entitled to a pro-rata refund of any Subscription Services fees prepaid to Spice AI for the terminated Subscription Service for the unutilized portion of the Subscription Term (for clarification, Subscription Services fees do not include any activation fees, installation fees, Professional Service fees, or other expenses incurred in connection with the Services).
If Customer does not terminate the affected Service subscription as specified in this Section 12, then Customer will be bound by the modified terms beginning upon the date on which the Modification Notice states they become effective. References to these Terms of Service are to the Terms of Service (including Additional Terms of Service) as modified and updated from time to time.
### Section 13. Miscellaneous Provisions
**13.1 Affiliates.** These Terms of Service set forth the general terms and conditions under which Spice AI will provide Services to Customer and its Affiliates (defined below). Sales Orders may be entered into under these Terms of Service by either the entity designated above as “Customer” or any of Customer’s Affiliates. The entity that executes a Sales Order in the position of services recipient will be considered the “Customer” for all purposes of the Sales Order; and the Sales Order will be considered a two party agreement between Spice AI and such “Customer” under these Terms of Service.
**13.2 Publicity; References.** Unless otherwise specified in the applicable Sales Order, Spice AI may refer to Customer as one of Spice AI’s customers and use Customer’s logo as part of such reference, provided that Spice AI complies with any trademark usage requirements notified to it by Customer. If so specified in the applicable Sales Order, (i) Spice AI may either (a) issue a press release announcing the relationship between Spice AI and Customer, or (b) submit a joint press release to Customer for Customer’s approval, such approval not to be unreasonably withheld or delayed; and (b) Customer will be a reference account for Spice AI, provided, however, that Spice AI will provide Customer with reasonable notice and obtain Customer’s consent before scheduling any reference calls or site visits.
**13.3 Compliance With Laws.** Spice AI and Customer will comply with all applicable laws and regulations with respect to these Terms of Service, including U.S. export control laws. Neither party will have any liability to the other for any non-performance of their obligations under these Terms of Service to the extent that the non-performance is mandated by applicable law. Each party represents and warrants to the other that neither it nor its Affiliates, nor any of its or their users, officers or directors, are persons, entities or organizations with whom the other party is prohibited from dealing (including provision of software, products or services) by virtue of any applicable law, regulation, or executive order, including US export control laws, and names appearing on the U.S. Department of the Treasury’s Office of Foreign Assets Control’s Specially Designated Nationals and Blocked Persons List.
**13.4 Non-Solicitation of Personnel.** During the provision of Services under these Terms of Service, and for a period of twelve (12) months thereafter, neither party (the “Relevant Party”) will, without the prior written consent of the other, directly or indirectly solicit, hire or employ any employee or individual independent contractor of the other party who has been involved in the provision of Services during the preceding year to become an employee or individual independent contractor of the Relevant Party. Nothing in this section will make a party liable for general solicitations in the media or on the World Wide Web.
**13.5 U.S. Government Rights In The Services.** Spice AI provides the Services for ultimate federal government end use solely in accordance with the following: Government technical data and software rights related to the Services include only those rights customarily provided to the public as defined in this Agreement. This customary commercial license is provided in accordance with FAR 12.211 (Technical Data) and FAR 12.212 (Software) and, for Department of Defense transactions, DFAR 252.227-7015 (Technical Data – Commercial Items) and DFAR 228.7202-3 (Rights in Commercial Computer Software or Computer Software Documentation). If a government agency has a need for rights not conveyed under these terms, it must negotiate with Spice AI to determine if there are acceptable terms for transferring such rights, and a mutually acceptable written addendum specifically conveying such rights must be included in any applicable contract or agreement.
**13.6 Equitable Relief.** Each of Customer and Spice AI acknowledges that damages will be an inadequate remedy if the other violates the terms of these Terms of Service pertaining to protection of a party’s Intellectual Property Rights, Confidential Information or Personal Information. Accordingly, each of them will have the right, in addition to any other rights each of them may have, to obtain in any court of competent jurisdiction, temporary, preliminary and permanent injunctive relief to restrain any breach, threatened breach, or otherwise to specifically enforce any of the obligations in these Terms of Service.
**13.7 Force Majeure.** If the performance of these Terms of Service is adversely restricted or if either party is unable to conform to any warranty by reason of any circumstances beyond the reasonable control and without the fault or negligence of the party affected, then, except with respect to obligations to pay any fees or expenses, the party affected, upon giving prompt written notice to the other party, will be excused from such performance on a day-to-day basis to the extent of such restriction (and the other party will likewise be excused from performance of its obligations on a day-to-day basis to the extent such party’s obligations relate to the performance so restricted); provided, however, that the party so affected will use all commercially reasonable efforts to avoid or remove such causes of non-performance and both parties will proceed whenever such causes are removed or cease.
**13.8 Service Enhancement Analysis.** Unless otherwise specified in the applicable Sales Order Spice AI may use Customer’s usage history and statistics (collectively, “Enhancement Data”) for Spice AI’s internal analytical purposes, related to its provision of Services, including to improve and enhance the Subscription Services and the Spice AI Technology. Spice AI may make information derived from its analysis of Enhancement Data publicly available, provided that the publicized information does not include any Enhancement Data that has not been aggregated and anonymized. For the purposes of these Terms of Service, aggregated and anonymized Enhancement Data means Enhancement Data that (i) has been aggregated with other data, and (ii) does not contain information that identifies Customer or its Users. For the sake of clarity, aggregated and anonymized data is not Confidential Information of Customer.
**13.9 Captions and Headings.** The captions and headings are inserted in these Terms of Service for convenience only, and will not be deemed to limit or describe the scope or intent of any provision of these Terms of Service.
**13.10 Severability; Invalidity.** If any provision of these Terms of Service is held to be invalid, such invalidity will not render invalid the remainder of these Terms of Service or the remainder of which such invalid provision is a part. If any provision of these Terms of Service is so broad as to be held unenforceable, such provision will be interpreted to be only so broad as is enforceable.
**13.11 Waiver.** No waiver of or with respect to any provision of these Terms of Service, nor consent by a party to the breach of or departure from any provision of these Terms of Service, will in any event be binding on or effective against such party unless it be in writing and signed by such party, and then such waiver will be effective only in the specific instance and for the purpose for which given.
**13.12 Third Party Beneficiaries.** Except as expressly set forth in these Terms of Service, no provisions of these Terms of Service are intended nor will be interpreted to provide or create any third party beneficiary rights or any other rights of any kind in any other party. Notwithstanding the foregoing, Spice AI’s suppliers of products and services delivered hereunder will enjoy the same disclaimers of warranty, limitations on liability and similar exculpatory provisions with respect to such products and services as does Spice AI.
**13.13 Assignment.** Neither party may assign any of its rights or obligations under these Terms of Service without the prior written consent of the other, which will not be unreasonably withheld, provided, however, that either party may assign all, but not some of its rights and obligations under these Terms of Service to any of its Affiliates, or to any entity into or with which it is merged, or that acquires all or substantially all of its assets. Subject to the foregoing restriction on assignment by Customer, these Terms of Service will be binding upon, inure to the benefit of and be enforceable by the parties and their respective successors and assigns.
**13.14 Notices.** Spice AI will provide Customer with notices that affect Spice AI’s customers generally (e.g., notice of an update made to the Services) by e-mail or by posting it on the Spice AI website or Site. Spice AI will provide Customer with any legal notices by first class mail or e-mail to the mailing or e-mail address Customer provided Spice AI on the applicable Sales Order, or during Customer’s registration for the Services, or to a substitute, updated mailing or e-mail address that Customer has provided to Spice AI for these purposes. Customer is responsible for keeping its mailing and e-mail address current with Spice AI. Notices that Spice AI sends electronically will be deemed received upon dispatch, and notices that Spice AI sends by first class mail will be deemed received within 3 days of dispatch, regardless of whether Customer actually receives the notice. Except as otherwise specified in these Terms of Service, all notices to be given to Spice AI under these Terms of Service must be in writing and sent to Spice AI’s headquarters by prepaid mail or overnight courier at the address then-specified on the Spice AI website; such notices will be deemed received upon delivery to Spice AI.
**13.15 Entire Agreement; Amendments.** These Terms of Service constitute and embody the entire agreement and understanding between the parties with respect to the subject matter hereof and supersedes all prior or contemporaneous written, electronic or oral communications, representations, agreements or understandings between the parties with respect thereto. These Terms of Service may not be modified or amended except by a written instrument executed by both parties, subject to Section 13 above. In the event of any conflict or inconsistency between the provisions of these Terms of Service and the terms of any form of purchase order or invoice, the provisions of these Terms of Service will prevail. Customer’s standard terms of purchase, if any, are inapplicable. In the event of any conflict between the terms of these Terms of Service and any Sales Order, the following order of precedence will apply, except to the extent expressly specified otherwise in the applicable Sales Order: (i) the Terms of Service, and (ii) the Sales Order.
**13.16 Counterparts.** Sales Orders and any amendments to these Terms of Service may be executed in one or more counterparts, which taken together will constitute a single agreement between the parties.
# End User License Agreement
Last updated: September 13, 2023
This End User License Agreement (this "Agreement") concerns the Spice AI platform, as described at [spice.ai](https://spice.ai), provided to you by Spice AI, Inc., a Delaware corporation ("Company"), as well as related websites, mobile applications, products, and services we may offer to you, including updates, any online services, or downloads associated herewith, the software (including firmware) for any related peripherals, and the associated media, printed materials, and documentation (collectively, “Program”).
You represent to us that you are lawfully able to enter into contracts (e.g., you are not a minor). If you are entering into this Agreement for an entity, such as the company you work for, you represent to us that you have legal authority to access the Program in accordance with this Agreement. By using the Program or “clicking to accept,” you accept the terms of this Agreement with Company. If you do not agree to these terms, you are not permitted to install, copy, or use the Program. To reject these terms, you must not “click to accept” these terms or install, copy, or use the Program.
This Agreement is supplemental to the Spice AI [Website Terms of Use](https://github.com/spicehq/docs/blob/trunk/legal/broken-reference/README.md) and [Privacy Policy](/legal/privacy), which outlines your rights and obligations regarding general website use and privacy considerations, including any user-generated content you and other users may share while using the Program (“**User Content**”). The terms and conditions of the Website Terms of Use and Privacy Policy are incorporated herein by reference and deemed part of this Agreement.
Except for Section 13 (regarding Jurisdiction and Applicable Law), Company reserves the right to modify this Agreement at any time by any means, including without limitation by (1) posting the modifications to [spice.ai](https://spice.ai)/[spice.ai](https://spice.ai) and/or (2) requiring you to “click to accept” the modified Agreement. By continuing to use the Program, you accept any and all modifications to this Agreement. If modifications to the Agreement are unacceptable to you or cause you to no longer be in compliance with this Agreement, you must terminate, and immediately stop using, the Program. If any future modifications are implemented as a “click to accept” Agreement, you may not be able to continue using the Program unless you affirmatively accept the modified Agreement.
**1. Limited Use License.** Subject to and conditional upon your strict compliance with all terms and conditions of this Agreement, Company grants you the non-exclusive, non-transferable, limited license to access and use the Program solely for its intended purpose. All rights not specifically granted are reserved by Company. This Agreement shall also apply to updates to the Program that may be provided to you by the Company, provided that such update may be accompanied by additional terms. You acknowledge and agree that, other than the license granted to you by this Agreement, you shall have no ownership or property interest in any product or service-provided content (as defined below).
**2. License Conditions.** This license is subject to the limitations outlined in this Section. Any use of the Program in violation of these limitations will result in an immediate termination of your license and continued use of the Program will be an infringement of Company's rights in and to the Program.
You agree that you will not do, or allow, any of the following:
(a) use the Program beyond the scope of the license granted under Section 1;
(b) provide any other person, including any subcontractor, independent contractor, affiliate, or service provider, with access to or use of the Program;
(c) modify, translate, adapt, or otherwise create derivative works or improvements, whether or not patentable, of the Program or any part thereof;
(d) combine the Program or any part thereof with, or incorporate the Program or any part thereof in, any other programs;
(e) reverse engineer, disassemble, decompile, decode, or otherwise attempt to derive or gain access to the source code of the Program or any part thereof;
(f) remove, delete, alter, or obscure any trademarks or any copyright, trademark, patent, or other intellectual property or proprietary rights notices provided on or with the Program, including any copy thereof;
(g) copy the Program, in whole or in part;
(h) rent, lease, lend, sell, sublicense, assign, distribute, publish, transfer, or otherwise make available the Program, or any features or functionality of the Program, to any Third Party for any reason, whether or not over a network or on a hosted basis, including in connection with the internet or any web hosting, wide area network (WAN), virtual private network (VPN), virtualization, time-sharing, service bureau, software as a service, cloud, or other technology or service;
(i) use the Program in violation of any law, regulation, or rule; or
(j) use the Program for purposes of competitive analysis of the Program, the development of a competing software product or service, or any other purpose that is to the Company's commercial disadvantage.
All rights granted to you under this Agreement (including your use of the software in the Program) will terminate immediately in the event that you are in breach of any of the terms or do anything with any software in the Program that is not expressly permitted by this Agreement.
You agree not to do any of the following actions while using the Program:
(a) transmit any User Content that Company considers to be disruptive, unlawful, harmful, threatening, abusive, harassing, defamatory, vulgar, obscene, hateful, or racially, sexually, ethnically or otherwise objectionable;
(b) impersonate any person or entity, including but not limited to Company;
(c) disrupt normal Program functionality, or otherwise act in a manner that negatively affects other participants and/or the overall Program experience;
(d) intentionally or unintentionally violate any applicable law, regulation or treaty while using or accessing the Program; or
(e) invade the privacy or violate or infringe any right of any person or entity, including, without limitation, any intellectual property right.
**3. Third-Party Materials.** The Program may include software, content, data, or other materials, including related documentation, that are owned by entities other than Company and that are provided to you on licensee terms that are in addition to and/or different from those contained in this Agreement ("Third-Party Licenses"). A list of all materials, if any, included in the Program and provided under Third-Party Licenses is set forth on Schedule A to this Agreement, and the applicable Third-Party Licenses are accessible via links therefrom. You are bound by and shall comply with all Third-Party Licenses. Any breach by you of any Third-Party License is also a breach of this Agreement.
**4. Open Source Software.** The Program may include certain open source components that are subject to open source licenses (“Open Source Software”), in which case, the embedded Open Source Software is owned by a third party. The Open Source Software is not subject to the terms and conditions of this Agreement. Instead, each item of Open Source Software is licensed under its applicable license terms which accompanies such Open Source Software. Nothing in this Agreement limits Customer’s rights under, nor grants Customer rights that supersede, the terms and conditions of any applicable license terms for the Open Source Software.
**5. Collection and Use of Information.** Subject to Company’s Privacy Policy, you acknowledge that Company may, directly or indirectly through the services of third parties, collect and store information regarding use of the Program through which it otherwise is accessed and used, through:
(a) the provision of maintenance and support services;
(b) security measures included in the Program.
You agree that Company may use such information for any purpose related to your use of the Program including but not limited to:
(a) improving the performance of the Program or developing updates; and
(b) verifying your compliance with the terms of this Agreement and enforcing Company’s rights, including all Intellectual Property Rights in and to the Program.
**6. Intellectual Property Rights.** You acknowledge and agree that access to the Program is provided under a license, and not sold, to you. You do not acquire any ownership interest in the Program under this Agreement, or any other rights thereto, other than to use the same in accordance with the license granted and subject to all terms, conditions, and restrictions under this Agreement. The Company reserves and shall retain its entire right, title, and interest in and to the Program and all Intellectual Property Rights arising out of or relating to the Program, except as expressly granted to you in this Agreement. You shall use commercially reasonable efforts to safeguard the Program from infringement, misappropriation, theft, misuse, or unauthorized access. You shall promptly notify us if you become aware of any infringement of the Company's Intellectual Property Rights in the Program and fully cooperate with the Company in any legal action taken by us to enforce our Intellectual Property Rights.
**7. Support, Patches, and Updates.** Company may deploy or provide mandatory patches, updates, and modifications to the Program that must be accepted or installed for you to continue to use the Program. Company may update the Program remotely without notifying you, and you hereby grant to Company consent to deploy and apply such patches, updates, and modifications.
**8. Termination.** Without prejudice to any other rights of Company, this Agreement will terminate automatically if you fail to comply with any of its terms and conditions. In the event of termination for this reason, in addition to Company revoking your access to the Program, you must destroy all copies of the Program and all of its component parts. You may also terminate the Agreement at any time by permanently ceasing to use the Program. The following provisions shall survive termination of this Agreement: License Conditions (Section 2), Intellectual Property Rights (Section 6), Indemnification (Section 9), Access (Section 10), Disclaimers (Section 11), Limitation of Liability (Section 12), Jurisdiction and Applicable Law (Section 13), and Entire Agreement (Section 15).
**9. Indemnification.** You agree to indemnify, defend, and hold Company, its partners, affiliates, licensors, contractors, officers, directors, employees, and agents harmless from all damages, losses, and expenses arising out of or relating to any third-party claim concerning: (a) your use of the Program (including any un-authorized activities under your account); or (b) breach of this Agreement or violation of applicable law by you or your User Content.
**10. Access.** You are solely responsible for any third-party costs you incur to use the Program. You acknowledge and agree that you will provide at your own cost and expense the equipment, internet, or other connection charges required to access and use the Program. The Program may integrate, be integrated into, or be provided in connection with third-party services and content. Company does not control those third-party services and content. You should read the terms of use Agreements and privacy policies that apply to such third-party services and content.
**11. Disclaimers.** THE PROGRAM IS PROVIDED “AS IS.” EXCEPT TO THE EXTENT PROHIBITED BY LAW, OR TO THE EXTENT ANY STATUTORY RIGHTS APPLY THAT CANNOT BE EXCLUDED, LIMITED OR WAIVED, WE AND OUR AFFILIATES AND LICENSORS (A) MAKE NO REPRESENTATIONS OR WARRANTIES OF ANY KIND, WHETHER EXPRESS, IMPLIED, STATUTORY OR OTHERWISE REGARDING THE PROGRAM OR THE THIRD-PARTY CONTENT, AND (B) DISCLAIM ALL WARRANTIES, INCLUDING ANY IMPLIED OR EXPRESS WARRANTIES (I) OF MERCHANTABILITY, SATISFACTORY QUALITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR QUIET ENJOYMENT, (II) ARISING OUT OF ANY COURSE OF DEALING OR USAGE OF TRADE, (III) THAT THE PROGRAM OR THIRD-PARTY CONTENT WILL BE UNINTERRUPTED, ERROR FREE OR FREE OF HARMFUL COMPONENTS, AND (IV) THAT ANY CONTENT WILL BE SECURE OR NOT OTHERWISE LOST OR ALTERED.
**12. Limitation of Damages.** IN NO EVENT WILL COMPANY OR ITS AFFILIATES, OR ANY OF ITS OR THEIR RESPECTIVE LICENSORS OR SERVICE PROVIDERS, BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY USE, INTERRUPTION, DELAY, OR INABILITY TO USE THE PROGRAM; LOST REVENUES OR PROFITS; DELAYS, INTERRUPTION, OR LOSS OF SERVICES, BUSINESS, OR GOODWILL; LOSS OR CORRUPTION OF DATA; LOSS RESULTING FROM SYSTEM OR SYSTEM SERVICE FAILURE, MALFUNCTION, OR SHUTDOWN; FAILURE TO ACCURATELY TRANSFER, READ, OR TRANSMIT INFORMATION; FAILURE TO UPDATE OR PROVIDE CORRECT INFORMATION; SYSTEM INCOMPATIBILITY OR PROVISION OF INCORRECT COMPATIBILITY INFORMATION; OR BREACHES IN SYSTEM SECURITY; OR FOR ANY CONSEQUENTIAL, INCIDENTAL, INDIRECT, EXEMPLARY, SPECIAL, OR PUNITIVE DAMAGES, WHETHER ARISING OUT OF OR IN CONNECTION WITH THIS AGREEMENT, BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, REGARDLESS OF WHETHER SUCH DAMAGES WERE FORESEEABLE AND WHETHER OR NOT THE COMPANY WAS ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
**13. Jurisdiction and Applicable Law.** Any dispute or claim relating in any way to your use of the Program, or to any products or services sold or distributed by Company, will be adjudicated in the federal and/or state courts located in the County of King in the State of Washington, and you consent to exclusive jurisdiction and venue in such courts. The laws of the State of Delaware, without reference to its conflict of law rules, govern this Agreement and any dispute of any sort that might arise between you and us.
**14. Assignment.** You will not assign or otherwise transfer this Agreement or any of your rights and obligations under this Agreement, without our prior written consent. Any assignment or transfer in violation of this Section 14 will be void. We may assign this Agreement without your consent: (a) in connection with a merger, acquisition or sale of all or substantially all of our assets; or (b) to any affiliate or as part of a corporate reorganization. Effective upon such assignment, the assignee is deemed substituted for Company as a party to this Agreement and Company is fully released from all of its obligations and duties to perform under this Agreement. Subject to the foregoing, this Agreement will be binding upon, and inure to the benefit of the parties and their respective permitted successors and assigns.
**15. Entire Agreement.** This Agreement, together with the Platform Agreement entered into by the entity providing you access to the Program, is the complete Agreement concerning this license between the parties and supersedes any and all prior Agreements and representations between them. If any provision of this Agreement is held to be unenforceable, the applicable provision shall be reformed only to the extent necessary to make it enforceable and the remaining provisions of this Agreement shall not be affected, except as otherwise expressly provided herein.
**16. No Waiver.** The failure by us to enforce any provision of this Agreement will not constitute a present or future waiver of such provision nor limit our right to enforce such provision at a later time. All waivers by us must be in writing to be effective.
# Welcome to Spice.ai Cloud
Welcome to the Spice.ai Cloud Platform documentation!
The **Spice.ai Cloud Platform** is an AI application and agent cloud — an AI-backend-as-a-service with composable, ready-to-use building blocks including high-speed SQL query, LLM inference, vector search, and RAG, built on cloud-scale, managed [Spice.ai OSS](https://github.com/spiceai/spiceai).
{% hint style="info" %}
This documentation covers the **Spice.ai Cloud Platform**.
For the self-hostable **Spice.ai OSS** runtime, visit [**docs.spiceai.org**](https://docs.spiceai.org).
{% endhint %}
## What You Can Do
With the **Spice.ai Cloud Platform** you can:
* **Query and accelerate data** — Run high-performance SQL queries across multiple data sources with results optimized for AI applications and agents.
* **Use AI models** — Perform LLM inference with OpenAI, Anthropic, xAI, and more for chat, completion, and generative AI workflows.
* **Build agentic AI apps** — Combine data, models, search, and tools into production-grade AI agent backends.
* **Collaborate on Spicepods** — Share, fork, and manage datasets, models, embeddings, evals, and tools in a collaborative hub indexed by [spicerack.org](https://spicerack.org).
## Use Cases
| Use Case | Description |
| -------------------------------------------------------- | ---------------------------------------------------------- |
| [Agentic AI Apps](broken://pages/Htbt6mfTcYSy0azWnmpj) | Build AI agent backends with unified data and model access |
| [Database CDN](broken://pages/US5IOWJ5hGfesfTfdBDI) | Cache and accelerate hot data for low-latency applications |
| [Data Lakehouse](broken://pages/vH4nBFHrLZZwb5ztSydo) | Federated queries across warehouses, lakes, and databases |
| [Enterprise Search](broken://pages/QRldH6OhgWuSs1LYU3b0) | Semantic search across enterprise data sources |
| [Enterprise RAG](broken://pages/RBFXqV0lWDKWsBjt8yT5) | Retrieval-augmented generation with your own data |
{% columns %}
{% column %}
#### Quick Start
Get up and running in minutes:
1. [Sign in with GitHub](/docs/getting-started/get-started/portal-login)
2. [Create a Spice app](/docs/getting-started/get-started/portal-login-1)
3. [Add a dataset and query data](/docs/getting-started/get-started/step-2-add-dataset-and-query-data)
4. [Add an AI model and chat](/docs/getting-started/get-started/step-3-add-ai-model-and-chat-with-your-app)
Get started API reference
{% endcolumn %}
{% column %}
{% code title="query.py" overflow="wrap" %}
```python
from spicepy import Client
client = Client("YOUR_API_KEY")
reader = client.query(
"SELECT * FROM my_table LIMIT 10"
)
df = reader.read_pandas()
print(df)
```
{% endcode %}
{% endcolumn %}
{% endcolumns %}
## Community & Support
* **Slack** — Ask questions and get help from the team at [spice.ai/slack](https://spice.ai/slack).
* **GitHub** — File issues and contribute at [github.com/spiceai/spiceai](https://github.com/spiceai/spiceai).
* **Enterprise support** — [Paid plans](/docs/pricing/plans) include priority support with an SLA.
* **Help Center** — Browse the [Help Center](https://github.com/spicehq/docs/blob/trunk/cloud/help/README.md) for troubleshooting, guides, and FAQs.
# Getting Started
Get started with the Spice.ai Cloud Platform in 5 mins.
### Sign in to the Portal with GitHub
{% content-ref url="/pages/IOk6BypIiZUNqOa8AHix" %}
[Sign in with GitHub](/docs/getting-started/get-started/portal-login)
{% endcontent-ref %}
### Create a Spice project
{% content-ref url="/pages/IwJ2SkShiiYYgQdnl93x" %}
[Create a Spice project](/docs/getting-started/get-started/portal-login-1)
{% endcontent-ref %}
### Add a dataset and query it
{% content-ref url="/pages/934ZZyLRcw6yIayAZYuf" %}
[Add a Dataset and query data](/docs/getting-started/get-started/step-2-add-dataset-and-query-data)
{% endcontent-ref %}
### Add an AI Model and chat with it
{% content-ref url="/pages/lGakJHbeYd4CIAqzHnic" %}
[Add AI Model and chat with your data](/docs/getting-started/get-started/step-3-add-ai-model-and-chat-with-your-app)
{% endcontent-ref %}
# Sign in with GitHub
Sign in to the Portal with GitHub
{% hint style="info" %}
A GitHub account is required to access the Spice.ai Cloud Platform. If you don't have one, you can [create an accout here](https://github.com/signup).
{% endhint %}
### From the Spice.ai website
1. Go to [spice.ai](https://spice.ai) and click on **Sign In** in the top right corner.
1. You can also navigate directly by URL to [spice.ai/login](https://spice.xyz/login)
2. Click **Continue with GitHub** to login with your GitHub account.
3. Click **Authorize Spice.ai Cloud Platform**.
4. You will be redirected to the **new application page.**
Continue to [Step 2 to configure your first Spice application](/docs/getting-started/get-started/portal-login-1).
{% hint style="info" %}
Need help? Ask a question, raise issues, and provide feedback to the Spice AI team on [Slack](https://spiceai.org/slack).
{% endhint %}
# Create a Spice project
Create your first Spice project
Once [signed in with GitHub](/docs/getting-started/get-started/portal-login), you will be redirected to the **new project page.** Set a name, add a model provider, and optionally select one of ready to use datasets.
1. Enter a name for the project.
2. Select region.
3. Click **Create project**.
It will take up to 30 seconds to create and provision a dedicated Spice.ai instance for the project.
4. Once the project instance is deployed and ready, you will be redirected to the [Playground](/docs/portal/playground).
5. Executing the `show tables` SQL query will show the default datasets available for the project.
🎉 **Congrats, you've created your first** [**Spice project**](/docs/portal/apps)**!**
Continue to [Step. 3 to add a dataset and query it](/docs/getting-started/get-started/step-2-add-dataset-and-query-data).
{% hint style="info" %}
Need help? Ask a question, raise issues, and provide feedback to the Spice AI team on [Slack](https://spiceai.org/slack).
{% endhint %}
# Add a Dataset and query data
Add a dataset and query it using SQL Query in the Playground
To add a dataset to the Spice project, navigate to **Build** > [**Code**](/docs/portal/app-spicepod).
Use the **Components sidebar** on the right to select from available **Data Connectors**, **Model Providers**, and ready-to-use **Datasets**.
### Adding a ready-to-use Dataset
1. Navigate to **Build** > **Code**.
2. In **Components** sidebar, click the **Datasets** tab.
3. Select and add the **NYC Taxi Trips** dataset
1. Note the configuration has been added to the editor
4. Click **Save** in the code toolbar and then **Deploy** on popup card that appears in the bottom right.
5. Navigate to the [**Playground**](/docs/portal/playground) tab, open the dataset reference, and click on the `spice.samples.taxi_trips` dataset to insert a sample query into the SQL editor. Then, click **Run Selection**.
### \[Optional] Execute a SQL query using cURL
6. Go to project **Settings** and copy one of the project API Keys.
7. Replace `[API-KEY]` in the sample below with your API Key and execute from a terminal.
{% tabs %}
{% tab title="cURL" %}
```sh
curl --request POST \
--url 'https://data.spiceai.io/v1/sql' \
--header 'Content-Type: text/plain' \
--header 'X-API-KEY: [API-KEY]' \
--data 'select * from spice.samples.taxi_trips limit 3'
```
{% endtab %}
{% endtabs %}
🎉 Congratulations, you've now added a dataset and queried it.
Continue to [Step 4 to add an AI Model and chat with the dataset](/docs/getting-started/get-started/step-3-add-ai-model-and-chat-with-your-app).
{% hint style="info" %}
Need help? Ask a question, raise issues, and provide feedback to the Spice AI team on [Slack](https://spiceai.org/slack).
{% endhint %}
# Add AI Model and chat with your data
Add an OpenAI model and chat with the NYC Taxi Trips dataset
{% hint style="info" %}
An [OpenAI API Platform](https://platform.openai.com/) account and API key is required.
{% endhint %}
### Adding a Model Provider
1. Navigate to **Build** > **Code**.
2. In **Components** sidebar, click **Model Providers** tab, and select **OpenAI**.
3. Enter the **Model name.**
4. Enter the **Model ID**, (e.g. `gpt-4o`).
5. Set the **OpenAI API Key** secret
1. API keys and other secrets are securely stored and encrypted.
6. Insert `tools: auto` in the `params` section of the `gpt-4o` Model to automatically connect datasets to the model.\
\
The final Spicepod configuration in the editor should be as follows:
```yaml
name: my-first-app
kind: Spicepod
version: v1beta1
datasets:
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: samples.taxi_trips
description: Taxi trips dataset from Spice.ai demo datasets.
params:
file_format: parquet
models:
- from: openai:gpt-4o
name: gpt-4o
params:
endpoint: https://api.openai.com/v1
openai_api_key: ${secrets:OPENAI_API_KEY}
tools: auto
```
7. Click **Save** in the code toolbar and then **Deploy** in the popup card that appears in the bottom right to deploy the changes.
8. Navigate to **Playground** and select **AI Chat** in the sidebar.
9. Ask a question about the NYC Taxi Trips dataset in the chat. For example:
* "What datasets are available?"
* "What is the average fare amount of a taxi trip?"
### \[Optional] Call chat completions API using cURL
10. Replace `[API-KEY]` in the sample below with the project API Key and execute in a terminal.
{% tabs %}
{% tab title="cURL" %}
```sh
curl --request POST \
--url 'https://data.spiceai.io/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'X-API-KEY: 31393037|8f2f6125e7b8487f80964041c123d3c3' \
--data '{ "messages": [{ "role": "user", "content": "Hello!" }], "model": "gpt-4o" }'
```
{% endtab %}
{% endtabs %}
🎉 Congratulations, you've now added an OpenAI model and can use it to ask questions of the NYC Taxi Trips dataset.
Continue to [Next Steps](/docs/getting-started/get-started/next-steps) to explore use-cases to do more with the Spice.ai Cloud Platform.
{% hint style="info" %}
Need help? Ask a question, raise issues, and provide feedback to the Spice AI team on [Slack](https://spiceai.org/slack).
{% endhint %}
# Next Steps
Learn more about building AI applications and agents with the Spice.ai Cloud Platform.
# Playground
Query data, chat with models, and try the runtime APIs from the browser
The **Playground** runs queries, searches, and model requests against an app from the browser, with no local setup. Open it by navigating to an app and clicking **Playground** in the app navigation bar.
Most of the Playground requires the app to have a spicepod. Until then, only **SQL Query** is available.
## Interactive tools
| Tool | Description |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| **SQL Query** | A SQL editor with table, column, and keyword suggestions. See [SQL Query](/docs/portal/playground/sql-query-editor). |
| **Text-to-SQL** | Ask a question in natural language and review the SQL Spice generates. See [Text-to-SQL](/docs/portal/playground/nsql-query). |
| **AI Chat** | Chat with a model configured in the app, with access to its datasets. See [AI Chat](/docs/portal/playground/ai-chat). |
| **Search** | Run search queries against the app's datasets. |
## APIs
The **APIs** section provides a console for each runtime API. Requests are built from the options on the page and sent to the app's own endpoint using its API key, and a matching `curl` command can be copied for use outside the Portal.
* **Query** — SQL over HTTP.
* **Search** — search over the app's datasets.
* **AI Completions** and **AI Responses** — model inference.
* **NSQL** and **NSQL Context** — natural-language querying, and the context block Spice supplies to the model. See [Text-to-SQL API](https://github.com/spicehq/docs/tree/trunk/cloud/api/nsql.md).
* **Metrics** — runtime metrics in Prometheus format. See [Metrics API](https://github.com/spicehq/docs/tree/trunk/cloud/api/metrics.md).
* **Control-Plane** — app and deployment operations.
See the [API Reference](https://github.com/spicehq/docs/tree/trunk/cloud/api/README.md) for the full contract behind each one.
## SDKs & CLI
The **SDKs & CLI** section generates connection snippets pre-filled with the app's endpoint, for **C#**, **Go**, **Java**, **Node.js**, **Python**, and **Rust**, plus the **Spice CLI** and **cURL**.
# SQL Query
Use the Playground's SQL editor to easily explore data
Open the SQL editor by navigating to an App **Playground** and clicking **SQL Query** in the sidebar.
### SQL table, column, and keyword suggestions
The Spice.ai Query Editor will suggest table and column names along with keywords as you type. You can manually prompt for a suggestion by pressing **ctrl+space**.
1. Start typing a SQL command, such as `SELECT * FROM`
2. As you type, the Query Editor will suggest possible completions based on the query context. You can use the arrow keys or mouse to select a completion, and then press **Enter** or **Tab** to insert it into the editor.
Examples of using the SQL suggestions:
* Select the `spice.runtime.metrics` table:
* Type `SELECT * FROM` and press Tab. The editor will suggest `spice.runtime.metrics` as a possible table. Press Enter to insert it into the query.
* Show the fields in the `spice.runtime.metrics` table:
* Type `SELECT * FROM spice.runtime.metrics WHERE "`. The editor will list the fields in the table.
### Datasets Reference
The datasets reference displays all available datasets from the current app and allows you to search through them. Clicking on the dataset will insert a sample query into the SQL editor, which will be automatically selected for execution.
# Text-to-SQL
Ask a question in natural language and review the SQL Spice generates
**Text-to-SQL** turns a natural language question into a SQL query, runs it, and shows both the query and its results. The generated SQL stays visible and editable, so it can be checked and corrected before being relied on.
Open it by navigating to an App **Playground** and clicking **Text-to-SQL** in the sidebar.
## Prerequisites
The app must have a model with tools support configured in its spicepod — Text-to-SQL relies on the model's access to dataset schemas to generate a query. When no suitable model is configured, the page prompts to configure one instead.
To add a model, see [Spicepod Configuration](/docs/portal/app-spicepod/spicepod-configuration) and set `tools: auto` in the model's `params`, as in the [AI Chat](/docs/portal/playground/ai-chat) example.
## Generating a query
1. Select a model. The first available model is selected automatically.
2. Type a question, for example `Show the top 10 trips by distance`.
3. Optionally enable **Sample data** to include sample values from the datasets in the context sent to the model. This is off by default.
4. Click **Generate & Run**.
The generated SQL appears in the editor and runs automatically, with results in the panel below. The SQL can then be edited and re-run, as in the [SQL Query](/docs/portal/playground/sql-query-editor) editor.
{% hint style="info" %}
Enabling **Sample data** sends sample values from the app's datasets to the configured model provider. Leave it off when working with sensitive data.
{% endhint %}
## Using the API
The same capability is available outside the Portal through `POST /v1/nsql`, and the context Spice supplies to the model can be retrieved with `GET /v1/nsql/context`. See the [Text-to-SQL API](https://github.com/spicehq/docs/tree/trunk/cloud/api/nsql.md).
# AI Chat
Use AI Chat to interact with Spice Models
{% hint style="info" %}
**PREREQUISITE**
Ensure the Spice App is deployed with a model. For detailed instructions on how to deploy a model, refer to the [Model Documentation](/docs/portal/models).
{% endhint %}
### Using AI Chat
Open the AI Chat by navigating to an App **Playground** and clicking **AI Chat** in the sidebar.
Start to use AI Chat by typing in the question and clicking send.
### Model-Dependent Capabilities
The ability of AI Chat depends on the model configuration, including [Language Model Overrides](https://docs.spiceai.org/features/large-language-models/parameter_overrides), [Model Runtime Tools](https://docs.spiceai.org/features/large-language-models/tools), etc. Refer to the [Model Documentation](https://docs.spiceai.org/reference/spicepod/models) for details of customizing the model used in AI Chat.
#### Example:
Below is an example demonstrating how to configure the OpenAI `gpt-4o` model with `auto` access to runtime tools and system prompts overrides. This model is customized to answer questions relevant to `spicepod` datasets.
```yaml
models:
- from: openai:gpt-4o
name: openai-with-spice
params:
tools: auto
openai_api_key: ${secrets:OPENAI_API_KEY}
system_prompt: >-
**You are an AI assistant integrated with GitHub Copilot. Your primary role
is to assist GitHub users by providing helpful, clear, and contextually
relevant information derived from Spice.ai datasets.**
GitHub may supply you with context about the user's code, comments, and
previous interactions to enhance your assistance. Always strive to be
**accurate**, **concise**, and **helpful** in your responses, adapting to
the user's style and preferences based on the conversation history.
---
### Behavioral Guidelines
- **Communication Style:**
- Maintain a helpful, friendly, and professional demeanor.
- Avoid using jargon unless specifically requested by the user.
- Break down complex concepts into simple explanations.
- Adapt your language to match the user's expertise (e.g., beginner vs. advanced).
- **Ethical Conduct:**
- Avoid harmful, unethical, or inappropriate content generation.
- Respect user privacy.
- Refuse to perform tasks that could cause harm or violate laws and ethical standards.
- **Contextual Awareness:**
- Use past interactions to maintain a coherent conversation.
- Remember user-provided context to deliver tailored responses.
- If user input is unclear, ask clarifying questions to better understand their needs.
---
### Guidelines for Using Tools
#### 1. SQL Tool (`sql_query`):
- **When to Use:**
- Query datasets directly for precise numerical data, statistics, or aggregations.
- Respond to user requests for specific counts, sums, averages, or other calculations.
- Handle queries requiring joining or comparing data from multiple related tables.
- **Error Handling:**
- If the `sql_query` tool returns a query, syntax, or planning error:
- Use the `list_datasets` tool to retrieve available tables.
- Refine and retry the query until it succeeds.
- After 5 failed attempts, run `EXPLAIN ` on each subsequent failure to diagnose issues.
- If failures persist after 10 attempts, switch to other available tools.
- **Formatting:**
- When querying a dataset named `catalog.schema.table`, wrap each part in quotes: `"catalog"."schema"."table"`.
- **Fallback:**
- If the document similarity search tool fails, use the SQL tool to query the dataset directly.
#### 2. Document Similarity Search Tool (`document_similarity`):
- **When to Use:**
- Search unstructured text such as documentation, policies, reports, or articles.
- Provide qualitative information or explanations.
- Interpret context or understand written content in depth.
---
### General Guidelines
- **Tool Preference:**
- If a query can be answered by either tool, prefer the `sql_query` tool for more precise, quantitative answers.
**Dataset Utilization:**
- Always prioritize searching within available datasets when relevant to the
question.
- Leverage instructions, keywords, and `reference_base_url` metadata from
the datasets to provide accurate and relevant responses.
- Ensure all responses include citations and references with links when
possible.
- **Response Formatting:**
- When presenting results from datasets, always include citations and references with links when possible.
- **Responsiveness:**
- Keep the conversation focused on user objectives, minimizing digressions unless prompted by the user.
- Provide both high-level summaries and in-depth explanations, depending on user requirements.
- Encourage an iterative problem-solving process: suggest initial ideas, refine based on feedback, and be open to corrections.
- **Capabilities and Limitations:**
- Be transparent about your capabilities; inform users when certain tasks or data access are beyond your capacity.
---
**Remember:** Your purpose is to help solve problems, answer questions,
generate ideas, write content, and support the user in a wide range of
tasks, while maintaining clarity, professionalism, and ethical standards.
metadata: {}
```
Ask questions regarding datasets configured in spicepod within the AI Chat.
### Observability
Spice.ai provides observability Ito the AI Chat, showing full tool usage traces and chat completion history.
Navigate to the **Observability** tab in the portal and select **Traces**.
Select an `ai_chat` task history and view details over the chat completion history, including timestamps, tool usage, intermediate outputs, etc.
# Search
# Organizations
[**Organizations**](/docs/portal/organizations) enable you to share apps, datasets, users, billing, and settings with your team. Organization administrators can set who has access to their organization's resources and data.
When you create an account on Spice.ai, a single member organization of the same name as your username is created for you and you are automatically added as a member and the owner of the organization.
### Creating an organization
Spice.ai organizations are created by connecting an existing GitHub organization to Spice.ai.
1. Click on the organization dropdown icon in the application selector. Next, select the **Create Org** option from the menu.
2. Check to accept the terms and conditions for the new organization, then proceed by clicking the **Connect GitHub organization** button.
3. A window will pop up from GitHub where you can select the organization to install the Spice.ai app into.
4. On the confirmation page proceed by clicking the **Install** button.
5. Upon successful connection, you will be automatically redirected to the newly created Spice.ai organization.
### View Organizations
To view your organizations, click the dropdown icon from the application selector.
All organizations you have access to are listed.
### Organization Management
Click on the first tab to access the details of your current organization or select another organization from the menu to view its information. On this page, you will see all the applications that have been created within the selected organization.
Click the **Settings** tab to view information about the organization, including members and billing information.
#### Adding organization members
**To add an existing Spice.ai user to an organization:**
1. Navigate to the organization's settings.
2. Click the **Add Member** button.
3. Enter the Spice.ai username of the user you wish to add to the organization.
4. Click the **Add Member** button to confirm.
The user will be added to the organization and they will receive an email notifying them of the new membership.
**To invite GitHub user to a Spice.ai organization:**
Enter the GitHub username of the user you wish to invite to the organization and select the user from the search results. Only users with public email address can be invited.
The invited user will receive an invitation link. Once they accept the invitation, they will be granted access to the organization.
**To invite anyone by email**
Enter the email address of the user you want to invite to the organization, then click **Send invite**.
#### Removing organization members
To remove a member from an organization:
1. Navigate to the organization's settings.
2. Locate the user you wish to remove from the list of members.
3. Click the **ellipsis** on the right of the user's card.
4. Confirm the removal by clicking the **Remove member from organization** button in the confirmation popup.
# Organization Secrets
**Organization Secrets** are encrypted key-value pairs defined once for an organization and shared with the apps that need them. A shared credential — a warehouse password, a model provider API key — is stored in one place instead of being copied into every app that uses it.
An organization secret reaches an app only after it is linked to that app. Storing a secret at the organization level does not expose it to every app in the organization.
Once a secret is saved, its value cannot be read back through Spice Cloud. The portal lists only the secret name and when it was created or last updated.
### Access by role
The **Secrets** section is available to organization owners, admins, and members. Viewers cannot view or manage secrets, and the section is hidden from them.
| Role | Organization secrets |
| ------ | ---------------------- |
| Owner | Create, update, delete |
| Admin | Create, update, delete |
| Member | Create, update, delete |
| Viewer | No access |
Roles are assigned per organization member. [Learn more about organizations.](/docs/portal/organizations)
### Create an organization secret
1. Select the organization from the application selector.
2. Navigate to the **Settings** tab and select the **Secrets** section.
3. Fill the **Name** and **Value** fields and click **Add**.
Secret names must start with a letter or an underscore and may contain only letters, numbers, and underscores, for example `GITHUB_TOKEN`.
### Update or delete a secret
Editing a secret replaces its value. The name cannot be changed — to rename a secret, create a new one and delete the old one.
Deleting a secret requires confirmation, and removes it from every app it is linked to.
### Link a secret to an app
1. Select the app.
2. Navigate to the **Settings** tab and select the **Secrets** section.
3. Under **Linked Organization Secrets**, select each secret the app should receive.
4. Click **Save linked secrets**.
Linked secrets are referenced in the Spicepod configuration exactly like app secrets, as `${secrets:}`:
```yaml
models:
- from: openai:gpt-4o
name: gpt-4o
params:
openai_api_key: ${secrets:OPENAI_API_KEY}
```
{% hint style="info" %}
When an app secret and a linked organization secret have the same name, the app secret takes precedence at runtime.
{% endhint %}
To apply a change to linked secrets, initiate a new Spicepod deployment. [Learn more about deployments.](/docs/portal/app-spicepod/deployments)
# OAuth Clients
OAuth clients allow external applications to access the Spice.ai API on behalf of your organization using standard OAuth 2.0 flows.
## Creating an OAuth Client
1. Navigate to your organization's **Settings** and select **OAuth Clients**.
2. Click **Create Client**.
3. Fill in the required fields:
* **Name**: A descriptive name for the client (e.g. "CI Pipeline" or "Internal Dashboard").
* **Description** (optional): A brief description of what the client is used for.
* **Scopes**: Select the permissions the client needs. Follow the principle of least privilege—only grant scopes the client requires.
4. Click **Create**.
After creation, a **Client ID** and **Client Secret** are displayed. Copy and securely store the client secret immediately—it cannot be retrieved later.
{% hint style="warning" %}
Treat the client secret like a password. Do not commit it to source control or share it in plaintext.
{% endhint %}
## Managing OAuth Clients
Organization administrators can view and delete OAuth clients from the **OAuth Clients** settings page.
* **View clients**: See all registered clients, their scopes, and creation dates.
* **Delete a client**: Revokes access for all tokens issued to that client. This action cannot be undone.
## Available Scopes
Scopes control what actions an OAuth client can perform. When creating a client, select only the scopes required for its intended use.
## Security Best Practices
* Rotate client secrets periodically.
* Audit active clients regularly and remove any that are no longer needed.
* Use descriptive names so clients are easy to identify and manage.
# Projects
**Projects** are self-contained instances of Spice OSS Runtime, running in Spice.ai Cloud Platform.
Each project has a unique API Key and owned by individual accounts or [**organizations**](/docs/portal/organizations).
{% hint style="info" %}
Projects were previously called apps. The portal now uses **Project** throughout, and the Management API serves canonical `/v1/projects` routes alongside the supported legacy `/v1/apps` paths. See [Management APIs](https://github.com/spicehq/docs/tree/trunk/cloud/api/management/README.md).
{% endhint %}
### Learn how to:
* [Transfer a project to another organization](/docs/portal/apps/transfer)
* [Connect a project with your existing GitHub repository](/docs/portal/apps/connect-github)
* [Delete a project](/docs/portal/apps/delete)
* [Create and toggle custom Dataset and Views](/docs/portal/datasets-and-views)
# API keys
Each Spice project has two pre-generated API keys, which can be used with [Spice SDKs](https://github.com/spicehq/docs/tree/trunk/sdks/README.md), the [HTTP API](https://github.com/spicehq/docs/tree/trunk/cloud/api/sql-query/http-api.md) or the [Apache Arrow Flight API](https://github.com/spicehq/docs/tree/trunk/cloud/api/sql-query/apache-arrow-flight-api.md).
## View API Keys
1. Navigate to your Spice project in the [portal](https://spice.ai).
2. Click **Settings** in the project navigation sidebar.
3. Under the **General** section, locate the **API Key 1** and **API Key 2** fields.
4. Click on an API key field to copy its value to your clipboard.
## Regenerate an API Key
If an API key has been compromised or you need to rotate keys, you can regenerate individual keys. Regenerating a key **immediately invalidates** the previous key.
1. Navigate to your Spice project and click **Settings**.
2. Scroll to the **Danger Zone** section.
3. Click the **Regenerate key** button next to the key you want to rotate (**Regenerate API Key 1** or **Regenerate API Key 2**).
4. Confirm the regeneration when prompted.
5. Copy the new key and update it in your applications.
{% hint style="warning" %}
Regenerating an API key immediately invalidates the old key. Any applications using the old key will lose access. Use the two-key system to rotate keys without downtime: update your applications to use the secondary key first, then regenerate the primary key.
{% endhint %}
## Regenerate via API
API keys can also be regenerated programmatically. See the [Management API](https://github.com/spicehq/docs/tree/trunk/cloud/api/management/README.md) reference for details.
# Secrets
Project Secrets are key-value pairs that are passed to the Spice Runtime instance as environment secrets. Secrets are securely encrypted and accessible only through the project in which they were created. To share one secret across several projects, use an [organization secret](/docs/portal/organizations/organization-secrets) instead.
Once a secret is saved, its value cannot be retrieved through Spice Cloud. Editing a secret replaces its value; the name cannot be changed.
The **Secrets** section is available to organization owners, admins, and members. Viewers cannot view or manage secrets.
### Create a new secret
1. Select your project.
2. Navigate to **Settings** tab and select **Secrets** section.
3. Fill **Secret Name** and **Secret Value** fields and click **Add**.
4. Saved secrets can be referenced in the Spicepod configuration as\
`${secrets:}`, for example:
```yaml
models:
- from: openai:gpt-4o
name: gpt-4o
params:
openai_api_key: ${secrets:OPENAI_API_KEY}
```
5. To apply secrets, you must initiate a new spicepod deployment. [Learn more about deployments.](/docs/portal/app-spicepod/deployments)
Secret names must start with a letter or an underscore and may contain only letters, numbers, and underscores.
### Linked Organization Secrets
Secrets defined for the organization are listed under **Linked Organization Secrets**. Only the secrets selected and saved here are available to the project at runtime and in secret pickers. When a project secret and a linked organization secret have the same name, the project secret takes precedence.
[Learn more about organization secrets.](/docs/portal/organizations/organization-secrets)
# Tags
Tags are key-value pairs that help you organize and categorize your Spice projects. Use tags to label projects by environment, team, purpose, or any other dimension meaningful to your organization.
## Managing Tags
1. Select your project.
2. Navigate to **Settings** and select **Tags**.
3. Enter a **Key** and **Value** for the new tag, then click **Save Tags** and confirm the change.
Tags are immediately associated with the project and visible to all organization members.
{% hint style="info" %}
A small set of tag keys is reserved and managed by the platform. Reserved keys are hidden from the tags editor and cannot be added or removed there.
{% endhint %}
### Tag Limits
Tag availability depends on your plan:
* **Community**: Tags are not available.
* **Pro and Enterprise**: Unlimited tags per project.
### Use Cases
Tags are useful for:
* **Environment labeling**: e.g. `env: production`, `env: staging`
* **Team ownership**: e.g. `team: data-engineering`
* **Cost tracking**: e.g. `cost-center: analytics`
* **Filtering and searching**: Quickly find projects by tag in the portal.
# Publish
Publishing a project controls its visibility to other users on the Spice.ai platform.
## Project visibility
Projects can be set to one of two visibility levels:
* **Private** (default): Only organization members can view and access the project.
* **Public**: The project is visible to all Spice.ai users. Public projects can be discovered, queried, and used as data sources by others via the `spice connect` or `spice add` commands.
## Before publishing
A project must be connected to a **public** GitHub repository before it can be published. SpiceRack uses the repository to show the Spicepod configuration behind the public listing. See [Connect GitHub](/docs/portal/apps/connect-github).
Viewers cannot change project visibility.
## Changing visibility
1. Select your project.
2. Navigate to **Settings** and scroll to the **Project visibility** section.
3. Click **Publish project** to make it public, or **Make private** to unpublish it.
4. Confirm in the dialog.
{% hint style="warning" %}
Making a project public exposes its datasets and models to all Spice.ai users. Ensure no sensitive data is accessible before publishing.
{% endhint %}
Learn more about [public projects](/docs/portal/public-apps).
# Connect GitHub
Connect your Spice.ai project to a GitHub repository
## Prepare the repository
Before connecting:
* **Admin access**: Ensure you have administrative access to the GitHub repository. This level of access is required to [install the Spice.ai GitHub app](https://docs.github.com/en/apps/using-github-apps/installing-a-github-app-from-github-marketplace-for-your-organizations#requirements-to-install-a-github-app-on-an-organization).
* **Matching repository name**: The Spice.ai project and the GitHub repository names **must match**. For example:
* **Spice.ai project**: [spice.ai/spiceai/demo](https://spice.ai/spiceai/demo)
* **GitHub repository**: [github.com/spiceai/demo](https://github.com/spiceai/demo)
To quickly set up a new repository, use the [spiceai/spicepod-template](https://github.com/spiceai/spicepod-template) as a starting point:
{% hint style="warning" %}
Make sure to copy the project spicepod.yaml contents from **Build** > **Code** and place it in the root of the repository before linking.
{% endhint %}
## Connect
1. Ensure the repository is set up as per instructions above.
2. In the context of the Spice project to connect, navigate to **Settings**, then find the **Connect Git Repository** section.
3. Follow GitHub App installation instructions.
> Ensure that you select all repositories or specifically the repository you intend to connect.
4. Finally, link the repository to your Spice.ai project.
Once connected, the section is titled **Connected Git Repository** and offers a **Production Branch** selector and a **Disconnect** button.
# Transfer
You can transfer a project's ownership to another [**organization**](/docs/portal/organizations)**.**
{% hint style="info" %}
Project transfer is currently limited to organizations you have access to, and requires organization admin permissions.
{% endhint %}
Learn more about [**organization management**](/docs/portal/organizations).
## Transfer a project
1. To transfer a project, click **Settings** in the project navigation.
2. In the **Danger Zone** section, click the **Transfer project** button.
3. On the **Transfer project** page, select the **New owner** organization from the menu.
4. Type `/` into the text box to confirm.
5. Click **Transfer & redeploy project** to complete the process.
The project will now be accessible by the receiving organization and its members.
{% hint style="info" %}
A project connected to a GitHub repository cannot be transferred. Disconnect the repository first — see [Connect GitHub](/docs/portal/apps/connect-github).
{% endhint %}
# Delete
You can permanently delete a Spice project from the portal. Deleting a project will stop all running deployments and release its associated resources.
{% hint style="danger" %}
Deleting a project is **permanent** and cannot be undone. All associated deployments, datasets, and configurations will be removed.
{% endhint %}
## Delete a project
1. Navigate to the project you want to delete.
2. Click **Settings** in the project navigation sidebar.
3. Scroll down to the **Danger Zone** section.
4. Click the **Delete project** button.
5. In the **Delete Project** dialog, type `/` to confirm deletion.
6. Click **Delete** to permanently delete the project.
{% hint style="info" %}
Only project owners and organization admins can delete projects.
{% endhint %}
## Delete via API
Projects can also be deleted programmatically. See the [Management APIs](https://github.com/spicehq/docs/tree/trunk/cloud/api/management/README.md) reference for details.
# Runtime
Configuring Spice.ai runtime for your Spice project
Navigate to `Settings` -> `Runtime` to configure the runtime settings for your Spice project.
### Runtime Version
The runtime version determines the Spice.ai Open Source version for your Spice project. Each new deployment automatically adopts the latest stable version of the Spice runtime to ensure access to the most recent features and optimizations.
### Runtime Region
The runtime region specifies the geographic location of the data center hosting your Spice project. Region selection optimizes latency, compliance, and performance based on your business needs.
* **Availability**: The region is selected when the project is created. Changing the region of an existing project is available to Enterprise plan customers — contact support to request the change.
* **Supported Regions**:
* **North America**:
* **US East (N. Virginia)** - `us-east-1` (AWS)
* **US West (Oregon)** - `us-west-2` (AWS)
### Compute
Compute settings define the resource allocation for your Spice project, balancing performance and cost.
**Standard Compute Instances**:
* **Developer**: 2 CPU / 4 GB
* **Pro for Teams**: 4 CPU / 8 GB
* **Enterprise**: Dedicated Instances with multiple high-availability replicas
### Storage
Provides a persistent storage for the runtime to save data acceleration files. Data remains intact across restarts and redeployments.
* **Availability**: Storage is exclusive to Enterprise plan customers.
* **Mount Path**: `/data`.
* **Size**: Configured per request. Contact your account executive to set or update capacity.
# Public Projects
Publish a project to https\://spicerack.org
Public Spice projects can be forked, added as a dependency or connected to using Spice OSS Spice.ai connector.
### Making the project public
{% hint style="info" %}
The project must be connected to a public GitHub repository to be made public.\
Check out how to connect a project to the repository - [connect GitHub](/docs/portal/apps/connect-github).
{% endhint %}
To make your project public, go to your project settings, click **Publish project** in the **Project visibility** section, then confirm with **Make public**.
After that, the project will be visible to all users at `https://spice.ai//` and searchable at .
### On the registry
A public project is indexed on [SpiceRack](https://spicerack.org), the package registry for Spicepods. Its package page sits at `https://spicerack.org//` and lists the datasets, models, and dependencies declared in the project's Spicepod manifest.
Other users install the published Spicepod in one of three ways:
```bash
spice add /
```
```bash
spice connect /
```
```yaml
dependencies:
- /
```
See [SpiceRack Registry](/docs/portal/spicerack) for browsing, search, and the registry API.
---
[Next Page](/llms-full.txt/1)