Skip to main content

Helm - Kubernetes

TL;DR​

helm repo add spiceai https://helm.spiceai.org
helm upgrade --install spiceai spiceai/spiceai

Deploy Spice using Helm in Kubernetes.

Values​

The following table lists the configurable parameters of the Spice.ai chart and their default values. Override the default values by creating a values.yaml file.

helm upgrade --install spiceai spiceai/spiceai -f values.yaml

Spicepod​

Add a custom Spicepod to be loaded by the Spice.ai runtime by overriding the spicepod value in the values.yaml file.

spicepod:
name: app
version: v1beta1
kind: Spicepod

datasets:
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: taxi_trips
description: Demo taxi trips in s3
params:
file_format: parquet
acceleration:
enabled: true

Common Parameters​

NameDescriptionValue
image.repositoryThe repository of the Docker image .spiceai
image.tagReplace with a specific version of Spice.ai to run.latest
monitoring.podMonitoring.enabledEnable Prometheus metrics collection for the Spice pods. Requires the Prometheus Operator CRDs.false
replicaCountNumber of Spice.ai replicas to run1
image.pullSecretsSpecify docker-registry secret names as an array[]
tolerationsList of node taints to tolerate[]
resourcesResource requests and limits for the Spice.ai container{}
additionalEnvAdditional environment variables to set in the Spice.ai container[]

Adding extra environment variables​

Add extra environment variables using the additionalEnv property. This can be useful when combining with the Environment Secret Store.

additionalEnv:
- name: SPICE_SECRET_SPICEAI_KEY
valueFrom:
secretKeyRef:
name: spice-secrets
key: spiceai-key

Monitoring​

The Spice Helm chart includes compatibility with the Prometheus Operator for collecting Prometheus metrics that can be visualized in the Spice Grafana dashboard. To enable this feature, set the monitoring.podMonitoring.enabled value to true. This will create a PodMonitor resource for the Spice.ai pods that will configure Prometheus to scrape metrics from the Spice.ai pods.

Install the Prometheus Operator

The easiest way to install the Prometheus Operator along with Grafana is to use the kube-prometheus-stack Helm chart.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus-stack prometheus-community/kube-prometheus-stack \
--set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
--set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

Deploy the Spice.ai Helm chart with monitoring enabled:

helm upgrade --install spiceai spiceai/spiceai --set monitoring.podMonitoring.enabled=true

Once the monitoring is enabled, import the Spice Grafana dashboard to visualize the Spice.ai metrics.

Health and Readiness​

Spice provides two HTTP endpoints to help monitor the state of the Spice runtime: /health and /ready:

Health probe​

The /health endpoint indicates whether or not the Spice process is up and running and ready to receive requests. You can add a probe to query this endpoint like so:

livenessProbe:
httpGet:
path: /health
port: 8090

In Kubernetes, this pod will not be marked as Healthy until the /health endpoint returns 200.

Readiness probe​

The /ready endpoint indicates whether or not the Spice datasets are ready. This means that while /health might indicate that Spice is up and running, until /ready reports a status of 200, your queries may return 0 results

readynessProbe:
httpGet:
path: /v1/ready
port: 8090
note

For more information on how Kubernetes uses probes to determine the health of a pod, see here

Example values.yaml​

image:
repository: spiceai/spiceai
tag: 0.10.0-alpha
replicaCount: 1
additionalEnv:
- name: SPICE_SECRET_SPICEAI_KEY
valueFrom:
secretKeyRef:
name: spice-secrets
key: spiceai-key
monitoring:
podMonitor:
enabled: true
spicepod:
name: app
version: v1beta1
kind: Spicepod

datasets:
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: taxi_trips
description: Demo taxi trips in s3
params:
file_format: parquet
acceleration:
enabled: true
# Uncomment to refresh the acceleration on a schedule
# refresh_check_interval: 1h
# refresh_mode: full