Deep Learning AI

The Spice.ai engine learns and provides recommendations to your application using a type of AI called deep reinforcement learning.

Reinforcement learning (RL) is a general framework where agents learn to perform actions in an environment so as to maximize a reward according to a policy. In deep reinforcement learning, the policy is trained by a neural network based on a deep learning algorithm.

The agent and environment continuously interact with each other. At each time step, the agent takes an action on the observation space based on its policy (aka brain), and receives a reward and the next observation from the environment. The goal is to improve the policy so as to maximize the sum of rewards (score).

Spice.ai provides a standard interface that a deep learning algorithm can be implemented with. At launch Spice.ai supports two deep learning algorithms and more will be added over time.

By default, Spice.ai will use Vanilla Policy Gradient. To use a different algorithm, set the environment variable SPICE_DEEPRL_ALGORITHM to one of the following values:

SPICE_DEEPRL_ALGORITHMAlgorithm
vpgVanilla Policy Gradient
dqlDeep Q-Learning

Example

SPICE_DEEPRL_ALGORITHM=dql spice run

Interface

Learn how to add a new deep RL algorithm by implementing the Spice.ai interface

Vanilla Policy Gradient

Spice.ai implementation of the Vanilla Policy Gradient algorithm (VPG)

Deep Q-Learning

Spice.ai implementation of the Deep Q Learning algorithm (DQL)