Deep Learning AI

The engine learns and provides recommendations to your application using a type of AI called deep reinforcement learning.

Reinforcement learning (RL) is a general framework where agents learn to perform actions in an environment so as to maximize a reward according to a policy. In deep reinforcement learning, the policy is trained by a neural network based on a deep learning algorithm.

The agent and environment continuously interact with each other. At each time step, the agent takes an action on the observation space based on its policy (aka brain), and receives a reward and the next observation from the environment. The goal is to improve the policy so as to maximize the sum of rewards (score). provides a standard interface that a deep learning algorithm can be implemented with. At launch supports two deep learning algorithms and more will be added over time.

By default, will use Deep Q-Learning. To use a different algorithm, call spice train with the parameter --learning-algorithm set to one of the following values:

dqlDeep Q-Learning
vpgVanilla Policy Gradient
sacdSoft Actor-Critic (Discrete)


spice train --learning-algorithm vpg


Learn how to add a new deep RL algorithm by implementing the interface

Vanilla Policy Gradient implementation of the Vanilla Policy Gradient algorithm (VPG)

Deep Q-Learning implementation of the Deep Q Learning algorithm (DQL)

Soft Actor-Critic implementation of the Soft Actor-Critic algorithm (SAC)