Continuous Stirred Tank Reactor

The continuous stirred tank reactor (CSTR) agent controls a chemical manufacturing process. Chemicals are mixed in a tank, and the role of the agent is to regulate the temperature in the tank by controlling a cooling mechanism.

The higher the termperature, the more product is formed --- but the tank needs to stay under a temperature threshold at which dangerous irreversible heating occurs, called “thermal runaway.” The agent's job is to manage the tradeoffs between the risks and benefits of higher temperatures, keeping the tank as hot as possible while avoiding thermal runaway.

The reaction occurs in three phases, with two steady states at the beginning and end and a transition phase where the temperature is difficult to predict.

Comparing Agent Designs

There is more than one way that an intelligent agent can successfully control the CSTR temperature. The Composabl team built five different agents for the CSTR problem, and 4 out of the 5 outperformed the industry state of the art control method on both sides of the goal tradeoff, producing more product and more effectively avoiding any chance of thermal runaway.

The first two agents in the illustration are "monolithic," or single-skill, agents, the first using linear model predictive control, and the second using the AI method of deep reinforcement learning.

The second two agents represent two different ways to define and orchestrate multiple skills together.

AI Skill-Based Agent

The AI skills-based agent uses three skills, each assigned to a diffent phase of the reaction. A selector determines the phase of the reaction using sensor inputs and then passes control to the appropriate skill.

This agent avoided thermal runaway 100% of the time and also achieved high productivity.

A variant of this agent is even more effective because it adds a perception layer.

This agent uses perceptors to interpret the sensor data using a machine learning model that can predict when thermal runaway might be likely to occur. This helps the agent make better decisions about how hot it is safe to run.

Skill Group Agent

The skill group agent uses two skills working together. This agent uses an AI skill that determines the ideal set point for the temperature using deep reinforcement learning. Then it passes this information to a second skill that makes the decision about how to actually get the tank to that set point, using model predictive control.

The skill group agent uses the two technologies that were each used to create a single-skill agent. By orchestrating these two skills together in a structured way, it achieves the best results of any of the agents, with no thermal runaway and the highest production of product. This approach lets each technology do what it does best and performs better than either technology by iteslf.

Defining Skills

The skills for theese two agents can be defined as follows.

Start Reaction

The start reaction skill controls the temperature during the steady state phase at the beginning of the reaction.

Define the skill like this:

start_reaction_skill = Skill("start reaction", start_reaction_Teacher)

Control Transition

The start reaction skill controls the temperature during the steady state phase at the beginning of the reaction.

Define the skill like this:

start_reaction_skill = Skill("start reaction", start_reaction_Teacher)

Produce Product

The produce product skill controls the temperature during the steady state phase at the end of the reaction.

Define the skill like this:

produce_product_skill = Skill("produce product", produce_product_Teacher)

Selector

The selector skill determines when to utilize skills in the AI skills-based design. In this case, the selector determines the phase of the reaction and activates the appropriate action skill.

Define the skill like this:

selector_skill = Skill("selector", selectorTeacher)

Determine Set Point

The determine set point skill determines the temperature that the tank should be at to maximize productivity without risk of thermal runaway.

Define the skill like this:

determine_set_point_skill = Skill("determine set point", determine_set_point_Teacher)

Control to Set Point

The control to set point skill uses linear model predictive control (MPC) to control the cooling mechanism in the tank. This is a skill that can be programmed rather than taught.

Define the skill like this:

control_set_point_skill = Skill("control set point", trainable=False)