Adding a Perception Layer

Adding perception modules to your agent can provide more rich, complex, condensed, and nuanced information to the decision-making parts of the agent. For example, you might include a computer vision model in your perception layer that inputs images or video from a camera and outputs classifications of objects that it identifies.

Creating Perceptors

Each module in the perception layer for a Composabl agent inputs the sensor variables, processes those variables in some way, and outputs one or more new variables that the platform will automatically add to the list of sensors. We call these modules perceptors.

Perceptors can use any supported Python function or library to calculate outputs. They can even call machine learning and large language models or their APIs.

In this simple perceptor example we calculate the perceptor outputs that will be added as new sensor variables and we create a list of perceptors that comprise the perception layer.

class DeltaCounter():
    def __init__(self):
        self.key = "state1"
        self.previous_value = None

    def compute(self, obs):
        if self.previous_value is None:
            self.previous_value = obs[self.key]
            return {"delta_counter": 0, "state2": 0}

        delta = obs["state1"] - self.previous_value
        self.previous_value = obs["state1"]
        return {"delta_counter": delta, "state2": 0}

    def filtered_observation_space(self, obs):
        return ["state1"]

delta_counter = Perceptor(["delta_counter", "state2"], DeltaCounter, "the change in the counter from the last two steps")

perceptors = [delta_counter]

Creating a Perceptor Definition File (Optional)

A cleaner agent.py file helps keep your agent organized. So, we recommend creating a separate file (e.g. perceptors.py) to contain the perceptor definitions. Alternatively, you can include the perceptor definitions in the agent Python file. Use the Perceptor() method to create each perceptor. The first argument is the perceptor name that you can use to identify the variable during machine teaching and the second argument is a description. Create a list that contains each perceptor variable.

from composabl import Perceptor

class DeltaCounter():
    def __init__(self):
        self.key = "state1"
        self.previous_value = None

    def compute(self, obs):
        if self.previous_value is None:
            self.previous_value = obs[self.key]
            return {"delta_counter": 0, "state2": 0}

        delta = obs["state1"] - self.previous_value
        self.previous_value = obs["state1"]
        return {"delta_counter": delta, "state2": 0}

    def filtered_observation_space(self, obs):
        return ["state1"]

delta_counter = Perceptor(["delta_counter", "state2"], DeltaCounter, "the change in the counter from the last two steps")

perceptors = [delta_counter]

This is a trivially simple perceptor, so here's a list of examples of more useful and complex perceptors that you might add to an agent. For each perceptor we list the sensors, perceptor, and example agent:

Agent	Sensors	Perceptor
Drone Control	angular velocity, angular momentum, angular, acceleration	predicts stability
Autonomous Driving	camera	classifies oncoming object
Cybersecurity Network Optimization	traffic statistics	detects anomalies in network traffic
Process Control	quality measurements	predicts whether current agent actions will lead to product passing quality checks
Machine Control	microphone	classifies sounds machine is making

Here's what our agent file looks like after we import the Python file that contains the perceptors and call the add_perceptor() method to add the perceptor list to the agent.

from composabl import Agent, Skill, Sensor, Perceptor, Scenario, Runtime
from .sensors import sensors
from .perceptors import perceptors

config = {
    "target": {
        "docker": {
            "image": "composabl/sim-demo:latest"
        }
    },
    "env": {
        "name": "sim-demo",
    },
    "license": "<This is your license key>",
    "training": {},
}

runtime = Runtime(config)
agent = Agent()
agent.add_sensors(sensors)
agent.add_percepors(perceptors)