Composabl

Appearance

Skills Example (Lunar Lander) ​

Lunar lander is an old Atari game that has become a benchmark for autonomous AI. In this game, the player’s task is to land the spacecraft in an area defined by two flags.

Defining Skills ​

To complete the lunar lander task, the agent needs three action skills and one selector.

Stabilize ​

The stabilize skill keeps the craft stable. It minimizes the angle the craft is tilted in either direction.

Define the skill like this:

python
stabilize_skill = Skill("stabilize", stabilizeTeacher) # This skill is learned with Deep Reinforcement learning and taught through a specific set of functions defined by a machine teacher.

Move to Center ​

The move to center skill moves the craft inside the flags toward the horizontal center of the landing area.

[animated .gif of trained move to center skill]

Define the skill like this:

python
move_to_center_skill = Skill("move to center", centerTeacher)

Land ​

The land skill descends the craft gently to a resting position on the ground.

[animated .gif of the trained land skill]

Define the skill like this:

python
land_skill = Skill("land", landTeacher)

Selector ​

The selector skill determines when to utilize each of the above skills. It's like a supervisor that assigns each skill like workers based on the scenario.

Define the skill like this:

python
selector_skill = Skill("selector", selectorTeacher) # This selector skill is the top most selector in the agent, so it's goals reflect the overall goals of the agent.

Orchestration ​

Let's look a two orchestration options for this agent using the same four skills that we defined. First, we orchestrate the skills in a fixed-order sequence that stabilizes first, then moves to center, then lands.

Use this call to add the selector skill:

python
agent.add_selector_skill(selector_skill, [stabilize_skill, move_to_center_skill, land_skill], fixed_order=True, fixed_order_repeat=False)

We can also orchestrate the skills in a variable-order sequence that stabilizes, moves to center, and lands in any order.

python
agent.add_selector_skill(selector_skill, [stabilize_skill, move_to_center_skill, land_skill], fixed_order=False, fixed_order_repeat=False)

Now, let's orchestrate the first two skills in a variable-order sequence that stabilizes and moves to center in any order, first, then lands when ready (Coming Soon: Skill Groups Feature).

We call the first selector in the definition of the second selector:

python
agent.add_selector_skill(selector_skill_move_or_stabilize, [stabilize_skill, move_to_center_skill], fixed_order=False, fixed_order_repeat=False)
agent.add_selector_skill(selector_skill_land_or_fly, [selector_skill, land_skill], fixed_order=True, fixed_order_repeat=False)