Skip to content

Skills Example (Lunar Lander)

Lunar lander is an old Atari game that has become a benchmark for autonomous AI. In this game, the player’s task is to land the spacecraft in an area defined by two flags.

Defining Skills

To complete the lunar lander task, the agent needs three action skills and one selector.

Stabilize

The stabilize skill keeps the craft stable. It minimizes the angle the craft is tilted in either direction.

Define the skill like this:

python
stabilize_skill = Skill("stabilize", stabilizeTeacher) # This skill is learned with Deep Reinforcement learning and taught through a specific set of functions defined by a machine teacher.

Move to Center

The move to center skill moves the craft inside the flags toward the horizontal center of the landing area.

[animated .gif of trained move to center skill]

Define the skill like this:

python
move_to_center_skill = Skill("move to center", centerTeacher)

Land

The land skill descends the craft gently to a resting position on the ground.

[animated .gif of the trained land skill]

Define the skill like this:

python
land_skill = Skill("land", landTeacher)

Selector

The selector skill determines when to utilize each of the above skills. It's like a supervisor that assigns each skill like workers based on the scenario.

Define the skill like this:

python
selector_skill = Skill("selector", selectorTeacher) # This selector skill is the top most selector in the agent, so it's goals reflect the overall goals of the agent.

Orchestration

Let's look a two orchestration options for this agent using the same four skills that we defined. First, we orchestrate the skills in a fixed-order sequence that stabilizes first, then moves to center, then lands.

Use this call to add the selector skill:

python
agent.add_selector_skill(selector_skill, [stabilize_skill, move_to_center_skill, land_skill], fixed_order=True, fixed_order_repeat=False)

We can also orchestrate the skills in a variable-order sequence that stabilizes, moves to center, and lands in any order.

python
agent.add_selector_skill(selector_skill, [stabilize_skill, move_to_center_skill, land_skill], fixed_order=False, fixed_order_repeat=False)

Now, let's orchestrate the first two skills in a variable-order sequence that stabilizes and moves to center in any order, first, then lands when ready (Coming Soon: Skill Groups Feature).

We call the first selector in the definition of the second selector:

python
agent.add_selector_skill(selector_skill_move_or_stabilize, [stabilize_skill, move_to_center_skill], fixed_order=False, fixed_order_repeat=False)
agent.add_selector_skill(selector_skill_land_or_fly, [selector_skill, land_skill], fixed_order=True, fixed_order_repeat=False)