Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,7 @@ topics:
- motion primitive
- action primitive
- AtomicAction
- ActionTarget
- AtomicActionEngine
- TrajectoryBuilder
- 原子动作
Expand All @@ -425,6 +426,11 @@ topics:
- TrajectoryBuilder
- ActionResult
- WorldState
- held_objects
- ObjectActionTarget
- PlaceTarget
- PressTarget
- target ownership
- ActionCfg
- motion_source
- plan_arm_traj
Expand All @@ -433,6 +439,7 @@ topics:
- topics/atomic-actions/atomic-actions.md
source_of_truth:
- embodichain/lab/sim/atomic_actions/core.py
- embodichain/lab/sim/atomic_actions/targets.py
- embodichain/lab/sim/atomic_actions/engine.py
- embodichain/lab/sim/atomic_actions/trajectory.py
- embodichain/lab/sim/atomic_actions/primitives/
Expand Down
72 changes: 53 additions & 19 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@

| What | Path |
|---|---|
| Base classes, typed targets, configs | `embodichain/lab/sim/atomic_actions/core.py` |
| Base classes, configs, runtime state | `embodichain/lab/sim/atomic_actions/core.py` |
| Shared target contracts | `embodichain/lab/sim/atomic_actions/targets.py` |
| Engine and global registry | `embodichain/lab/sim/atomic_actions/engine.py` |
| Trajectory helpers | `embodichain/lab/sim/atomic_actions/trajectory.py` |
| Built-in primitives | `embodichain/lab/sim/atomic_actions/primitives/` |
| Built-in primitives and their targets | `embodichain/lab/sim/atomic_actions/primitives/` |
| Legacy re-export facade | `embodichain/lab/sim/atomic_actions/actions.py` |
| Public API | `embodichain/lab/sim/atomic_actions/__init__.py` |

Expand All @@ -21,7 +22,7 @@ AtomicActionEngine
│ │
│ └── TrajectoryBuilder ← IK/interpolation and MotionGenerator dispatch
└── WorldState ← last_qpos + held_object/coordinated_held_object
└── WorldState ← last_qpos + per-control-part held-object maps
```

All tensor shapes carry a leading batch dim `B = n_envs`.
Expand All @@ -30,26 +31,46 @@ All tensor shapes carry a leading batch dim `B = n_envs`.

### Typed Targets

Frozen dataclasses accepted by actions via their `TargetType` class variable.
Frozen, identity-equality dataclasses accepted by actions via their `TargetType`
class variable. Built-in and third-party targets inherit the open `ActionTarget`
marker; `BuiltinTarget` is only the closed union of targets shipped by EmbodiChain.
Each action-exclusive target is defined beside its owning action and config in
`primitives/<action>.py`. The package root re-exports all targets, so callers
should import them from `embodichain.lab.sim.atomic_actions`. A genuinely shared
target belongs in a neutral target module, not in one primitive that another
primitive must import.

`ObjectActionTarget(semantics)` is the neutral shared base for actions operating
on a semantic object. It intentionally does not define a generic pose field:
object poses, single-arm grasp poses, and dual-arm grasp pairs are distinct
contracts.

| Target | Holds | Used by |
|---|---|---|
| `EndEffectorPoseTarget(xpos)` | `(4,4)`, `(B,4,4)` or `(B,n_waypoint,4,4)` EEF pose | `MoveEndEffector`, `Place`, `Press` |
| `EndEffectorPoseTarget(xpos)` | `(4,4)`, `(B,4,4)` or `(B,n_waypoint,4,4)` EEF pose | `MoveEndEffector` |
| `PlaceTarget(xpos, tcp_symmetry)` | Release EEF pose plus optional TCP z-roll symmetry | `Place` |
| `PressTarget(xpos)` | One `(4,4)` or `(B,4,4)` contact pose | `Press` |
| `JointPositionTarget(qpos)` | `(dof,)`, `(B,dof)` or `(B,n_waypoint,dof)` joint positions | `MoveJoints` |
| `NamedJointPositionTarget(name)` | Name resolved from `MoveJointsCfg.named_joint_positions` | `MoveJoints` |
| `GraspTarget(semantics)` | `ObjectSemantics` describing the object to grasp | `PickUp` |
| `ObjectActionTarget(semantics)` | Shared semantic-object contract; no generic pose | Base of object-centric targets |
| `GraspTarget(semantics)` | Object semantics plus optional single-arm `grasp_xpos` | `PickUp` |
| `HeldObjectPoseTarget(pose)` | `(4,4)` or `(B,4,4)` target pose for the held object | `MoveHeldObject` |
| `CoordinatedPickmentTarget(...)` | Shared object + left/right object-to-EEF transforms | `CoordinatedPickment` |
| `CoordinatedPlacementTarget(...)` | Two held-object states + target poses | `CoordinatedPlacement` |
| `CoordinatedPickTarget(semantics, ...)` | Shared object + target object pose + left/right object-to-EEF transforms | `CoordinatedPickment` |
| `CoordinatedPlacementTarget(...)` | Placing/support target poses and per-call offsets | `CoordinatedPlacement` |

`CoordinatedPickmentTarget` remains an alias of `CoordinatedPickTarget`.

### WorldState

Threaded between actions:
- `last_qpos: torch.Tensor` — shape `(B, robot.dof)`, robot joint positions at the start of the next action.
- `held_object: HeldObjectState | None` — object held by one gripper.
- `coordinated_held_object: CoordinatedHeldObjectState | None` — object jointly held by two grippers.
- `held_objects: dict[str, HeldObjectState]` — independently held objects keyed by arm/control part.
- `coordinated_held_objects: dict[tuple[str, str], CoordinatedHeldObjectState]` — jointly held objects keyed by an ordered control-part pair.

`HeldObjectState` stores the object's semantics plus the object-to-EEF transform and grasp pose (both `(B, 4, 4)`).
Use `get_held_object(control_part)`, `get_coordinated_held_object(first, second)`,
and `with_updates(...)`; `with_updates` copies both maps so successor actions do
not alias their containers.

### ActionResult

Expand Down Expand Up @@ -116,24 +137,28 @@ success, traj, final_state = engine.run(steps=[("move_end_effector", target)])
|---|---|---|
| `MoveEndEffector` | `EndEffectorPoseTarget` | EEF pose move |
| `MoveJoints` | `JointPositionTarget` / `NamedJointPositionTarget` | Joint-space interpolation |
| `PickUp` | `GraspTarget` | Approach → close gripper → lift; populates `held_object` |
| `MoveHeldObject` | `HeldObjectPoseTarget` | Move held object; preserves `held_object` |
| `Place` | `EndEffectorPoseTarget` | Lower → open gripper → retract; clears `held_object` |
| `Press` | `EndEffectorPoseTarget` | Close gripper → press down → return |
| `CoordinatedPickment` | `CoordinatedPickmentTarget` | Dual-arm shared-object pick |
| `CoordinatedPlacement` | `CoordinatedPlacementTarget` | Dual-arm placement |
| `PickUp` | `GraspTarget` | Approach → close gripper → lift; populates `held_objects[cfg.control_part]` |
| `MoveHeldObject` | `HeldObjectPoseTarget` | Moves the object at `held_objects[cfg.control_part]` |
| `Place` | `PlaceTarget` | Lower → open gripper → retract; clears its control-part entry |
| `Press` | `PressTarget` | Close gripper → press down → return |
| `CoordinatedPickment` | `CoordinatedPickTarget` | Replaces the two individual entries with one coordinated held state |
| `CoordinatedPlacement` | `CoordinatedPlacementTarget` | Reads both individual held states from `WorldState` |

## Implementing a New Action

1. Create a flat `@configclass` extending `ActionCfg` with a unique `name`.
2. Reuse an existing target or define a new frozen dataclass in `core.py`.
3. Subclass `AtomicAction` directly (do not inherit from another action). Set `TargetType` and compose a `TrajectoryBuilder`.
2. Define an action-exclusive `@dataclass(frozen=True, slots=True, eq=False)`
target beside the action. Reuse or promote a target to a neutral module only
when the contract is genuinely shared. Inherit `ObjectActionTarget` when
multiple object-centric actions share only `semantics`; keep pose roles in
the concrete target.
3. Subclass `AtomicAction[YourTarget]` directly (do not inherit from another action). Set `TargetType` for runtime checking and compose a `TrajectoryBuilder`.
4. Implement `execute(self, target, state: WorldState) -> ActionResult`:
- Resolve batched targets and start qpos via `self.builder`.
- Call `self.builder.plan_arm_traj(..., cfg=self.cfg)` if using arm motion.
- Return per-env `success` (a `(B,)` tensor if any env can fail, or `torch.ones(...)` for always-succeeding paths).
- Embed the arm trajectory into full-DoF shape `(B, n_wp, robot.dof)`.
- Advance `last_qpos` to the final row and preserve/update/clear `held_object`.
- Advance `last_qpos` with `state.with_updates(...)` and preserve/update the held-object maps.
5. Register an instance with the engine or globally via `register_action(name, ActionClass)`.
6. Export from `primitives/__init__.py` and `atomic_actions/__init__.py`.

Expand All @@ -143,4 +168,13 @@ success, traj, final_state = engine.run(steps=[("move_end_effector", target)])
- **Treating `success` as scalar** — `ActionResult.success` is `(B,)` for all built-ins; use `success_all` or `success.all()` for a single bool.
- **Using `bool(action_result)` in new code** — still works but emits a `DeprecationWarning`; prefer `.success_all`.
- **Returning arm-only trajectory** — actions must embed into `(B, n_wp, robot.dof)` before returning.
- **Putting runtime held state into a target** — desired state belongs in the
target; objects already held by a control part belong in `WorldState`.
- **Using a target dataclass with default Tensor equality** — use `eq=False`;
generated dataclass equality is invalid for multi-element tensors.
- **Importing a target from a sibling primitive** — give the action its own
contract or promote the shared contract to a neutral module.
- **Putting a generic `xpos` on a shared object target** — use explicit names
such as `object_target_pose`, `grasp_xpos`, or left/right grasp transforms;
their frames and cardinalities are not interchangeable.
- **`motion_source="motion_gen"` without a MotionGenerator** — the engine passes its own `motion_generator` to each action's `TrajectoryBuilder`; if it is `None`, the action raises `ValueError` at execute time.
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,21 @@ resulting trajectories along the time axis.
CoordinatedPlacementCfg
CoordinatedPlacement

.. rubric:: Built-in Target Contracts

.. autosummary::

EndEffectorPoseTarget
JointPositionTarget
NamedJointPositionTarget
GraspTarget
HeldObjectPoseTarget
PlaceTarget
PressTarget
CoordinatedPickTarget
CoordinatedPickmentTarget
CoordinatedPlacementTarget

.. currentmodule:: embodichain.lab.sim.atomic_actions.primitives

MoveEndEffector
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,20 @@ embodichain.lab.sim.atomic_actions
AntipodalAffordance
InteractionPoints
ObjectSemantics
ActionTarget
ObjectActionTarget
EndEffectorPoseTarget
PlaceTarget
PressTarget
JointPositionTarget
NamedJointPositionTarget
GraspTarget
HeldObjectPoseTarget
CoordinatedPickTarget
CoordinatedPickmentTarget
CoordinatedPlacementTarget
Target
BuiltinTarget
HeldObjectState
CoordinatedHeldObjectState
WorldState
Expand Down Expand Up @@ -48,6 +54,7 @@ embodichain.lab.sim.atomic_actions
:maxdepth: 1
:hidden:

embodichain.lab.sim.atomic_actions.targets
embodichain.lab.sim.atomic_actions.primitives

.. currentmodule:: embodichain.lab.sim.atomic_actions
Expand All @@ -58,6 +65,8 @@ Layout
The public API is exported from ``embodichain.lab.sim.atomic_actions``. Built-in
primitive implementations live under
``embodichain.lab.sim.atomic_actions.primitives`` and
shared target contracts live in
``embodichain.lab.sim.atomic_actions.targets``.
``embodichain.lab.sim.atomic_actions.actions`` remains a compatibility re-export
for existing imports.

Expand All @@ -80,10 +89,26 @@ Core
:members:
:show-inheritance:

.. autoclass:: ActionTarget
:members:
:show-inheritance:

.. autoclass:: ObjectActionTarget
:members:
:show-inheritance:

.. autoclass:: EndEffectorPoseTarget
:members:
:show-inheritance:

.. autoclass:: PlaceTarget
:members:
:show-inheritance:

.. autoclass:: PressTarget
:members:
:show-inheritance:

.. autoclass:: JointPositionTarget
:members:
:show-inheritance:
Expand All @@ -100,16 +125,20 @@ Core
:members:
:show-inheritance:

.. autoclass:: CoordinatedPickmentTarget
.. autoclass:: CoordinatedPickTarget
:members:
:show-inheritance:

.. autodata:: CoordinatedPickmentTarget

.. autoclass:: CoordinatedPlacementTarget
:members:
:show-inheritance:

.. autodata:: Target

.. autodata:: BuiltinTarget

.. autoclass:: HeldObjectState
:members:
:show-inheritance:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
embodichain.lab.sim.atomic_actions.targets
=========================================

.. automodule:: embodichain.lab.sim.atomic_actions.targets

Overview
--------

Shared target contracts for object-centric atomic actions. These contracts
contain only fields whose meaning is consistent across multiple actions.
Action-specific pose roles remain on the concrete target dataclasses in
``embodichain.lab.sim.atomic_actions.primitives``.

.. currentmodule:: embodichain.lab.sim.atomic_actions.targets

Object Target
-------------

.. autoclass:: ObjectActionTarget
:members:
:show-inheritance:
Loading
Loading