mbodied.agents.motion package∞

Submodules∞

mbodied.agents.motion.motor_agent module∞

class mbodied.agents.motion.motor_agent.MotorAgent(recorder: Literal['omit', 'auto'] | str = 'omit', recorder_kwargs=None, api_key: str = None, model_src=None, model_kwargs=None)[source]∞

Bases: Agent

Abstract base class for motor agents.

Subclassed from Agent, thus possessing the ability to make remote calls, etc.

abstract act(**kwargs) → Motion[source]∞

Generate a Motion based on given parameters.

Parameters:: **kwargs – Arbitrary keyword arguments for motor agent to act on.
Returns:: A Motion object based on the provided arguments.
Return type:: Motion

mbodied.agents.motion.openvla_agent module∞

class mbodied.agents.motion.openvla_agent.OpenVlaAgent(recorder='omit', recorder_kwargs=None, model_src=None, model_kwargs=None, **kwargs)[source]∞

Bases: MotorAgent

OpenVLA agent to generate robot actions.

Specify gradio server endpoint in model_src to make inference via API. See openvla_example_server.py for the an exmaple of the gradio server code.

actor is a gradio server taking: image, instruction, and unnorm_key as input.

Example

>>> openvla_agent = OpenVlaAgent(model_src="https://api.mbodi.ai/community-models/")
>>> openvla.act("move hand forward", Image(size=(224, 224)))
HandControl(pose=Pose6D(x=1,y=2,z=3,roll=0,pitch=0,yaw=0), grasp=JointControl(value=0))

act(instruction: str, image: Image, unnorm_key: str = 'bridge_orig') → Motion[source]∞

Act based on the instruction and image using the remote server.

Parameters:

instruction (str) – The instruction to act on.
image (Image) – The image to act on.
unnorm_key (str) – The key for the unnormalized image.

Returns:

The HandControl generated by the agent.

Return type:

Motion

mbodied.agents.motion.openvla_example_server module∞

Module contents∞

class mbodied.agents.motion.MotorAgent(recorder: Literal['omit', 'auto'] | str = 'omit', recorder_kwargs=None, api_key: str = None, model_src=None, model_kwargs=None)[source]∞