Recorder

Overview

Dataset Recorder can record your conversation and the robot’s actions to a dataset as you interact with/teach the robot. You can define any observation space and action space for the Recorder.

Here’s an example of recording observation, instruction, and the output HandControl (x, y, z, r, p, y, grasp).

observation_space = spaces.Dict({
'image': Image(size=(224, 224)).space(),
'instruction': spaces.Text(1000)
})
action_space = HandControl().space()
recorder = Recorder('example_recorder', out_dir='saved_datasets', observation_space=observation_space, action_space=action_space)

# Every time robot makes a conversation or performs an action:
recorder.record(observation={'image': image, 'instruction': instruction,}, action=hand_control)

The dataset is saved to ./saved_datasets. Learn more about augmenting, and finetuning with this dataset by filling out this form.

Dataset Replayer

The Replayer class is designed to process and manage data stored in HDF5 files generated by Recorder. It provides a variety of functionalities, including reading samples, generating statistics, extracting unique items, and converting datasets for use with HuggingFace. The Replayer also supports saving specific images during processing and offers a command-line interface for various operations.

Here’s a simple example on iterating through a dataset from Recorder with Replayer:

replayer = Replayer(path=str("path/to/dataset.h5"))
for observation, action in replayer:
    ...