Hi, is the demonstration collection script available somewhere? The one in robosuite repo does not seem to output demos with the right format.

Thanks!

Hi,

I am trying to run all the algorithms on the TwoArmTransport environment, and I ran into Segmentation issue when trying td3_bc, bcq and cql on our school's cluster (with GeForce GTX 1080 with 8120 MB memory). Here is an example of the segmentation fault when running the td3_bc algorithm on the low_dim dataset. I tried to investigate a little bit, but it's not clear to me what is causing this segfault issue (I've attached the error message below from the terminal). There is no such issue if I run these algorithms on my own laptop. It would be great if there are solutions to the segfault so that I can run my experiments on the cluster. Thanks a lot in advance.

SequenceDataset (
	path=robomimic_data/low_dim.hdf5
	obs_keys=('object', 'robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos')
	seq_length=1
	filter_key=none
	frame_stack=1
	pad_seq_length=True
	pad_frame_stack=True
	goal_mode=none
	cache_mode=all
	num_demos=200
	num_sequences=93752
)

 10%|#         | 519/5000 [00:28<04:03, 18.43it/s]Segmentation fault (core dumped)

I'm trying to use code from run_trained_agent.py to collect some rollout statistics and it seems like this particular script uses a .reset_to() call that resets an environment to its current state. To my knowledge, this trick isn't present in other robomimic evaluation scripts, like run_rollout() in train_utils.py.

When collecting human demonstrations on the robosuite task, was the .reset_to() trick used? I'm seeing some performance differences between the two versions of eval scripts, and I'm trying to pinpoint the issue.

https://github.com/ARISE-Initiative/robomimic/blob/b5d2aa9902825c6c652e3b08b19446d199b49590/robomimic/scripts/run_trained_agent.py#L103

Hello,

I noticed that there is a functionality to use goal-conditioned images in robomimic. I was interested to figure out how this worked and found out that we can use the get_goal() function in the env_robosuite.py file. However, this function was used only in the rollout runs and I couldn't find its use during the training. Is it possible to train using goal-conditioned observations in robomimic? I was thinking of instances such as goal-conditioned imitation learning where the image from the last time-step in the demonstration is used as the goal observation in the policy.

There was also a documentation comment in the get_goal() function noting that not all environments support this. I went to check all the environments in robosuite and saw that none of them had a _get_goal() function. If I were to write my own get_goal() function in a robosuite environment, is it possible to return an agentview image in the function as a goal observation once the task has succeeded? Would appreciate any help on this, thank you!

I get the following error when I pass in the path to transport data Environment TwoArmTransport not found. Make sure it is a registered environment among: Lift, Stack, NutAssembly, .....

Does this error arise due to robosuite not having the TwoArmTransport environment ? Is yes, how do I reproduce the paper results on transport. Any suggestions will be helpful.

I get this error when I run a coupe of scripts such as : python examples/train_bc_rnn.py --debug or even the playback_dataset.py script:

from robosuite.utils.mjcf_utils import postprocess_model_xml ImportError: cannot import name 'postprocess_model_xml' from 'robosuite.utils.mjcf_utils' (/home/xyz/anaconda3/envs/robomimic_venv/lib/python3.7/site-packages/robosuite/utils/mjcf_utils.py)

Is 'postprocess_model_xml' removed/moved or renamed?

Hi there, amazing project.

I'm considering building on top of this framework, but I would like to understand what are your plans for the future. I see that many libraries are outdated (mujoco-py not active, pytorch 1.6, python 3.7).

Which one of the below would you say is more accurate? -Project will be maintaned with basic fixes and small updates. -Project will be updated and will keep following the state of the art. -The project will be not be maintained anymore. -Other

Hi, I followed tutorials accoding to this documentation, I have exported lines in my: .bashrc, .zshrc (i am using .zsh):

export LD_LIBRARY_PATH=/home/dato/.mujoco/mujoco210/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia
export PATH="$LD_LIBRARY_PATH:$PATH"
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so

when i run: python examples/train_bc_rnn.py --debug to test my installation it gives me an error: ERROR: GLEW initalization error: Missing GL version when i check my environmental variables in .zsh/bash terminal, LD_PRELOAD env variable is present, I don't know how to proceed forward

When trying to play the trajectory:

Playing back demo key: demo_0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-b0a68c6d2406> in <module>
      2 for ep in demos[:5]:
      3     print("Playing back demo key: {}".format(ep))
----> 4     playback_trajectory(ep)
      5 
      6 # done writing video

<ipython-input-13-455a7b8775a3> in playback_trajectory(demo_key)
     13 
     14     # reset to initial state
---> 15     env.reset_to(initial_state_dict)
     16 
     17     # playback actions one by one, and render frames

~/.local/lib/python3.8/site-packages/robomimic/envs/env_robosuite.py in reset_to(self, state)
    133             self.reset()
    134             xml = postprocess_model_xml(state["model"])
--> 135             self.env.reset_from_xml_string(xml)
    136             self.env.sim.reset()
    137             if not self._is_v1:

~/.local/lib/python3.8/site-packages/robosuite/environments/base.py in reset_from_xml_string(self, xml_string)
    537 
    538         # Now reset as normal
--> 539         self.reset()
    540 
    541         # Turn off deterministic reset

~/.local/lib/python3.8/site-packages/robosuite/environments/base.py in reset(self)
    263 
    264         # Reset necessary robosuite-centric variables
--> 265         self._reset_internal()
    266         self.sim.forward()
    267         # Setup observables, reloading if

~/.local/lib/python3.8/site-packages/robosuite/environments/manipulation/lift.py in _reset_internal(self)
    387         Resets simulation internal configurations.
    388         """
--> 389         super()._reset_internal()
    390 
    391         # Reset all object positions using initializer sampler if we're not directly loading from an xml

~/.local/lib/python3.8/site-packages/robosuite/environments/robot_env.py in _reset_internal(self)
    508         """
    509         # Run superclass reset functionality
--> 510         super()._reset_internal()
    511 
    512         # Reset controllers

~/.local/lib/python3.8/site-packages/robosuite/environments/base.py in _reset_internal(self)
    316         # additional housekeeping
    317         self.sim_state_initial = self.sim.get_state()
--> 318         self._setup_references()
    319         self.cur_time = 0
    320         self.timestep = 0

~/.local/lib/python3.8/site-packages/robosuite/environments/manipulation/lift.py in _setup_references(self)
    333         in a flatten array, which is how MuJoCo stores physical simulation data.
    334         """
--> 335         super()._setup_references()
    336 
    337         # Additional object references from this env

~/.local/lib/python3.8/site-packages/robosuite/environments/robot_env.py in _setup_references(self)
    311         in a flatten array, which is how MuJoCo stores physical simulation data.
    312         """
--> 313         super()._setup_references()
    314 
    315         # Setup robot-specific references as well (note: requires resetting of sim for robot first)

~/.local/lib/python3.8/site-packages/robosuite/environments/base.py in _setup_references(self)
    216         """
    217         # Setup mappings from model to IDs
--> 218         self.model.generate_id_mappings(sim=self.sim)
    219 
    220     def _setup_observables(self):

~/.local/lib/python3.8/site-packages/robosuite/models/tasks/task.py in generate_id_mappings(self, sim)
    114             id_groups = [
    115                 get_ids(sim=sim, elements=model.visual_geoms + model.contact_geoms, element_type="geom"),
--> 116                 get_ids(sim=sim, elements=model.sites, element_type="site"),
    117             ]
    118             group_types = ("geom", "site")

~/.local/lib/python3.8/site-packages/robosuite/utils/mjcf_utils.py in get_ids(sim, elements, element_type, inplace)
    887     else:  # We assume this is an iterable array
    888         assert isinstance(elements, Iterable), "Elements must be iterable for get_id!"
--> 889         elements = [get_ids(sim=sim, elements=ele, element_type=element_type, inplace=True) for ele in elements]
    890 
    891     return elements

~/.local/lib/python3.8/site-packages/robosuite/utils/mjcf_utils.py in <listcomp>(.0)
    887     else:  # We assume this is an iterable array
    888         assert isinstance(elements, Iterable), "Elements must be iterable for get_id!"
--> 889         elements = [get_ids(sim=sim, elements=ele, element_type=element_type, inplace=True) for ele in elements]
    890 
    891     return elements

~/.local/lib/python3.8/site-packages/robosuite/utils/mjcf_utils.py in get_ids(sim, elements, element_type, inplace)
    880             elements = sim.model.body_name2id(elements)
    881         else:  # site
--> 882             elements = sim.model.site_name2id(elements)
    883     elif isinstance(elements, dict):
    884         # Iterate over each element in dict and recursively repeat

wrappers.pxi in mujoco_py.cymj.PyMjModel.site_name2id()

ValueError: No "site" with name gripper0_ee_x exists. Available "site" names = ('table_top', 'robot0_ee', 'robot0_ee_x', 'robot0_ee_z', 'robot0_ee_y', 'gripper0_ft_frame', 'gripper0_grip_site', 'gripper0_grip_site_cylinder', 'cube_default_site').

Hello,

I read the study paper for robomimic and saw that around 200-300 demonstrations were collected for various tasks. I collected 200 demonstrations for the Wipe task in robosuite and converted them and extracted image observations from the MuJoCo states as described here and created a new hdf5 file.

I then used this new converted and extracted hdf5 file to conduct training in robomimic using the train_bc_rnn.py script while including agentview_image and robot0_eye_in_hand_image in the config.observation.modalities.obs.rgb. When I commenced training, the process keeps getting killed when dataset was being loaded into memory. I ran htop on my terminal and noticed that the Mem bar was full (125G/126G) right before the process was killed. The size of my hdf5 file is around 7 GB.

When I reduced batch_size all the way to 1 and tried again, after about 50 epochs, the process gets killed again due to memory. Does this mean that I have to reduce the size of my demonstration dataset, or is there something that I may have missed here? Would appreciate any help, thank you!

Hi! I want to use SequenceDataset in my project so I don't have to write it myself, but all other modules are not relevant for me.

When I try to create a dataset, I get an error:

AssertionError: error: must call ObsUtils.initialize_obs_utils_with_obs_config first

Can I somehow use a dataset without creating (or using minimal only relevant args) configs? For example, as in https://arise-initiative.github.io/robomimic-web/docs/introduction/examples.html, but without model creation and training loop.

Thanks!

The ObsUtils.unprocess_obs_dict() seems to modify the observation dictionary that is passed in, in addition to returning it. For example, I observed that in the lines referenced below, next_obs images are between 0 and 1, while after line 147, next_obs images are between 0 and 255. This leads to a problem, as obs is derived from next_obs, which means that in the next loop around, we will pass already unprocessed images into unprocess_obs_dict(). This has led to some significant issues, as the saved images are corrupted from what is observed. As a simple fix, I wrapped the next_obs in deepcopy(next_obs) on line 147.

https://github.com/ARISE-Initiative/robomimic/blob/b5d2aa9902825c6c652e3b08b19446d199b49590/robomimic/scripts/run_trained_agent.py#L142-L147

Hi! I want to visualize my agent's rollouts. However, the resulting videos turns out to be strange, I see some color artifacts. Should it be like this?

What I do:

# on each step
render_frames.append(env.render(mode="rgb_array", width=256, height=256))

# at the end
imageio.mimsave(render_path, render_frames, fps=32)

Result:

• adding functions log_warning and flush_warnings to utils/log_utils.py, allowing us to log warnings (in yellow text by default) at the start of training, and cache them so that they are displayed once again right before training starts, so that they appear all together in a convenient location that's easy to check while debugging
• to leverage, call log_warning with the warning message, and optionally specify the text color (default is "yellow"), and whether you want to print the warning immediately (in addition to printing later when flush_warnings is called)

When running examples/train_bc_rnn.py on python 3.8, if you set config.train.hdf5_cache_mode = "low_dim" and config.train.num_data_workers = 2, the training fails. It appears to be related to an issue with global variables in utils/obs_utils.py and the torch dataloader workers not having those set properly.

Fixes issues where urllib would return "503: Service Temporary Unavailable", despite the following url being valid: http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/low_dim.hdf5

For more information, see: https://stackoverflow.com/a/25936312

And for the progress bar implementation: https://stackoverflow.com/a/37573701