Refactor for vectorized environment supported by RenderServer #44
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- fix PickClutter (obs and download location) - fix download location for cabinet - fix how obs_mode=None is dealt in make
StoneT2000
requested changes
Jan 7, 2023
- remove old camera.py - change how to deal with agent cameras
Jiayuan-Gu
added a commit
that referenced
this pull request
Feb 9, 2023
…new features (#57) * Refactor for vectorized environment supported by RenderServer (#44) * Introduce `CameraConfig` to configure cameras to know demanded resources before creating actual `sapien.CameraEntity` * All the raw environments only support `image` for visual observations. `rgbd` and `pointcloud` are supported by extra wrappers. However, this modification should be transparent to users if they use `gym.make` to create environments. * Add `VecEnv` powered by `sapien.RenderServer` * Fix PickClutter (obs and download location), download location for cabinets * Make ManiSkill2 pip installable (#51) - update setup.py - replace environment.yml with requirements.txt - remove open3d requirement - add __init__.py to all python packages * [WIP] Refactor assets management - add `format_path` to unify how path is parsed - remove unused descriptions - move assets outside package data * Rename `uuid` to `uid` * Refactor download_asset * [BC] Fix `solver_velocity_iterations` to 1 to avoid undefined behavior * Update vec_env.py * Update `flatten_state_dict` for vectorized states * Remove empty controller state * Maniskill to sb3 vec env wrapper (#53) * RecordEpisode: ignore empty transition when `save_on_reset=True` * Add extrinsic into camera params and rename * Fix typos in download_assets.py * Tutorials (#50) Merge in RL and IL tutorials using PyTorch/SB3 Adds in some utilities and wrappers as well Fixes some compatability issues with older gym * update shaders for 2.2 * Refactor examples and tools for pip wheel * Deprecate Vulkan and Kuafu to use SapienRenderer * Fix asset path for mpm envs * Fix shader dir * Support stereo depth camera * Add warning of shader_dir for MPMBaseEnv * Change render camera for some envs * Bump version to 0.4.0rc0 * Remove keyword arguments for BaseEnv * Created using Colaboratory * fix setup code * add liftcube to datasets * fix RL tutorial setup * Created using Colaboratory * Created using Colaboratory * Created using Colaboratory * Created using Colaboratory * Fix bug in manual_test_venv.py * Fix missing download links for ManiSkill1 envs * Update numpy requirement * Experimentally support visual background * Refactor stereo depth camera * Improve tutorials * support background * Refactor background * Fix a minor bug * Bump version to 0.3.1 * fix typo * Refactor download_demo * Bump version to 0.3.2 * Improve calls of render_client * Minor update - render camera for PickAndPlace - default env for demo_random_action * Add docs * update install * Update 2_reinforcement_learning.ipynb * update * Update customize_environments.ipynb * Include gdown in requirements --------- Co-authored-by: StoneT2000 <stonezt2019@gmail.com> Co-authored-by: Fanbo Xiang <fxiang@eng.ucsd.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To test this pull-request, please first install the nightly-built SAPIEN. Please refer to
examples/demo_vec_env.pyfor the usage ofVecEnv.Here are important modifications for reference:
CameraConfigto configure cameras to know demanded resources before creating actualsapien.CameraEntityimagefor visual observations.rgbdandpointcloudare supported by extra wrappers. But we make efforts to keep users unaffected if they usegym.maketo create environments.enable_robot_segto enable robot segmentation instead of specifying observation modes likergbd_robot_segVecEnvto make use ofsapien.RenderServerto accelerate visual observations