Retune Franka deformable lift for stable grasping - #6831
Draft
mmichelis wants to merge 1 commit into
Draft
Conversation
The Franka deformable lift tasks could not grasp reliably. The gripper tunneled through the beam between soft vertices, and the shared lift MDP terms offered no deformable-aware rewards or divergence guards, so diverged environments kept poisoning the rollout. Add a task-local mdp package with deformable-aware rewards, observations, terminations, events, a gravity curriculum, and a pose command that tracks the deformable center of mass. Retune the beam material, the collider contact and rest offsets, and the arm and hand actuator gains so the gripper no longer crushes the deformable. Enable full-surface rigid-soft contact backed by signed-distance fields on the gripper so contacts are caught between vertices. Give the cloth task a kinematic rigid support that is registered with the coupler so the cloth no longer passes through it. Select the action space through a preset, with joint-space targets as the default and task-space inverse kinematics available through presets=ik. Support the above in Newton with NewtonShapeSDFCfg to provision volume SDFs on collider shapes selected by label regex, a collision pipeline flag for full-surface rigid-soft contact, and a configurable per-body particle contact buffer so contacts are not silently dropped. Full-surface contact on the coupled proxy solver additionally requires Newton with proxy-body contact harvesting, tracked in newton-physics/newton#3756.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Franka deformable lift tasks could not grasp reliably. The gripper tunneled through the beam between soft vertices, and the shared lift MDP terms offered no deformable-aware rewards or divergence guards, so diverged environments kept poisoning the rollout.
This PR retunes
Isaac-Lift-Soft-FrankaandIsaac-Lift-Cloth-Frankafor stable grasping and adds the Newton configuration surface the retuning depends on.Task side. A task-local
mdppackage supplies deformable-aware rewards, observations, terminations, events, a gravity curriculum, and a pose command that tracks the deformable center of mass. The beam material, the collider contact and rest offsets, and the arm and hand actuator gains are retuned so the gripper no longer crushes the deformable, and full-surface rigid-soft contact backed by signed-distance fields on the gripper catches contacts that previously slipped between vertices. The cloth task gains a kinematic rigid support registered with the coupler so the cloth no longer passes through it. The action space is now selected through a preset, with joint-space targets as the default and task-space inverse kinematics available throughpresets=ik.Newton side.
NewtonShapeSDFCfgandNewtonCfg.sdf_shape_cfgsprovision volume SDFs on collider shapes selected by label regex,NewtonCollisionPipelineCfg.enable_rigid_soft_full_surface_contactturns on edge and triangle-interior soft contacts against rigid SDFs, andVBDSolverCfg.rigid_body_particle_contact_buffer_sizesizes the per-body particle contact list so contacts are not silently dropped from a body's reaction list. These additions are opt-in and change no existing behavior.Dependencies
Full-surface rigid-soft contact on the coupled proxy solver requires Newton with proxy-body contact harvesting, tracked in newton-physics/newton#3756. Until that lands and the repository's Newton revision is advanced,
Isaac-Lift-Soft-FrankaraisesNotImplementedErroron reset because upstream Newton rejects the flag unconditionally forSolverCoupledProxy. The Newton configuration added here is correct and inert in the meantime; no other task is affected.The
isaacsim_physxpreset ofIsaac-Lift-Soft-Frankarequires #6830, which lets mesh spawners acceptcollision_propsalongsidedeformable_props. The preset passescollision_props=[PhysxCollisionCfg(rest_offset=0.0005, contact_offset=0.005)]so the beam rests on the table instead of floating on PhysX's default 20 mm deformable rest offset. Without #6830 the spawner raisesValueError: Cannot use both deformable and collision properties at the same time., so the preset fails at spawn time. The Newton presets are unaffected.Note on
environments.rstdocs/source/overview/environments.rstis regenerated withtools/update_environments_rst.py, which rewrites the generated block wholesale and offers no row filter. Only the fourIsaac-Lift-Soft-Franka,Isaac-Lift-Soft-Franka-Camera,Isaac-Lift-Cloth-FrankaandIsaac-Lift-Cloth-Franka-Camerarows come from this PR, where they pick up the newik/jointaction selector. The remaining rows reflect pre-existing registry drift from #6669 and #6807 that the stale file was hiding. Separately,--checkexists but is not wired into CI or pre-commit, which is why the drift accumulated unnoticed; adding it would be a cheap follow-up.Type of change
This PR contains two breaking changes to the Franka deformable lift tasks, both documented with migration guidance in the changelog fragment:
presets=ikto restore the previous behavior.experiment_namechanges fromfranka_deformabletofranka_soft. Move existinglogs/rsl_rl/franka_deformablerun directories to resume from an older checkpoint.Screenshots
Not applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there