Porting HOST algorithm from Isaac Gym to Isaac Lab

[francescacraievich]

I’m working with the team on porting a custom reinforcement learning algorithm from Isaac Gym (Legged-Gym) to Isaac Lab using Isaac Orbit, and I’d really appreciate any advice or guidance from the community.

---

The original implementation is based on the paper:

"Learning Humanoid Standing-Up Control Across Diverse Postures" by Tao Huang and partners.

The code is built upon Nvidia’s Legged-Gym library (using Isaac Gym), and defines a multi-stage standing-up behavior for a humanoid robot. The agent is trained with PPO and leverages custom design features like:

• Multi-critic reward optimization, grouped by task stages (righting, kneeling, rising)
• Curriculum learning, with a vertical force applied at the start of training
• Action rescaler β to control joint speed/torque smoothly
• Smoothness regularization to reduce oscillatory motion
• Domain randomization for sim2real transfer and terrain diversity

---

I want to recreate the same learning environment and behavior inside Isaac Lab, using the Orbit framework. Specifically:
What I'm looking for:

1. How can I implement a multi-critic RL setup or simulate one using Orbit's task and reward structures?
2. Any recommendations for building custom curriculum learning (applying force, changing difficulty)?
3. Best practices to add a PD controller with action rescaling β to a humanoid robot in Isaac Orbit?
4. How to separate and log multiple reward components (for better interpretability / debugging)?
5. Examples of domain randomization and initial posture variation using Orbit’s Scene + Randomization API?
6. Are there existing examples or repos that implement something similar with humanoids in Isaac Lab?

---

If you’ve worked on similar problems or have seen relevant examples, I’d love to hear from you. Thanks in advance for your time and any suggestions 🙏

Best regards,

Francesca

[Coderxrohan]

Hi @francescacraievich,

That’s an exciting project. Porting from Legged-Gym to Isaac Lab (Orbit) is a non-trivial but very rewarding effort, especially given the structured approach of your algorithm. I’ve worked a bit with both, so I’ll try to point you in the right direction based on your questions:

1. Multi-Critic Reward Structure:

Isaac Orbit’s reward functions are modular and defined via RewardTerms. While multi-critic PPO isn’t built-in, you can simulate the structure by:

Splitting the reward into stage-specific components using custom flags or environment variables.

Logging these components separately (more on that below).

Training a unified policy but conditioning the reward functions dynamically per stage.

You could also explore using RL Games with Orbit, which supports custom reward shaping and has hooks you can leverage to implement multi-critic-like behavior.

2. Curriculum Learning:

Orbit provides CurriculumTerm support under RandomizationCfg, or you can implement curriculum via a callback that modifies environment parameters based on training step or agent success rate.

Example strategy:

Define curriculum variables in your environment config (like external forces).

Use a custom CurriculumManager or implement the logic in your reset() and step() hooks to update difficulty.

Also take a look at Orbit’s FrankaCubeStack curriculum example for inspiration.

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬