Isaac Sim Environment Configuration

Prerequisites

Before using Agilebot Isaac Sim integration, ensure your system meets the following requirements:

Component	Minimum Version	Recommended Version	Status
IsaacSim	5.0	5.1	✅ Verified
CUDA	-	12.x	⚠️ Optional
ROS 2	-	Humble	⚠️ Optional
Ubuntu	22.04 LTS	22.04 LTS	✅ Verified

Note: It is recommended to use the latest NVIDIA driver version for optimal performance.

Installation Steps

Step 1: Prepare NVIDIA Driver

Ensure your system has the NVIDIA driver installed. It is recommended to use the latest version. You can download and install the driver from the official NVIDIA website.

Step 2: Install IsaacSim and IsaacLab

Download and install NVIDIA IsaacSim and IsaacLab

Step 3: Clone the Repository

# Clone the repository
git clone https://github.com/sh-agilebot/agilebot_isaac_sim.git
cd agilebot_isaac_sim

Environment Configuration

4.1 Configure RMPflow Motion Policy

Copy the motion_policy_configs/Agilebot directory to the IsaacSim configuration directory:

cp -r agilebot_integration/motion_policy_configs/Agilebot \
  ~/isaacsim/exts/isaacsim.robot_motion.motion_generation/motion_policy_configs/

4.2 Configure Agilebot Integration Code

Copy the code/agilebot directory to the IsaacSim extension directory:

cp -r agilebot_integration/code/agilebot \
  ~/isaacsim/exts/isaacsim.robot.manipulators.examples/isaacsim/robot/manipulators/examples/

Note: This path may cause package prefix conflicts in VS Code, preventing code suggestions. Users can adjust the path according to their actual needs.

4.3 Prepare Robot USD Files

Download robot USD model files from the agilebot_isaac_usd_assets repository and place them in the specified path, then configure the correct USD path in the example files.

Tip: For Isaac Sim asset cache configuration, please refer to Isaac Sim Official Documentation - Installation Tips - Assets.

Verify Installation

After completing the configuration, verify the installation by following these steps:

Step 1: Download USD Files

Download robot USD model files from the agilebot_isaac_usd_assets repository and place them in a local directory:

mkdir -p ~/agilebot_assets
cd ~/agilebot_assets
git clone https://github.com/sh-agilebot/agilebot_isaac_usd_assets.git

Step 2: Configure USD Path

In the example entry file (such as follow_target.py ), search for and modify the usd_path variable to point to the actual USD file path.

Step 3: Run the Example

cd {demos_directory}
python follow_target.py

Alternative: Standalone Project Configuration

If you want your project to run completely independently without relying on copying code to the Isaac Sim installation directory, you can refer to the implementation in the Wrist Camera Pick and Place Example:

1. Localize Configuration Files: Copy the required motion_policy_configs configuration files to your project local directory (like rmpflow/ in the example)
2. Copy Core Code: Copy the core classes (controllers, tasks, etc.) from agilebot_integration/code/agilebot/ to your project local directory
3. Adjust Import Paths: Modify Python import statements to use local relative paths instead of Isaac Sim extension paths

Benefits of This Approach:

• ✅ Completely independent project, facilitating version control and team collaboration
• ✅ No dependency on specific Isaac Sim installation paths
• ✅ More convenient for deployment and migration
• ✅ Avoid package path conflicts that cause code suggestion issues

FAQ

Q: IsaacSim fails to start, missing dependencies

A: Ensure all system dependencies are installed, especially CUDA-related libraries. Run the installation script provided by IsaacSim.

Q: Isaac Sim GUI is unresponsive

Symptoms: Window freezes, no error messages Reason: Resources are being loaded from the network Solutions:

1. Ensure stable network connection
2. Use VPN to speed up resource downloads
3. Set IsaacSim resource path in advance

Q: Unable to load robot USD files

A: Check if the USD file path is correct and ensure the file has been copied to the IsaacSim assets directory.

Q: RMPflow configuration not taking effect

A: Confirm that the configuration file has been copied to the correct path and the file name matches the robot model.

Q: URDF import collision bodies missing

Symptoms: Robot cannot perform collision detection after import Environment: Isaac Sim 5.1 Solution: Use Isaac Sim version 5.0 or lower for URDF import, then upgrade the generated USD file to version 5.1 for use

Next Steps

After installation, please refer to the Quick Start Guide to start using Agilebot Isaac Sim integration.

Isaac Ecosystem Resources

Official Documentation

• Isaac Sim Official Documentation - Complete documentation and tutorials for Isaac Sim
• Isaac Lab Documentation - Isaac Lab reinforcement learning framework documentation
• Isaac ROS Documentation - Isaac ROS acceleration package documentation
• Omniverse Documentation - Omniverse platform documentation

Learning Resources

• NVIDIA Isaac Sim Tutorials - Official tutorials and examples
• NVIDIA Developer Forum - Community support and discussion
• Isaac Sim GitHub - Isaac Sim source code and issue tracking

Related Projects

• Isaac Sim - Isaac Sim download and introduction
• Isaac Lab - Isaac Lab robot learning framework
• Isaac ROS - Isaac ROS acceleration package repository