Skip to content

Resolve MAP runtime failure due to CUDA runtime lib missing by specifying cuda version when packaging MAP#577

Open
MMelQin wants to merge 3 commits intomainfrom
mq/fix_notebook
Open

Resolve MAP runtime failure due to CUDA runtime lib missing by specifying cuda version when packaging MAP#577
MMelQin wants to merge 3 commits intomainfrom
mq/fix_notebook

Conversation

@MMelQin
Copy link
Collaborator

@MMelQin MMelQin commented Jan 30, 2026

The "monai-deploy package" command for creating the MAP container image has changed to use CUDA 13 base image. However

  • MD App SDK still remains dependent on holoscan SDK of the CUDA 12 version, holoscan-cu12
  • While apps that have dependency on torch will not have runtime issue due to torch installation will make cuda12 runtime shared lib available in the image, but apps such as the Simple Imaging App that do not have dependency on torch will encounter runtime error
  • MD App SDK will change to depend on CUDA 13 in the next release

So, the changes:

  • The simple change is to specify the CUDA version with the packager command option --cuda 12 so that the CUDA 12 version base image will be used, though the whole output of running the Jupyter notebook was added to this PR
  • Unrelated changes to fix formatting complaints
  • Updated commandline option value in the documents

Signed-off-by: M Q <mingmelvinq@nvidia.com>
@MMelQin MMelQin requested a review from Copilot January 30, 2026 03:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MMelQin MMelQin requested a review from Copilot January 30, 2026 03:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Signed-off-by: M Q <mingmelvinq@nvidia.com>
@MMelQin MMelQin changed the title Specify cuda version instead of using packager default Resolve MAP runtime failure due to CUDA runtime lib missing by specify cuda version when packaging MAP Jan 30, 2026
@MMelQin MMelQin requested review from bluna301 and chezhia January 30, 2026 04:01
Signed-off-by: M Q <mingmelvinq@nvidia.com>
@MMelQin MMelQin changed the title Resolve MAP runtime failure due to CUDA runtime lib missing by specify cuda version when packaging MAP Resolve MAP runtime failure due to CUDA runtime lib missing by specifying cuda version when packaging MAP Jan 30, 2026
@sonarqubecloud
Copy link

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bluna301
Copy link
Contributor

bluna301 commented Feb 2, 2026

@MMelQin - will review in full before EOD, but wanted to quickly comment to say I concur with your assessment and too found that explicitly specifying CUDA 12 in the MAP build command resolved the issue when using monai-deploy-app-sdk==3.5.0, holoscan-cu12==3.10.0, and holoscan-cli==3.10.0. Running these MAPs with the MAR gave the expected behavior. Had to dig into the Holoscan CLI - Package command documentation to see the default parameter of CUDA 13.

Thanks for taking a look into this.

Copy link
Contributor

@bluna301 bluna301 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@chezhia
Copy link
Contributor

chezhia commented Feb 4, 2026

I was able to build a test MAP with cuda flag set to 12. However, Holoscan version should be 3.10 for the flag to work (holoscan-cli>=3.10.0 and holoscan-cu12>=3.10.0). The current requirements.txt and setup.cfg do not state this explicitly, consider editing those before merge..

@MMelQin
Copy link
Collaborator Author

MMelQin commented Feb 4, 2026

I was able to build a test MAP with cuda flag set to 12. However, Holoscan version should be 3.10 for the flag to work (holoscan-cli>=3.10.0 and holoscan-cu12>=3.10.0). The current requirements.txt and setup.cfg do not state this explicitly, consider editing those before merge..

Great point! I will add some text in the Jupyter notebook example. Another thing is that I was thinking to update the Docs at the next release of the SDK as Doc build and publishing is one of the steps in releasing a new version, AND in the next release we move to CUDA 13 eliminating the need to specify --cuda on the commandline. However, did not have time to discuss this move to CUDA 13 in the WG meeting today, and PyTorch latest still by default uses CUDA 12.8 runtime lib. so we may not want to move to CUDA 13 yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants