How to select single components when installing SageMaker ?

[rfarahmand]

I believe the new SageMaker is quite large in size when installed like pip install sagemaker ? Can you confirm how big it has become since SageMaker 2.* ? and is there a way to select different modules (or dependencies ) when installing it? like pip install sagemaker[train] or pip install sagemaker[xgboost] if I only need to train XGboost for example, and do not want all the PyTorch and all other cuda dependencies that makes it quite large for some use cases ( e.g. in a lambda) ? Could you advise and/or help with this regard?

[gbeasleytombola]

I'd also like to understand if there's a good way to deal with this. Sagemaker 2.* could be installed in less than 1GB and easy enough to be part of a lambda or container deployed to lambda. Now it appears to be 8GB minimum which would mean a significant change to how we deploy some services if upgrading to Sagemaker 3.*. A way to install without PyTorch/cuda would be incredibly helpful

[wdwatkins]

Same. There should be a way to have a lightweight environment that can interact with the Sagemaker API without being bogged down with a full set of GPU dependencies. Docker images become incredibly unwieldy otherwise.

[MarvinBeGood]

Thanks for raising this topic.

I recently upgraded from SageMaker SDK v2.x to v3.x in a feature Branch and noticed a very large increase in the resulting container image size.

In my case:

• v2.x image size: ~3.19 GB
• v3.x image size: ~17 GB

This significantly affects our workflow, especially regarding:

• container build times
• CI/CD pipeline performance
• container registry storage usage
• image pull times during deployments

It seems that many Python modules are now installed by default. While this is convenient, many environments only need a subset of these dependencies. I hope there is a plan to make some dependencies optional for example via extras.