2×24GB train PointLLM-7B

[user-deng]

Your work is truly impressive. I'm currently reproducing Point LLM and encountered an out-of-memory (OOM) issue in the first stage. I have 2×24GB RTX 4090 GPUs—are there any configuration adjustments I can make to proceed with the reproduction? For the second stage, I plan to fine-tune using LoRA. Given these computational resources, would it be feasible to implement a complete training pipeline with minor modifications to PointLLM? I look forward to your response!

[RunsenXu]

Hi,

I think you should use a new version of transformers. I would suggest you using the InternVL codebase and implement the input of point cloud modality. https://github.com/OpenGVLab/InternVL?tab=readme-ov-file#quick-start-with-huggingface

[Djerry-h]

你的工作真的令人印象深刻。我目前正在重现 Point LLM，并在第一阶段遇到了内存不足 （OOM） 问题。我有 2×24GB RTX 4090 GPU，是否可以进行任何配置调整以继续复制？对于第二阶段，我计划使用 LoRA 进行微调。考虑到这些计算资源，对 PointLLM 进行少量修改后实现完整的训练管道是否可行？我期待您的回复！

Have you used LoRA for fine-tuning? Could I take a look at your code for reference?