[Feature]: Add support for Apple MPS(Metal Performance Shaders)

[avinashsingh77]

🚀 The feature, motivation and pitch

I have been trying to install vLLM on MacOS M3 but installation fails most of the time. I have used both pip and uv approaches to install dependencies.
Apart from issues with my machine specific installation, vLLM only has CPU enabled for MacOS.
This issue is for adding GPU support for MacOS via Apple MPS(Metal Performance Shaders)

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[avinashsingh77]

I have been able to add and test feature for detection and loading of the model via MPS. Will have the first draft PR ready for review soon

[AlpinDale]

Are you interested in implementing the attention/activations/layernorm/rope kernels in Metal? I can help with that.

[avinashsingh77]

@AlpinDale Sure, happy to collaborate on this.
This is my Draft PR - #22636
Added you as a collaborator to https://github.com/avinashsingh77/vllm/

[AlpinDale]

@AlpinDale Sure, happy to collaborate on this. This is my Draft PR - #22636 Added you as a collaborator to https://github.com/avinashsingh77/vllm/

Hi, sorry I didn't get back to you earlier. I don't have the bandwidth to work on this at the moment, but I did write a few kernels here: dphnAI/sonar#1415. Feel free to pull them to your branch. The license there is AGPL-3.0 but since I'm the sole author of those changes, I give you permission.

[AdamLouly]

vLLM can't support MPS because it’s heavily dependent on PyTorch, and PyTorch dropped MPS support after version 2.2, you'll need to bring back torch support first then work on vllm, goodluck on that!

[avinashsingh77]

@AdamLouly That's not true. PyTorch does support Apple MPS. Just that its CI is unable to test MPS specific changes as they dont have MPS infra for CI. So its supported but new features/patches are not tested and verified by CI.

[reneleonhardt]

https://docs.pytorch.org/docs/stable/notes/mps.html (Created On: May 13, 2022)

[ricky-chaoju]

For Apple Silicon / Metal support, please check vLLM Metal:
https://github.com/vllm-project/vllm-metal