vlm-experiments

This is a set of notebooks contained in the nbs directory on using mlx-vision on MacOS to serve the local VLLMs. In particular, Phi3-vision and Qwen2-VL-7B, HuggingFace Space Demo.

• Porting Phi-3-Vision to MLX
• Phi-3-Vision-MLX GitHub
• mlx-vlm GitHub
• Phi3-Vision Instruct large context
• MLX Community on HuggingFace
• Simon Willison’s Blog on Qwen
• MLX Examples
• MLX Generate Response

API Frameworks for MLX

• FastMLX

Install vlm_experiments in Development mode

This repository contains a pyproject.toml generated by uv. For backwards compatibility there is a requirements.txt that can be used to construct a virtual environment. It is recommended to use a .venv in the local repository directory

# make sure vlm_experiments package is installed in development mode
$ uv sync

# make changes under nbs/ directory
# ...

# compile to have changes apply to vlm_experiments
$ nbdev_prepare

Usage

Installation

Install latest from the GitHub repository:

$ pip install git+https://github.com/nd-crane/vlm-experiments.git

or from conda

$ conda install -c nd-crane vlm_experiments

or from pypi

$ pip install vlm_experiments

Documentation

Documentation can be found hosted on this GitHub repository’s pages. Additionally you can find package manager specific guidelines on conda and pypi respectively.

How to use

Fill me in please! Don’t forget code examples:

1+1

2