Skip to content

tkalceca/vllm-openvino

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenVINO plugin for vLLM

Installation

Local installation

Installing via uv

To manage the installation of the plugin and its dependencies, utilize the uv package manager: installing uv

Install vLLM OpenVINO

git clone https://github.com/vllm-project/vllm-openvino.git
cd vllm-openvino
uv venv
VLLM_TARGET_DEVICE=empty uv pip install -e .
# test
source .venv/bin/activate
python examples/offline_inference_openvino.py

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%