Download, run and try any Hugging Face model on your own machine.
One command takes a model from the Hub to a local chat, or to an OpenAI-, Ollama- and Anthropic-compatible API. No account. No cloud of its own — by design.
pip install "hfl[llama,mlx]" # the MLX part installs only on Apple Silicon
hfl start # suggests a model that fits this machine, then chats
hfl run hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M # or any model on the Hub
hfl serve --model hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_MAny OpenAI, Ollama or Anthropic client can then talk to http://localhost:11434.
Opening that address in a browser gives you a chat page served by HFL itself.
GGUF repos run through llama.cpp, MLX builds run natively on Apple Silicon, and safetensors checkpoints are converted and quantized on pull. Copy a repo name from the Hub and run it.
No account and no cloud service behind it. HFL talks to one server, the Hugging Face Hub, to fetch weights. With no network, everything you have pulled keeps working.
OpenAI (chat, completions, embeddings, Responses), Ollama and Anthropic Messages, with structured
tool calling and JSON-schema outputs. Existing OLLAMA_* settings are honoured.
Each load is planned (weights and KV cache) against a memory budget you set. Models load side by side while they fit; one that cannot fit is refused with the numbers. GPU-aware on NVIDIA and AMD.
GGUF models are served by llama.cpp's llama-server with 4 parallel slots; MLX models
batch on Apple Silicon. Nothing to configure.
hfl search, hfl recommend and hfl pull-smart find models that
fit your hardware. Licenses are checked before download, and each pull is verified against the
Hub's sha256 and recorded with its commit.
| How | Command |
|---|---|
| pip | pip install "hfl[llama,mlx]", then hfl install llama-server |
| pip on Windows | pip install hfl, then hfl install llama-server (no compiler needed) |
| Docker | docker run -p 127.0.0.1:11434:11434 -v hfl:/var/lib/hfl ghcr.io/ggalancs/hfl |
| Installers | .dmg, .msi and standalone binaries on the
releases page |
The Docker images and installers include llama-server. The images are signed with cosign, and each release ships a CycloneDX SBOM.
hfl search qwen3 --gguf --max-params 8 # browse the live Hub; press a number to pull
hfl run qwen3-coder # a short name, looked up on the Hub
hfl run hf.co/mlx-community/Qwen2.5-0.5B-Instruct-4bit # MLX on Apple Silicon
hfl pull meta-llama/Llama-3.3-70B-Instruct --quantize Q5_K_M --alias llama70b
hfl import ~/.lmstudio/models/lmstudio-community/Qwen3-8B-GGUF # a model you already have
hfl recommend # models that fit this machine
hfl launch claude -m qwen-coder # Claude Code or Codex on a local modelMore than 50 commands in all: sessions, model verification, benchmarks, LoRA, MCP client and
server, speech, compliance reports and more. Run hfl --help for the full list.
/v1/chat/completions /v1/completions /v1/embeddings
/v1/responses /v1/audio/speech /v1/audio/transcriptions
/v1/images/generations
/api/chat /api/generate /api/embed /api/tags
/api/ps /api/pull /api/show and more
/v1/messages /v1/messages/count_tokens, with thinking blocks and tool use
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="not-needed")
reply = client.chat.completions.create(
model="hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M",
messages=[{"role": "user", "content": "Explain quantum computing in one paragraph"}],
)
print(reply.choices[0].message.content)| Model format | Engine | Where |
|---|---|---|
| GGUF | llama.cpp's llama-server, 4 parallel slots (llama-cpp-python as a fallback) | CPU, Metal, CUDA, ROCm, Vulkan |
| MLX | mlx-lm, with continuous batching | Apple Silicon |
| safetensors | Transformers, or converted to GGUF on pull | CUDA, MPS, CPU |
| safetensors | vLLM, when chosen | NVIDIA |
Also: embeddings, text-to-speech (Bark, Coqui XTTS), speech-to-text (Whisper) and image generation (diffusers), each behind its optional extra.
The same GGUF file served by three servers, measured together (Apple M3 Max, Phi-3.5-mini Q4_K_M, medians of 5 interleaved rounds, 2026-10-03):
| HFL | Ollama | llama-server | |
|---|---|---|---|
| Decode, one request | 89.9 tok/s | 84.3 tok/s | 88.5 tok/s |
| Throughput, 4 requests at once | 140.1 tok/s | 87.1 tok/s | 145.0 tok/s |
| Throughput, 8 requests at once | 137.6 tok/s | 92.1 tok/s | 139.4 tok/s |
Raw data, the Linux CPU results and how to reproduce them are in docs/benchmarks.md.
HFL is audited end to end on macOS, Linux, Windows, an NVIDIA GPU and an AMD Instinct GPU (over 200 checks of every command, endpoint and engine) and soak-tested for hours under mixed load. More than 6,000 automated tests run with at least 90% coverage in every component.