LLM
Local LLM (MLX)
Local LLM on Apple Silicon via MLX with Gemma 4, Qwen3.5, and LFM2.5 models. No API key required.
About
Local LLM (MLX) runs prompt processing entirely on your Mac. Rewrites, summaries, translations, and custom prompt workflows work without sending text to the cloud. It replaces the former Gemma 4 plugin and still recommends the dense Gemma 4 E2B (4-bit) and Gemma 4 E4B (4-bit) models, and it now also offers two smaller experimental models for Macs with less memory: Qwen3.5 2B and LFM2.5 2.6B.
Features
- Fully local on-device prompt processing via MLX
- Gemma 4, Qwen3.5, and LFM2.5 models in one plugin
- Generation temperature slider for more precise or more creative outputs
- Optional HuggingFace token for higher download rate limits
- HuggingFace token validation before it is saved
- Download, load, cancel, and unload controls for local models
- Experimental models stay visible with explicit warnings
- No API key required for the core local workflow
Available Models
| Model | Download Size | Recommended RAM | Status |
|---|---|---|---|
| Gemma 4 E2B (4-bit) | ~3.6 GB | 8 GB+ | Supported |
| Gemma 4 E4B (4-bit) | ~5.2 GB | 16 GB+ | Supported |
| Gemma 4 E4B (8-bit) | ~8 GB | 16 GB+ | Experimental |
| Gemma 4 26B-A4B (4-bit, MoE) | ~15.6 GB | 32 GB+ | Experimental |
| Qwen3.5 2B (4-bit) | ~1.8 GB | 8 GB+ | Experimental |
| LFM2.5 2.6B (4-bit) | ~1.6 GB | 8 GB+ | Experimental |
Choosing a Model
- Gemma 4 E2B or E4B gives the most reliable results for cleanup, translation, and restructuring.
- Qwen3.5 2B needs about half the memory of Gemma 4 E2B and handles dictation cleanup well, but it translates and restructures text less reliably.
- LFM2.5 2.6B thinks before it answers, so each result takes several seconds. It is released under the LFM Open License, which allows free commercial use only below USD 10 million in annual revenue.
Configuration
- Generation Temperature - Adjust the output style between more precise and more creative responses.
- HuggingFace Token - Optional. Increases download rate limits and is validated before saving. You can create one at
huggingface.co/settings/tokens. - Model - Choose which model to download and load. The 4-bit Gemma 4
E2BandE4Bmodels are the recommended path.
Setup
- Open
TypeWhisper > Settings > Integrations. - Find the
Local LLM (MLX)plugin and open its configuration. - Optionally paste a HuggingFace token and save it after validation.
- Pick a model, then click
Download & Load. Experimental models useTry anyway. - After the model is loaded, select
Local LLM (MLX)as the LLM provider in your prompt workflow.
Notes
- If you still have the former Gemma 4 plugin installed, install Local LLM (MLX), download your model again, and then remove the Gemma 4 plugin. Workflows and prompts that used Gemma 4 switch to Local LLM (MLX) once the old plugin is removed.
- Larger Gemma 4 variants are still experimental and may fail depending on hardware or the current app build.
- A HuggingFace token is optional, but it can make downloads more reliable by increasing rate limits.