Hardware Requirements
Mac and Linux hardware requirements for running BodhAI.
Hardware Requirements
Mac (development and small teams)
BodhAI on Mac uses the MLX framework for Metal GPU acceleration on Apple Silicon.
| Model size | Minimum RAM | Recommended |
|---|---|---|
| 7B | 16 GB | M2 Pro or later |
| 14B | 24 GB | M3 Pro / M4 Pro |
| 32B | 48 GB | M3 Max / M4 Max |
Required:
- Apple Silicon (M1 or later) — Intel Mac is not supported
- macOS 13 Ventura or later
- 50 GB free disk space (model + document storage)
Recommended:
- M3 Pro or later for teams of 3+ concurrent users
- External SSD for document storage on large deployments
Linux (production deployments)
BodhAI on Linux uses llama-server (llama.cpp) with NVIDIA GPU acceleration (CUDA).
| Model size | Minimum VRAM | GPU examples |
|---|---|---|
| 7B | 16 GB | RTX 3090, A4000 |
| 14B | 20 GB | RTX 3090 Ti, A5000 |
| 32B | 40 GB+ | A100, 2× RTX 3090 |
Required:
- NVIDIA GPU with CUDA 12.1+
- NVIDIA Driver 525+
- Docker + NVIDIA Container Toolkit
- Ubuntu 22.04 or later (RHEL 8+ also supported)
- 100 GB free disk space
Multi-GPU:
llama-server supports splitting model layers across multiple GPUs. The 32B model can run across 2× 24 GB GPUs.
CPU-only fallback (Mac)
If mlx-lm fails to start, BodhAI falls back to llama-server (llama.cpp) on CPU. This works but is significantly slower — expect 2–5 tokens/second vs 35–55 on Metal GPU. Not recommended for team use.
Network
No special network requirements for local use. For remote access via Cloudflare Tunnel, the server needs outbound internet access on port 443.