Hardware requirements
These requirements are only for running a local built-in model — the whole point of which is to work fully offline with nothing leaving your machine.
Which model you get
When you first set up OrbySystems, it chooses a built-in model from how much memory your computer has. You can change it at any time in Settings → Models.
| Your computer's memory | The model OrbySystems chooses | Context |
|---|---|---|
| Less than 16 GB (8 or 12 GB) | OrbySystems Small | 32,768 tokens |
| 16 GB or more | OrbySystems Medium | Sized to your memory: about 183,000 tokens on a 16 GB Mac at the default setting |
Context is how much of a conversation, and of the data it draws on, the model can keep in mind at once. Each model card in Settings → Models offers low, medium and high; a higher setting holds more and uses more memory.
OrbySystems Large is a bigger, more capable model you can choose yourself on a computer with plenty of memory to spare: it needs about 12 GB for the model alone at its usual context.
How much memory the model uses
Measured on an Apple M1 Mac with 16 GB of memory:
| OrbySystems Small | OrbySystems Medium | |
|---|---|---|
| While answering, usually | about 4 GB | about 7.6 GB |
| At its busiest | about 6 GB | about 10 GB |
Each model card in Settings → Models shows the estimate for your own computer at the setting you have chosen.
Computers with less than 16 GB
Where the model shares your computer's memory — any Mac, or a PC without a graphics card that can hold the model — OrbySystems Small keeps to 32,768 tokens at every setting, to leave the rest of the computer as much memory as it can. Its model card says so. Closing other memory-hungry apps while you chat helps.
Graphics cards and unified memory
OrbySystems detects which kind of memory your computer has and sizes the model to it:
- A graphics card with memory of its own (most Windows and Linux gaming and workstation cards). The model runs on the card when the card has room for it — about 4.6 GB free for Small, 5.2 GB for Medium and 9.2 GB for Large — and keeps its context in the card's memory, not your computer's. On a smaller card the model runs on the processor instead, which works but is slower.
- Unified memory (Apple silicon Macs). The model shares the Mac's memory with everything else, so the figures above come out of your Mac's total.
Hint: the Settings → Models page shows what OrbySystems found when it starts: the graphics card or unified memory it sees, and how much context each model card has room for.
Docling (document ingestion)
Docling is the local service OrbySystems uses to convert PDFs and other documents into text for ingestion (and to re-read a page's layout when redacting it — see the PII Redactor). It's deliberately CPU-only: on a GPU, its layout/OCR models use a lot of VRAM — worst on image-heavy pages — for little wall-clock gain over CPU, and that VRAM is worth more to your local model. So Docling adds no VRAM requirement, but it does need roughly 4 GB of free system RAM while it's running, on top of whatever your local model needs.
Presidio (PII detection)
Presidio is OrbySystems's default engine for detecting personal information — used by the PII Redactor. It's also CPU-only, and needs roughly 1.5 GB of free system RAM while running.
Adding it up
Docling and Presidio are both started on demand — they only run (and only use this memory) while you're actually ingesting a document or checking/redacting PII, not for the whole time OrbySystems is open. If you do that alongside chatting with your local model, budget their memory on top of the model's: roughly +4 GB of system RAM while Docling is active, and +1.5 GB of system RAM while Presidio is active — both independent of whether your model itself runs on a dedicated GPU or unified memory, since these two always run on CPU/RAM regardless.
If your hardware falls short
Nothing about OrbySystems requires a local model - you can connect to an external model running in the cloud or on a separate machine via a standard API endpoint instead (see Settings → Models). However, OrbySystems operates under the assumption that there is at least one trusted model configured that can be used to chat about your data. Be clear about which LLM services you trust.