Home / Offline AI / AI on a USB stick
Disclosure: some links on this site are affiliate links. If you buy through them, the site may earn a commission at no extra cost to you. It never changes which products we cover or what the data says about them.
AI on a USB Stick: A Portable Offline AI Kit
The short answer
A llamafile is a single computer file with a complete AI inside it: the running program and a model's weights are packaged together, and the file runs on the machine it is opened on, with nothing installed first. The project is llamafile, from Mozilla, built on the same llama.cpp engine the setup page in this corner describes. Copied onto a USB stick, one of these files is the portable form of the local AI this corner has been building toward: download it once, while the internet works, and the AI travels to whatever computer is available, including one that has never had an AI program on it.
The portability is wide, and it is also specific, and this page prints it at the strength the documentation supports. The documented systems are Windows 10 and later, macOS down to a stated kernel floor, Linux, and the three main BSD systems, on two processor families with floors the documentation dates to the mid 2000s. Three limits come from the documentation itself. Windows cannot run a single executable file over 4 GB, which splits the larger models into two files there. macOS asks for a permission step, and Apple Silicon Macs need a developer tool present before the file can set itself up. And the current release line is young enough that the project itself calls some of its platform paths best-effort. The stick math is the friendly part: the small models that suit borrowed machines are files of 1 to 3.5 GB, and paired with the offline library from the previous page, the combined kit lands on the same drive sizes that page already worked out.
One file, program and model together
llamafile began as a Mozilla Builders project and is now maintained by Mozilla.ai. It combines two pieces of software: llama.cpp, the engine that runs the models, and Cosmopolitan Libc, a library whose specialty is producing one executable that several operating systems can run. The file format that results is called APE, short for Actually Portable Executable, and it can carry extra data inside itself the way a ZIP archive does. A finished llamafile bundles three things in the one file: the executable program, a model's weights in GGUF form, and a small settings file that holds the defaults it starts with.
Running the file opens a chat in the terminal window, and the documentation adds that it also serves a chat page in the browser on the same machine, at a local address. Nothing is installed on the machine, and the project's quickstart describes the run as fully local, with no data leaving the computer. The project is free software under the Apache 2.0 license; its changes to llama.cpp are under the MIT license; and each model inside a file keeps its own license, which the project lists beside each download.
The same releases include a sibling file worth knowing about: whisperfile, a single file that transcribes speech into text, built on the same packaging idea. A stick that carries a chat model can carry it too, which makes the kit cover voice notes and recorded radio traffic as well as questions.
The documented map: which computers
The project's own summary is that a llamafile runs locally on most operating systems and processor types, with no installation. Its support page turns that summary into a list, and the list is what this page stands on:
| System | Documented support |
|---|---|
| Linux | Kernel 2.6.18 and later, which the documentation notes covers every distribution since about 2007 |
| macOS | Darwin 23.1.0 and later; graphics acceleration on Apple Silicon machines only |
| Windows | Windows 10 and later, on Intel and AMD processors only |
| FreeBSD | Version 13 and later |
| NetBSD | Version 9.2 and later, on Intel and AMD processors only |
| OpenBSD | Versions 7.0 to 7.4, on Intel and AMD processors only |
Read the list at that strength. It is six named systems with versions, not a promise about every computer, and one entry, OpenBSD, is a closed range that newer releases fall outside. Below the stated macOS floor, the support page says older kernels should work but are untested by the project, which is a weaker claim and is printed here as one.
On Intel and AMD machines, the processor needs a feature called AVX, which the documentation dates to Intel Core processors from about 2006 and AMD's K8 line from about 2003; faster instruction sets are switched on at runtime when a machine has them. On ARM machines the floor is the ARMv8a generation, a range the documentation describes as running from Apple Silicon down to 64-bit Raspberry Pi boards, with one condition stated plainly: the model's weights have to fit in the machine's memory.
Graphics acceleration is optional everywhere. It is built in for Apple Silicon, the main release file bundles libraries for NVIDIA and other cards on 64-bit Linux and Windows, and AMD's ROCm route needs a separate library download and its runtime installed. If graphics setup fails on a machine, the documentation says the file falls back to running on the main processor instead. The support page also carries the project's own caution about its current 0.10 series: it has not been tested on every graphics card and platform combination, and its AMD and Windows paths in particular are described as best-effort. The one speed figure the documentation offers is its smallest ready-made model generating on a Raspberry Pi 5 at about 8 tokens per second, printed here as the project's reference point for what modest hardware does with a small model.
Where the promise narrows
Windows and the 4 GB line
Windows will not run an executable file larger than 4 GB. The documentation is direct about the consequence: many of its own ready-made examples exceed that size, and it names one file that works on Windows because it sits about 30 MB under the limit. For the kit, the arithmetic is plain. Single-file portability on Windows holds for the small models, whose listed sizes are all under the line, and stops at the 8B class. The documented workaround keeps the stick and splits the cargo: the bare llamafile program and the model's weights travel as two separate files, and the program is started pointed at the weights. The documentation also notes that the Windows Subsystem for Linux runs the larger files whole.
macOS and the permission steps
On macOS, Linux, and the BSD systems, a downloaded llamafile needs one permission command before its first run, the standard step that marks a downloaded file as runnable. On macOS the system may also stop the file with a message that its developer cannot be verified; the troubleshooting page's documented remedy is to allow it in System Settings, under Privacy and Security. On Apple Silicon Macs, the documentation says the Xcode Command Line Tools must be installed for the file to set itself up, because on its first run it compiles a small graphics module, a one-time cost on a machine that has the tools. A borrowed Mac without them is the kit's least certain stop, and the page knows no documented way around it.
Managed machines
The troubleshooting page opens with a warning that applies on any platform: if the file's process is killed immediately, employer security software may be the cause (it names CrowdStrike), and the remedy it gives is to ask to be whitelisted. A work computer is therefore not a reliable target for the kit, whatever its specifications say. That is a technical limit and a permission limit at once, and the closing note returns to it.
Getting a file, or making one
The ready-made route is the documented quickstart: download a finished llamafile from the project's page on Hugging Face, grant it permission to run (or, on Windows, rename it to end in .exe), and run it. The project publishes one file per model, with the model's license listed beside each.
Making your own is also documented, and the recipe is short. Copy the llamafile program file, then use a small tool the project ships, called zipalign, to embed a GGUF weights file and a defaults file into the copy. Any model published in GGUF form can be packaged this way, including the ones the setup page sized.
The third shape is the two-file one from the Windows section: the bare program, run against a weights file stored beside it. It is the shape to choose when one stick should carry several models for a single program, and the documentation adds that weight files already downloaded by Ollama or LM Studio on a machine can be run this way as well, worked through there with examples on a Mac. One gap belongs in this section rather than the fine print: the documentation describes a full program variant with bundled graphics libraries and a thinner variant without them, and says which is smaller, but states no size for either that we could find. Every size in the next section is a finished file with a model inside, so the kit math does not depend on the missing figure.
Which model rides, and what the file weighs
These are the project's listed sizes for its ready-made files, checked in November 2026. The third column is this page's arithmetic against the documented Windows limit, not a project claim.
| Ready-made llamafile | Listed size | Against the Windows 4 GB line |
|---|---|---|
| Llama 3.2 1B Instruct, Q6_K build | 1.11 GB | Under |
| Qwen3.5 0.8B, Q8_0 build | 1.6 GB | Under |
| Llama 3.2 3B Instruct, Q6_K build | 2.62 GB | Under |
| Qwen3.5 2B, Q8_0 build | 3.2 GB | Under |
| Ministral 3 3B Instruct, Q4_K_M build | 3.4 GB | Under |
| Qwen3.5 4B, Q5_K_S build | 4.1 GB | Over |
| Llama 3.1 8B Instruct, Q4_K_M build | 5.23 GB | Over |
| Qwen3.5 9B, Q5_K_S build | 7.4 GB | Over |
| gpt-oss 20B | 12 GB | Over |
Two honesty notes ride on this table. The project's pre-built page adds a broader warning that most of its examples exceed the Windows limit; its own size list is what this table prints, and where the warning and a listed size seem to disagree, the figure shown at download is the one to trust. And the project's two pages disagree about the smallest file: the list says 1.6 GB for the Qwen3.5 0.8B file, and the quickstart describes the same download as 1.77 GB. The table prints the list figure; the difference changes no stick size anywhere on this page.
Set beside the setup page's figures, the same models weigh differently here, and the difference is the packaging. Ollama lists the Llama weights alone at 1.3 GB for the 1B, 2.0 GB for the 3B, and 4.7 GB for the 8B. The llamafiles above carry those weights at different compression, the 1B and 3B in Q6_K builds and the 8B in a Q4_K_M build, with the program folded in, so the two columns cannot be subtracted to reveal the program's size, and neither column is wrong. For a stick, the whole-file figure is the one that counts.
The machine's memory rule does not change with the packaging. The documentation states the condition directly for ARM machines, that the weights must fit in memory, and the setup page's ladder carries over: a machine with about 8 GB of memory free runs the 3B class comfortably, the 8B wants about 16 GB, and a 20B file, however portable it is as a file, wants a machine from the top of that ladder. A stick can carry a model the borrowed machine cannot run, which is why the kit below is built around the small files.
The stick math
The kit alone is small. The arithmetic below is the listed file sizes against common stick sizes, with the same caution the library page applied: a drive's usable space is somewhat less than the number on its label, and every total here keeps that margin.
| Kit alone | Total | Stick |
|---|---|---|
| Llama 3.2 1B llamafile | 1.11 GB | Any current stick; an 8 GB stick holds it with most of its space free |
| Llama 3.2 3B llamafile | 2.62 GB | The same; the recommended traveling class |
| Llama 3.1 8B llamafile | 5.23 GB | 16 GB, so the stick still has room to be useful; an 8 GB stick holds the file and little else |
Paired with the library, the model is counted once, inside its file, in place of the weights figure the library page used. That page's shelves were 9.2 GB of packages for the starter stack, 65.7 GB for the core stack, and 135.7 GB for the complete stack, and it counted the 8B model beside them at 4.7 GB. The same stacks with the 8B llamafile at 5.23 GB:
| Shelf plus the 8B llamafile | Arithmetic | Drive |
|---|---|---|
| Starter | 9.2 GB of packages + 5.23 GB = about 14.5 GB | 32 GB |
| Core | 65.7 GB of packages + 5.23 GB = about 71 GB | 128 GB |
| Complete | 135.7 GB of packages + 5.23 GB = about 141 GB | 256 GB |
The pairing reconciles with the library page rather than replacing it. Each stack grows by about half a gigabyte, the cost of folding the program into the model file, and no drive size changes. Swapping the 3B llamafile in as the traveling model drops each total by about 2.6 GB and changes no drive either. On a Windows-only shelf, the 8B model travels in the two-file shape instead, its weights at the 4.7 GB figure plus the bare program, and the drive sizes stand, because every stack keeps gigabytes of headroom on its drive.
Power, briefly
Portability changes where the AI runs, not what running it costs. The power page in this corner sets the planning rule: size any machine at its ceiling while it works, and count the hours, because the load is the hours. A borrowed laptop or desktop working through an evening of questions is that same arithmetic on someone else's battery or generator, and the generator wattage calculator takes the host machine's ceiling figure the same way it takes any other load. The kit adds no machine of its own to the load list; the host computer is the load.
If you are buying the stick
This is where the USB drive recommendations will go, with the affiliate links that keep this site running, once the site's affiliate accounts are approved. Nothing here is a live link yet. Buy on the arithmetic above: the kit alone fits any current stick, and the paired stacks want 32, 128, or 256 GB.
When the links go live, this box will name the specific drives, and the data basis note at the top of this page will say what that choice rests on.
What we could not verify
- Our own kit. We have not built one, run a llamafile, or carried a stick to a second machine. Every step and platform claim on this page is the project's documentation, checked in November 2026, not a report of use.
- The bare program's size. The documentation describes a full variant with bundled graphics libraries and a thinner variant without them, and says which is smaller, but states no size for either that we could find. All sizes printed here are finished files with a model inside.
- Anything about USB itself. The project documentation does not discuss USB sticks, drive formats, or removable media. The kit is this page's application of the file's documented properties, one file and no installation, to a drive. Whether a particular stick accepts a file of a given size is a property of the stick and its format, not of the project.
- The smallest file's exact size. The project's list says 1.6 GB and its quickstart says 1.77 GB for the same Qwen3.5 0.8B download. The table prints the list figure, and the figure at download governs.
- Speed on any given machine. The documentation offers one reference point, its smallest model on a Raspberry Pi 5 at about 8 tokens per second, and no per-model speed table. How fast a borrowed machine answers depends on that machine, as the setup page covers.
- Whether a specific computer will be allowed to run the file. That is set by the machine's owner and its security software, not by the file, and the documentation's own troubleshooting page names one employer security product that kills the process outright.
Questions
Does the other computer need anything installed first?
No program installation; that is the design. What it may need is permission: one command on Mac, Linux, and BSD systems, a rename on Windows, a possible allow step in macOS security settings, and, on Apple Silicon Macs, the Xcode Command Line Tools already present. A machine you control can be prepared for all of this in advance. A borrowed one may not have the tools step covered.
Will it run on an old computer?
The documented floors are old on two sides, Windows 10 and Linux kernels back to about 2007, and specific on the Mac side, Darwin 23.1.0 and later with older kernels described as should-work-but-untested. The working limit is memory rather than age. An older laptop with 8 GB of memory runs the 3B kit and cannot hold the 8B model, whatever year it was made.
Why does the 8B file refuse to open on Windows?
Windows will not run a single executable larger than 4 GB, and the 8B file is listed at 5.23 GB. The documentation's workaround is the two-file shape: the bare program and the weights file ride the same stick as separate items, and the program is started pointed at the weights. On Mac and Linux systems the same 8B file runs whole.
Is this the same AI as the setup page's?
The same class of model, in a different delivery. The setup page installs a program, Ollama or LM Studio, and downloads models into its folder; this page carries program and model fused into one file. The models overlap, the Llama names in both pages are the same models at different compression, the memory ladder is the same, and the documentation notes the bare program can even run the weight files those installed programs already downloaded.
Can one stick hold the library too?
Yes, and the stick math above is worked that way, shelf and AI side by side on one drive. The limit from the library page still stands: the AI does not read the library's package files directly, so you open the reader yourself and bring text across. The stick carries both halves; you remain the connection between them.
What does it cost?
The project is free software under the Apache 2.0 license, and the ready-made files are free downloads. The models inside carry their own licenses, listed by the project beside each file; the Llama models use Meta's community license, as the setup page notes. The only purchase the kit implies is the stick.
Two closing cautions. A borrowed computer is borrowed: run the kit on machines whose owners have said yes, and do not fight a work machine's security software to make it run. The documentation's remedy there is a whitelist request, which is the owner's call to grant. And the AI on the stick is the same fallible language model the setup page describes, right and wrong with the same confidence, in a more portable shape; portability is not accuracy. Data basis for this page: the llamafile project's GitHub repository and documentation site, checked November 2026; file sizes are the project's listed figures; stack totals reuse the library page's package figures. We have not built or run the kit described.