Power & Pantry

Disclosure: Power and Pantry is reader-supported. If you buy through links on this page, we may earn a commission from Amazon or another program at no extra cost to you. See the About page for how this site is funded.

Chat With Your Own Documents, Offline (2026)

The short answer

Yes, and this is the arrangement the earlier pages in this corner were missing. AnythingLLM is a free, open source program that takes documents you hand it, your equipment manuals, insurance papers, property records, and turns them into a private collection that a local AI can answer questions from, with no connection and, in the configuration this page describes, with nothing sent off the machine. The mechanism is documented rather than promised. The program converts each document to text, cuts the text into chunks, and stores searchable representations of those chunks in a database on your own computer. A question retrieves the chunks that match it and hands them to the same kind of local model that the setup page in this corner installs. The limits are just as documented. The AI answers from the chunks it is handed, retrieval can miss the passage that matters or split it in two, and the maker's own documentation says so in plain words. The original document remains the authority, exactly as the library page ruled for Wikipedia. And everything about it, the download, the setup, the loading of the documents themselves, happens in advance, on a working connection, on an ordinary day.

The gap this page closes

The library page in this corner stated a limit plainly, and it is worth restating before this page removes half of it. No documentation from Kiwix, Ollama, or LM Studio describes an AI reading Kiwix's package files directly, so with a downloaded library, you are the connection between the shelf and the chat: you find the article, read it, and paste the passage you want explained. That limit is real, and it still stands for the library. Document chat is the other arrangement, and it exists because of who owns the files. A Kiwix package is a sealed snapshot built by someone else, in a format made for its reader. A PDF of your own generator manual is a file on your disk that a program can open, convert, and index, and AnythingLLM's documentation describes exactly that pipeline for files you upload. The contrast is the spine of this corner. Reference knowledge you download stays in its reader, where its date and its references are printed on it. Your own paperwork goes into the document tool, which was built to be handed files. Each half is honest about which one it is, and neither pretends to do the other's job.

The readiness value follows from whose facts these are. A downloaded library knows the world's reference shelf. It does not know your generator's oil specification, your policy's deductible, or what your lease says about the water heater, because those facts exist in exactly one place: your paperwork. In an outage, the questions that carry consequences are usually the second kind. Document chat is how a local AI reaches them without a connection, and it is why this corner treats the household's own files as a collection worth building on an ordinary day, like the library beside it.

The tool, and its two forms

AnythingLLM is built by Mintplex Labs and published as open source under the MIT license, the same terms as Ollama. It comes in two forms, and the maker's installation guide separates them by audience. The Desktop form is a single-player application for Mac, Windows, and Linux, meant for one person on their own machine; the guide's own summary of who it is for includes the line that everything needs to stay only on your device. The Docker form is a server: several people at once, access through a browser, administrator controls and permissions, and the ability to publish chat widgets to the public internet. A household following this corner wants the Desktop form, and this page's setup describes it. The Docker form is named here so a reader who meets it in the documentation knows it is the multi-user variant, not a different product. One documented detail about the Desktop form belongs in the record early: by default its internal service listens only at the local address, 127.0.0.1, so nothing outside the machine can reach it. A setting exists to open it to other devices on a home network, and it is off unless someone turns it on.

FormBuilt forRuns onWho it fits
DesktopOne person, one machine, everything on the deviceMac, Windows, LinuxThe household setup on this page
DockerA shared server, several users, browser access, permissionsA server or an always-on computerTeams and organizations; named here for completeness

The maker's system requirements make the same point the setup page in this corner made about local AI generally: the application itself is lightweight, and the local model is the part that decides what your machine needs. Storage grows with the models you keep and the documents you load, because both live on your computer. Model sizing follows the RAM ladder on the setup page, and nothing about document chat changes it. A machine that runs the 8B class comfortably runs this page's arrangement; a machine limited to the 1B to 3B class runs it with a smaller brain answering from the same documents.

How a document becomes an answer

The maker's documentation describes the pipeline in steps, and each step is simple enough to state exactly. Uploading a document converts it to text, and that is all uploading does. Embedding it, which is what makes it answerable, cuts the text into chunks, sends the chunks to a second, smaller model called an embedder, and stores the embedder's output, a list of numbers representing each chunk, in a vector database alongside the text it came from. Asking a question runs the same embedder over the question, finds the chunks whose numbers sit nearest to it, and passes those chunks to the chat model as context together with your question. The chat window shows source citations for the chunks an answer used, which matters more than it might sound: the check on any answer is one glance at the passage it claims to rest on. Documents are organized into workspaces, and a workspace can only see the documents embedded in it, so a household can keep equipment, house papers, and medical paperwork in separate collections that never mix.

Two documented details keep the mechanism honest. First, your documents are not baked into the model. The maker's explainer is explicit that the model has no idea what is in each document; the text lives in the database and is fetched per question, which is also why deleting a document genuinely removes it, a point the privacy section below returns to. Second, retrieval is selective by design. The workspace settings discuss keeping the number of snippets handed to the model at four to six for most models, because the model's context window is shared among the question, the history, and the retrieved text. The model answers from a handful of passages, not from your whole shelf, and the accuracy section below is about what that selectivity costs and how the program itself compensates. Short documents have a second route: a file can be attached whole, its full text placed in the window, or pinned into a workspace for the same treatment. Long manuals belong in the embedded route, where the chunking is the point.

What can be loaded, at the level the maker's own pages name it: the project's summary names PDF, TXT, and DOCX files. Its release notes add spreadsheet uploads, and its transcription documentation covers audio and video files, which are turned into text by a built-in transcription model that downloads the first time it is used. The program can also pull documents in from online sources while you have a connection, through connectors the release notes name for code repositories, shared workspaces, and web pages, including transcripts for video links. Those imports are ordinary-day gathering, like every download in this corner. The offline arrangement itself needs none of them: it runs on files already on the machine.

The three pieces, and where each one runs

A document chat setup has three working pieces, and the privacy of the whole arrangement is decided piece by piece, in settings. This is the table the program's marketing pages do not print, assembled from its documentation's own lists of what is built in, what is local, and what is a cloud service.

PieceIts jobThe local route, as documentedWhat changes the answer
Chat modelWrites the answersThe desktop app ships with a built-in local engine, and it can also use Ollama or LM Studio running on the same machineSelecting a cloud provider, such as OpenAI, Anthropic, or Gemini, sends your question and the retrieved chunks to that company
EmbedderTurns text into searchable numbersA built-in embedder is the default; the documentation names the model it uses, all-MiniLM-L6-v2Selecting a cloud embedder sends document text to that provider to be converted
Vector databaseStores the numbers and the text they came fromLanceDB, built in; the documentation states that your vectors never leave the program when the default is usedSelecting a cloud database, such as Pinecone or Astra DB, stores your document vectors on that company's servers

Read the table as the definition of the local configuration: a local chat model, the built-in embedder, and the built-in database. Every default in the maker's documentation is the local one, which is a genuinely well-chosen set of defaults. Every cloud alternative is one settings change away, which is why this page describes a configuration to hold, not a property the program has in all its forms. The built-in chat engine deserves one dated note, because it shows how fast this layer moves: the maker announced in October 2026 that the desktop app's built-in engine now runs on llama.cpp, managed by AMD's Lemonade software, where earlier versions were built around Ollama. A reader who set the program up a year ago and a reader who sets it up today have different engines under the same screens, and both are documented routes to the same arrangement.

What "nothing leaves the machine" requires

The plan for this page used the phrase the incumbents use, nothing leaves the machine, and the documentation supports it at a specific strength, under named conditions. Here they are, all of them, because the value of the claim is exactly its conditions. Hold the three pieces at their local settings from the table above, and the documented picture is complete: your documents, their text, their stored vectors, and your questions are processed and kept on your computer, and the desktop program serves only the machine itself unless its network setting is deliberately opened. That is the claim, and it is a good one. These are the documented exceptions, each with its switch.

Stated as one sentence, the way this site states every claim it prints: with a local chat model, the built-in embedder, the built-in database, telemetry off, and web search off, the documentation supports saying that document chat happens entirely on your machine. Drop any one of those conditions and the sentence needs its exception attached. The program's defaults get a household most of the way there; the two switches are the rest of it.

The household shelf: what to load

The documents that earn their place are the ones an outage makes urgent and a connection normally makes easy to look up. The manuals come first: the generator, the power station, the water heater, the well pump, the appliances whose error codes mean nothing without the book. This site's power pages keep telling readers to keep those manuals; this is where they become answerable. Then the house papers: the insurance policy and its declarations page, the deed or lease, property records, the documents a claim or a dispute will ask for while the offices that hold copies are closed. Then the medical shelf: medication lists, care instructions, the paperwork that explains a condition or a device, kept as reference for understanding, in the same spirit and under the same caution as the medicine selection on the library page. A workspace per shelf keeps them separate, and this site's own guides, saved as files, can join the equipment shelf for readers who want the calculator's reasoning in the same place as the manual it reasons about.

Two habits from the documentation belong with the shelf. Loading a document copies it into the program's own storage on the machine; your original file stays where it was, and the program works on its copy. Keep the originals, backed up in two places, the way the library page advises for the library itself. And deletion means what it says, in two grades the privacy documentation separates: removing a document from a workspace stops that workspace using it and deletes its vectors there, while the document remains in the program's library for other workspaces; deleting it from the library removes the parsed file and the cached embeddings from the system entirely. A household that loads a sensitive document can therefore retire it completely, and knows the difference between shelving it and destroying the copy.

Setup path, as documented

  1. While you have a connection, download AnythingLLM Desktop for your system from the maker's site and install it. The documented installer fetches its engine and graphics support from the maker's servers as it goes, so installation is connection work, like the model downloads on the setup page.
  2. Choose the chat model. The built-in engine can fetch and run a local model by itself, or connect the Ollama or LM Studio setup from the first page in this corner and use the model already downloaded there. Size it by that page's RAM ladder: the documents do not change what your machine can run.
  3. Leave the embedder and the vector database at their defaults. The defaults are the built-in embedder and LanceDB, and they are the local route from the table above.
  4. Create a workspace for one shelf of paperwork, equipment first, and upload its documents. Embedding them into the workspace converts, chunks, and indexes them on the machine, in this configuration.
  5. Set the workspace to Query mode for paperwork. The maker's chat mode documentation describes Query mode as answering only from the uploaded documents and telling you when it cannot find relevant information, while Chat mode also draws on the model's general knowledge. For documents that decide money and safety, the mode that refuses is the right default.
  6. For the strict configuration, open the Privacy settings and switch telemetry off, and turn web search off as well. Both switches are documented, and both are described in the privacy section above.
  7. Finish with this corner's verification ritual, Wi-Fi off. Ask a question whose answer lives in one loaded manual and nowhere in the model's training: a model-specific clearance, a reset sequence, a torque figure. Then check the answer against the document itself, which the answer's citations point back to. If it answers correctly with the connection down, the arrangement is what this page says it is.

Accuracy: the document governs

Document chat changes where the AI's material comes from, and that is a real improvement with a specific boundary. The model is no longer limited to its training; it is handed passages from your own files. It can still misread a passage it was handed, and retrieval decides which passages it sees. The maker's own explainer of the technique warns that chunking offers no guarantee that relevant text stays together, and its chat mode documentation lists the everyday failure in its own words: the information might be in your document but worded differently from your question, and a differently worded question is the documented fix. A procedure split across two chunks can arrive as half a procedure. A figure can be retrieved from the wrong model's table in the same manual. None of this is hidden by the maker, which is to its credit, and none of it is a reason to skip the tool. It is the reason for the checking habits the tool itself supports: Query mode, which refuses rather than inventing when nothing relevant is found, and citations, which put the source passage one glance away from every answer.

The rule is the same one the library page set for Wikipedia, and it does not soften because the document is yours. When the chat and the document disagree, the document governs. When the answer decides anything that matters, a deductible, a dosage, a wiring step, read the passage in the original before acting, because the original is what an insurer, a clinician, or an inspector will honor, and a retrieved chunk is a copy of a piece of it. Used that way, as a finding aid over paperwork you already own, document chat is the strongest accuracy arrangement in this corner: the facts are yours, they are dated and signed by whoever issued them, and the AI's job is narrowed to locating and explaining them.

Power: the same load as the chat it extends

Document chat is local AI work, so its power cost is the one the power page in this corner already priced: budget the machine at its charger wattage for the hours you will actually ask questions, with a station delivering about 85 percent of its rated capacity. The one new load this page adds is not an outage load at all. Embedding a shelf of documents is a single ordinary-day job, done on household power while the documents are loaded, and the built-in embedder is a small model doing clerical work, not the chat model at full stretch. During the outage itself, asking questions of your paperwork costs what asking questions costs, and an evening of it sits comfortably inside the budget the library page described for reading. Enter the machine once, at its ceiling, in this site's generator wattage calculator, and the chat, the library, and the document shelf are all inside the same plan.

Matched offers are coming. This site is new, and the affiliate links for this corner are still being set up. When they are live, this box will hold current offers for the hardware this corner's pages name, such as external drives for storing models, libraries, and document collections, with prices re-checked at that time. The disclosure at the top of this page explains how the site earns.

What we could not verify

Questions

Can it answer questions about my documents with the internet down?

Yes, in the local configuration this page describes. The documents, their index, and the model are all on the machine before the connection goes, and asking questions needs nothing outside it. The parts of the program that do need a connection, web search and any cloud piece a user has selected, are separate features, and the setup above does not use them.

Is it really private?

Under the conditions the privacy section lists, the documentation supports it: local chat model, built-in embedder, built-in database, telemetry switched off, web search switched off. The defaults alone are close but not the whole sentence, because anonymous telemetry events and web search are both on until someone turns them off. The honest answer is the conditional one, and the conditions are short enough to hold.

How is this different from the offline library?

The library is published reference, frozen at its build date, that an AI cannot read directly; you carry passages between the two programs yourself. Document chat is your own files, which the program itself converts and indexes so the AI answers from them. The library knows the world's shelf and nothing about your house. The document shelf knows your house and nothing about the world. A complete setup has both, and each page in this corner is honest about which half it is describing.

Does loading my documents train the AI or change the model?

No. The maker's explainer is explicit that documents are not embedded into the model. They stay as text and stored vectors in the database, fetched passage by passage when a question matches them. The model is the same model afterward, which is why the same setup from the first page in this corner can serve a document shelf unchanged, and why deleting a document from the library genuinely removes it.

What happens when the answer is not in my documents?

In Query mode, the documented behavior is that the program tells you it cannot find relevant information, which for paperwork is the answer you want: a refusal you can act on, instead of a fluent guess wearing your manual's authority. Chat mode will answer anyway, drawing on the model's general knowledge, which is why the setup path sets paperwork workspaces to Query mode and leaves Chat mode for brainstorming.

How much does it cost?

Nothing in money. The program is a free download, open source under the MIT license, and the local setup needs no account and no API key; the maker's site says as much in its own summary. The models it runs are free under their makers' licenses, as the setup page in this corner describes. The real costs are the same ones this corner always charges: disk space for models and documents, electricity, and an ordinary day spent setting it up before it is needed.

A note on responsibility

A chat over your paperwork is a finding aid, not a reading of record. It retrieves pieces of documents, sometimes split, sometimes the wrong piece, and a model then explains them in fluent language that can still be wrong. For anything that decides money, coverage, health, or safety, the original document is the authority: read the passage itself before acting on an answer, and let the chat do what it is good at, which is finding the passage fast at 2 a.m. with the connection down. This page's data basis is stated at the top and stands: the maker's official documentation and public repository, checked in November 2026, with the gaps named where a reader would otherwise assume a test, a list, or a promise exists.