Local LLMs for Legal Teams: What Stays on Your Device

Local LLMs Explained

Local AI uses your computer’s processor and memory to generate an answer. The app loads a downloaded model, accepts your prompt and produces text without calling a hosted model.

The model file contains its learned numerical settings, called weights. Downloadable local models are commonly called open-weight models. The distinction between those models and open-source AI concerns what their licences let you do and what their makers release.

Offline AI is local AI that works with the internet disconnected. Download the app, its required runtime and the model first. Model searches and downloads need a connection, while a downloaded model can answer offline. LM Studio’s offline-operation documentation also describes local document processing.

For example, a lawyer can ask a downloaded model to shorten a fictional client letter while the laptop is offline. The computer generates the rewrite itself. Asking the same app to use a cloud model changes where that request is processed.

What Stays on Your Device

Prompts, attached files and answers stay on your computer when the app and every processing step run locally. A local chat history also remains a stored copy that someone with access to the device could read.

Check the actual workflow before introducing client material:

1. Select a downloaded local model. A cloud model listed in the same app is a different processing route.
2. Disable cloud features and review connected tools, web search and remote servers. A local model can still pass information to an external tool.
3. Review usage reporting and update settings. Distinguish requests for software updates or model downloads from requests containing document text.
4. Disconnect Wi-Fi and Ethernet, then start a fresh chat with fictional text. Confirm that the model produces a new answer and can process the intended document type.
5. Reconnect and inspect network activity with your IT team. Record which connections belong to downloads, updates or optional services before approving the setup.

The offline check proves that the tested task can work without a connection. It does not prove that the app never connects when the network returns. Keep the approved settings and network controls in place during real work.

Local processing also leaves device security, backups and staff access in your firm’s hands. Disk encryption protects a powered-off device differently from an unlocked account. A cloud-synced folder can upload source documents or exported answers even when the model stays offline. A backup can retain copies after you remove a chat.

If your team needs a hosted option instead, private AI requires a separate assessment of the provider’s access and processing terms.

Other Software That Can Still See Your Data

Other apps can receive client material through the permissions and features enabled on the same computer. A local model does not isolate the rest of the desktop.

Grammarly processes writing to provide suggestions. Its privacy and security FAQs describe server processing and storage. Pasting a local answer into an app where Grammarly is active introduces another data path. Control where the assistant operates, including desktop apps and the browser.

AI browser extensions also need individual review. An extension with access to page content can read information displayed in a browser-based local chat. Restrict its site permissions or use a dedicated browser profile without those extensions.

Windows Recall can save screen snapshots on supported Copilot+ PCs when a user opts in. Microsoft says snapshots stay on the device and saving is off by default. A visible client document can still become an additional local copy. Recall’s controls let you disable saving or exclude apps and websites.

Apple Intelligence uses on-device processing and Private Cloud Compute, depending on the request. Apple says cloud requests are used to fulfil the request and are not stored. Its privacy explanation describes that separate processing route. Invoking an Apple Intelligence feature on client text needs its own approval even when your chatbot runs locally.

Use this checklist for a computer handling confidential local AI work:

• Keep the operating system patched, enable disk encryption and lock unattended sessions.
• Restrict the working folder and chat history to authorised staff accounts.
• Review cloud sync and backup destinations for source files, chats and exported answers.
• Disable unapproved writing assistants and restrict browser-extension permissions.
• Check screen-capture features and AI services separately from the local model.
• Keep remote access to the model server disabled unless your IT team has secured it.

Apps That Run Models Locally

LM Studio suits staff who want a desktop interface for downloading models and chatting with documents. Ollama suits teams comfortable with model commands or connecting a model to another application. Both run downloadable models, and both now also have cloud features that require a separate choice.

Ollama and LM Studio Compared

Area	Ollama	LM Studio
Everyday use	App and command-line workflows, with a local API for other applications	Desktop model search, chat and document interface
Local privacy	Local model requests run on your machine	Downloaded models and local document processing stay on your machine
Cloud controls	Local-only mode disables cloud models and web search	Select local models and avoid cloud processing or web search
Model choice	Downloadable families including Qwen, Gemma and Llama	Compatible model files through its supported runtimes

Ollama’s FAQ documents local-only mode and the setting OLLAMA_NO_CLOUD=1. Restart Ollama after changing that setting. Its default server listens on the same computer. The dedicated Ollama guide covers its data handling in more detail.

Is LM Studio safe?

Yes, LM Studio can keep prompts and documents local when you use downloaded models without cloud services or external tools. Its privacy policy distinguishes local chats from cloud processing. It also describes network requests for model discovery, downloads and software updates. That makes it a usable local option, with the device and app controls above still required.

Model Families and Suitable Tasks

Choose a model variant for the task and available memory. A family name alone does not tell you whether a particular version runs locally.

Model family	Useful starting task	Limit to allow for
Llama 3.2 text models	Short rewrites and summaries on a modest computer	Smaller models have limited capacity for difficult reasoning
Qwen3 downloadable models	Multilingual text and reasoning tasks	Larger variants need much more memory
Gemma 3	Text tasks, with image input on supported variants	The smallest text variants do not accept images

Ollama’s model catalogue documents these families and their different sizes. These are task-based starting points, not evidence of legal accuracy. A model that writes fluently can still omit a qualification or invent an authority. Match its output to the source before using it in a matter.

Hardware and Limits

Many existing firm computers can run a small local model, while larger models need more memory and often a capable graphics processor. LM Studio’s system requirements recommend at least 16 GB of RAM. It supports Apple Silicon Macs and recommends at least 4 GB of dedicated graphics memory on Windows.

Use these ranges as planning estimates, not model guarantees:

Available memory	Practical starting point	Main constraint
8 GB system RAM	Small models around 1 to 3 billion parameters, with short inputs	Little room remains for other apps or long documents
16 to 32 GB RAM or unified memory	Small to medium quantised models, often around 4 to 14 billion parameters	Model format and input length determine whether it fits
32 to 64 GB or more	Larger models and longer inputs, subject to graphics-memory limits	Cost and processing speed still limit useful capacity

Parameters are the model’s learned values. Quantisation stores those values using fewer bits to reduce memory needs. The downloaded file size is only part of the requirement. The app also needs memory for the input and generated text, plus space for the operating system.

A modern processor can run some models without a dedicated graphics card, but generation can be slow. Graphics acceleration speeds compatible workloads. On a Windows workstation, system RAM and dedicated graphics memory are separate limits. Apple Silicon uses unified memory shared by the processor and graphics hardware.

A smaller local model trades capacity for a workable memory footprint. Hosted services can use hardware beyond a laptop’s resources, while local speed depends on your own machine. Longer inputs consume more memory and take longer to process. Local deployment does not guarantee that an entire matter fits into one prompt.

Start with an existing supported computer and a small model before buying hardware. Use fictional documents of the expected length, keep other work apps open and measure whether the answer arrives quickly enough. If memory runs out or the computer becomes unresponsive, reduce the input length or choose a smaller model before expanding the rollout.