Local AI, Enterprise Cloud, or Open Source: What Actually Keeps Client Data Safe?

Every attorney considering AI tools eventually hits the same wall: where does my client's data go after I press enter?

It's not a hypothetical concern. Under RPC 1.6 and New Jersey's evolving AI guidance, you have a duty to safeguard confidential client information — and that duty extends to every technology tool you use. When client facts, case strategy, or privileged communications flow through an AI system, the deployment architecture of that system becomes an ethics question, not just a tech preference.

There are three broad approaches available today, and each carries genuine advantages alongside real limitations. None of them is universally "right." The best choice depends on your practice, your clients, and how much infrastructure you're willing to manage.

Option One: Local (On-Premise) Language Models

Running a language model on your own hardware — a powerful workstation, a local server, or an air-gapped machine — gives you the strongest possible data control. Nothing leaves your office. No API calls. No cloud storage. No third-party data processing agreements to parse.

For attorneys handling highly sensitive matters — criminal defense, whistleblower cases, contested IOLTA disputes — this can be compelling. Your data never touches another company's servers. Period.

The tradeoffs are significant. Local models are smaller and less capable than the frontier models from OpenAI, Anthropic, or Google. A model you can run on a high-end workstation (like Llama 3 at 70 billion parameters) is impressive for its size but still trails behind GPT-4o or Claude Opus on complex legal reasoning, nuanced drafting, and multi-step analysis. You'll also need meaningful hardware — a machine with a capable GPU and substantial RAM — and someone with enough technical comfort to set it up and maintain it.

Local models also don't update themselves. When a cloud provider improves its model, you get the improvement automatically. With a local model, you're responsible for downloading, testing, and deploying new versions.

Best fit: Attorneys handling extraordinarily sensitive matters who need absolute data isolation and are willing to accept reduced model capability in exchange. Also useful for specific, narrow tasks (document summarization, intake organization) where frontier-level reasoning isn't critical.

Option Two: Paid Enterprise Cloud APIs

The major AI providers — OpenAI (via ChatGPT Enterprise or the API), Anthropic (Claude for Business or the API), Google (Gemini for Workspace), and Microsoft (Azure OpenAI Service) — all offer enterprise tiers with explicit contractual protections.

The key provisions to look for: your data is not used to train their models, data is encrypted in transit and at rest, you can specify data residency regions, and retention periods are either zero or configurable. Most enterprise agreements include Data Processing Addendums (DPAs), and some offer Business Associate Agreements (BAAs) for HIPAA-adjacent work.

This is the most popular path for law firms adopting AI right now, and for good reason. You get access to the most capable models available — the ones that are genuinely useful for complex legal research, drafting, and analysis — with contractual assurances about data handling.

The tradeoffs: You are trusting the provider to honor those contractual commitments. Your data does leave your network. If the provider experiences a breach or changes their terms, your data was in their infrastructure. You're also paying per-token or per-seat pricing that can add up with heavy use.

For most solo practitioners and small firms, the contractual protections offered by enterprise tiers are reasonable and proportionate. But "reasonable" is a judgment call, and it depends on what kind of matters you handle.

Best fit: Firms that need frontier model capability, handle a range of practice areas, and are comfortable with contractual data protections. This covers the majority of solo and small firm use cases.

Option Three: Enterprise Open Source (Self-Hosted or Managed)

Open-source models — Meta's Llama, Mistral, and others — sit in an interesting middle ground. You can download and run them locally (see Option One), but you can also deploy them on cloud infrastructure you control: your own AWS, Azure, or Google Cloud account.

This approach gives you more capable models than a local workstation can run (you can provision serious GPU resources in the cloud), while keeping the model weights and inference pipeline under your control. Your data flows through your cloud account, not the AI company's API. You set the retention policies. You control the access.

Managed open-source platforms — services that host open-source models for you with privacy guarantees — are another variant. These give you the convenience of a cloud API without sending data to OpenAI or Anthropic, though you're still trusting the hosting provider.

The tradeoffs: Cloud GPU resources aren't cheap. Running a 70B-parameter model on cloud infrastructure costs real money, especially under sustained use. You also need cloud engineering knowledge (or a consultant) to set up and maintain the deployment. And open-source models, while improving rapidly, still generally lag behind frontier proprietary models on the hardest reasoning tasks.

Best fit: Tech-forward firms or those with access to IT support who want strong model performance with data sovereignty, and who are willing to invest in cloud infrastructure management.

The Comparison That Matters

Factor	Local On-Premise	Enterprise Cloud API	Self-Hosted Open Source
Data leaves your network	No	Yes (contractually protected)	Depends on setup
Model capability	Limited	Frontier-level	Strong but not frontier
Setup complexity	Moderate-High	Low	High
Ongoing cost	Hardware only	Per-token / per-seat	Cloud GPU + maintenance
Maintenance burden	You	Provider	You (or your team)
Contractual protections	N/A	DPA, enterprise terms	Depends on host
Best for sensitive matters	Excellent	Good (with enterprise tier)	Good

So Which One Should You Choose?

That depends on questions only you can answer.

How sensitive are the matters you handle? If you routinely deal with information where any third-party exposure is unacceptable, local models or self-hosted open source deserve serious consideration.

How much capability do you need? If you're using AI for complex legal reasoning, multi-jurisdiction research, or nuanced brief drafting, the frontier enterprise models still have a meaningful edge.

What's your budget and technical comfort? Local and self-hosted options trade money and technical effort for data control. Enterprise APIs trade data control (within contractual bounds) for simplicity and power.

And critically: what does your AI use policy say? If you haven't documented your approach to data handling, tool selection, and verification — regardless of which option you choose — that's the gap to close first.

The Honest Answer

There is no single deployment model that perfectly solves every attorney's privacy and data retention concerns. Local models give you maximum control with capability tradeoffs. Enterprise cloud APIs give you maximum capability with trust tradeoffs. Open source splits the difference with complexity tradeoffs.

The right answer is the one that aligns with your ethical obligations, your practice needs, and your honest assessment of what you're able to maintain.

If you're weighing these options and want to think through which architecture fits your firm, reach out. This is exactly the kind of decision where a structured evaluation prevents expensive mistakes.