How to Build Your Own AI for Business Without OpenAI or Any AI Provider: Open Models, Private Data and Real Costs

Many business owners ask the same question: can we have our own AI without sending our data to OpenAI or any other AI provider? Yes. This guide explains how, in plain English, including the licence traps, the hardware you need, and a safe order to build in.

Last updated: 15 October 2026. Model lineups, licences and prices change quickly. This guide is based on recent technical write-ups and vendor guides, not independent benchmarks, so please check each linked source and test on your own task before you commit. Nothing here is legal advice.

The short version

• Do not train a model from scratch. That needs huge amounts of data and computing power. Instead, run an open-weight model: a model whose files are published so you can run it on your own server.
• Your own data stays with you. If you run the model and the search on your own machines, no text has to go to an outside AI service.
• Start with your documents, not with training. Letting the model search your own files (called RAG) comes first. Fine-tuning comes later, and only if tone or format is still wrong.
• It is not free. You take on the hardware, uptime, updates and security. It makes sense when privacy or contracts forbid outside services, or when your volume is high and steady.

What "your own AI" really means

A model's files are its "brain". Open-weight means you can download and run that brain yourself, so you are not renting access to somebody else's service.

Step 1: Choose a model, and read the licence

According to recent licensing roundups such as Codersera's 2026 overview and Torck's comparison of the main families, the licences vary a lot:

• Apache 2.0 or MIT, the simplest for commercial use. These roundups list Mistral Small 4 and Mistral Large 3, most of the Qwen 3.5 series and DeepSeek V4 (MIT).
• gpt-oss. The roundups list OpenAI's open-weight gpt-oss models under Apache 2.0. You run them yourself and no OpenAI service is involved, but if "no OpenAI" is a strict rule for your company, simply choose another family.
• Needs a careful read. Llama 4 uses Meta's custom community licence, with attribution and usage thresholds. The largest Qwen models have their own terms, and sources disagree on Gemma's licence. Read the licence file on each model card before you build.

Do not pick a model by benchmark score alone. Pick two or three that fit your hardware and licence, then compare them on 20 to 50 real questions from your business.

Step 2: Plan the hardware

Per sizing guides such as this one from Flavio Copes, a model compressed to 4 bits (called quantisation) needs roughly this much GPU memory just for its weights:

These are planning estimates, and sources disagree on exact figures. Real needs are higher, because many users and long documents use extra memory. One guide says 50 users at 8K context can push a 70B model past 80 GB.

On cost, one vendor page gives GPU cloud prices from about 1,500 US dollars per month for a single A100 up to 25,000 or more for an eight-H100 cluster, and estimates break-even against paid APIs at 10 to 50 million tokens a day. That is a vendor estimate. Run your own numbers before you buy anything.

Quantisation can cost a little quality. One source says the loss is often unnoticeable for many tasks, but test it on yours.

Step 3: Serve the model

Comparisons such as Red Hat's vLLM versus Ollama guide and SitePoint's scaling guide describe three common choices:

• Ollama is the easiest to set up and is fine for a developer or a small internal tool. It copes less well with many simultaneous users.
• vLLM is built for production traffic with many concurrent users on server GPUs, and it supports spreading a large model across several GPUs.
• llama.cpp sits in between, with more tuning work.

Both Ollama and vLLM can expose an OpenAI-compatible API. That is only a message format, so your app talks to your own server and nothing goes to a third party. It also means you can swap the model later without rewriting the app.

A common pattern from these guides is Ollama on developer laptops and vLLM behind customer-facing features. The speed claims in these articles come from vendor tests on different workloads, so benchmark your own model, GPU and number of users.

Step 4: Give it your business knowledge with private RAG

RAG means the system searches your documents first and hands the best passages to the model, which then writes an answer and can point to its sources. It is cheaper, faster to build and easier to keep up to date than training.

A May 2026 tutorial builds a fully local version with Ollama and PostgreSQL's pgvector extension in about 250 lines of Python, with citations to the source files and no outside API keys. That fits well with a PostgreSQL-based product. R2R is a heavier, packaged option with an admin interface.

Two warnings from those sources:

• Some "self-hosted" tutorials quietly send text to OpenAI to create embeddings. Check that yours does not.
• Plan document-level access control from the start. Most tutorials skip it, and staff should not see answers drawn from documents they cannot open.

Step 5: Fine-tune only if you still need to

Fine-tuning guidance draws a clear line: RAG adds current knowledge, while fine-tuning changes behaviour such as tone, terminology and format. Fine-tuning is a poor way to store facts, because facts change and you cannot easily remove one from a trained model.

If you do need it, LoRA trains a small add-on layer instead of the whole model, so a run is much cheaper. For style or format work, the guide suggests 500 to 2,000 high-quality examples is often enough. Most of the real effort goes into cleaning data and building a test set, not into the GPU bill.

What it means for your IT company

1. Ask why before you ask how. Is the reason privacy, a client contract, cost at high volume, or simply curiosity? The answer decides whether self-hosting is worth it.
2. Be honest about quality. Open models can trail the best paid ones on hard reasoning. Test on the client's real tasks and show the results.
3. Price the operations. Hardware, uptime, updates, backups and security are part of the project, not extras.
4. Offer it as a package. Model choice, a private document assistant, access control and monitoring is a clear, sellable service.
5. Keep an exit. Because the serving layer can use a standard API format, build so the model can be swapped when a better one arrives.
6. Read every licence. Check thresholds, attribution rules and any limits on offering the model as a service. Ask a legal adviser for client products.

A safe build order

1. Pick one narrow task, for example answering staff questions from the policy documents.
2. Write 20 to 50 real test questions with the answers you expect.
3. Try two or three models with a good prompt and score them.
4. Add RAG over your documents on your own server, using Ollama and pgvector as a starting point.
5. Move to vLLM only when the number of users demands it.
6. Fine-tune with LoRA only if the tone or format is still wrong.
7. Add logging, access control and human review before anything reaches customers.

Is it right for you? A quick checklist

1. Do contracts or regulations stop you sending data to outside AI services?
2. Is your usage high and steady enough to justify a GPU server?
3. Do you have someone who can run and update a server?
4. Have you tested an open model on your real questions?
5. Have you read the licence of the model you chose?
6. Do you have a plan for access control, logging and human review?

If you answered no to most of these, a managed AI service with a good data agreement may still be the better fit. There is no shame in that, and you can move later.

How SimpleApp can help

We build AI-powered applications and can set up a private, self-hosted assistant on your documents, with testing, access control and clear logs. See our AI-powered applications and custom software development services, or ask for a free quote.