How to Install a Local AI Model on a Remote Server

Running language models locally on a laptop is convenient for experiments, but quickly runs into hardware limitations. A remote VPS server eliminates this problem: you get dedicated resources, constant availability, and the ability to access the model via API from any application. In this article, we will walk through the entire process — from ordering a server to running inference with Ollama, one of the most popular tools for self-hosted LLMs.

Step 1. Creating a Server in NodexGo

Go to the NodexGo control panel and click the Create Server button. First, select a location — the real ping from your browser is displayed next to each data center, which helps you choose the nearest point for minimal latency when accessing the model API. Then choose a plan: for running small models (7B parameters), 8–16 GB of RAM is sufficient; larger models will require more RAM and CPU cores.

As the operating system image, select Ubuntu 24.04 — it has the best tested compatibility with Ollama and most ML tools. Set a server name, choose a billing period (1, 3, 6, or 12 months), and confirm the order. In about a minute, the server will be ready: the IP address and root password will appear in the panel and will be sent to your email.

Step 2. Connecting via SSH and Updating the System

Once the server is ready, connect to it via SSH using the IP address from the panel and the root password sent to your email. If for some reason the SSH connection cannot be established, use the VNC web console directly in the server card — it is available without any additional configuration and is a lifesaver in unexpected situations.

ssh root@your-server-ip

After logging in, first update the system packages to get the latest versions of dependencies and close potential vulnerabilities.

apt update && apt upgrade -y

Step 3. Installing Ollama

Ollama is a tool that packages language models together with all their dependencies and provides a simple REST API for inference. It supports dozens of open models: Llama 3, Mistral, Gemma, Phi, and others. Installation is done with a single command — the official script will automatically detect the system architecture and install the required components.

curl -fsSL https://ollama.com/install.sh | sh

After installation, Ollama is automatically registered as a systemd service and runs in the background. You can check the status with the standard systemctl command. By default, the API is available on port 11434 only on the loopback interface — this is a sensible behavior from a security standpoint, and we will cover how to open external access a bit later.

systemctl status ollama

Step 4. Downloading and Running a Model

The ollama pull command downloads a model from the official registry and saves it to the server's disk. For a first run, we recommend Mistral 7B — a good balance of quality and resource consumption. The model size is about 4 GB, so make sure there is enough disk space beforehand.

ollama pull mistral

After downloading, you can immediately chat with the model in interactive mode right in the terminal to make sure everything is working correctly.

ollama run mistral

Step 5. Opening the API for External Requests

By default, Ollama only listens on localhost. To access the API from your computer or a third-party application, you need to change the OLLAMA_HOST environment variable. Open the systemd unit configuration file for editing.

systemctl edit ollama

In the editor that opens, add a Service section with an environment variable that instructs Ollama to listen on all interfaces.

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Save the file and restart the service. After that, the API will be accessible via the server's external IP address on port 11434. Be sure to restrict access to this port via a firewall — allow connections only from trusted IP addresses, otherwise anyone will be able to use your model and load the server.

systemctl daemon-reload
systemctl restart ollama
ufw allow from YOUR_IP to any port 11434
ufw enable

Step 6. Testing the API and Making the First Request

Ollama provides a REST API compatible with the OpenAI format. This means that most libraries and tools written for OpenAI will work with your local model without any changes — just change the base URL. Let's verify the API is working by sending a simple text generation request via curl.

curl http://your-server-ip:11434/api/generate \
  -d '{"model": "mistral", "prompt": "Hello! Who are you?", "stream": false}'

If the response returns a JSON object with a response field, the model is working and ready to accept requests. You can now connect it to your application, chatbot, RAG system, or any other tool that can work with an HTTP API.

Scaling and Server Management

If you eventually need more powerful models or the load increases, you will not have to migrate to a new server from scratch. The NodexGo control panel has an Upgrade Resources button — it moves the server to a higher-tier plan while preserving the disk, all data, and the IP address. Ollama and the downloaded models will remain in place; only a service restart will be required.

For convenient management of multiple models and request monitoring, you can additionally install Open WebUI — a web interface that runs on top of Ollama and provides a user-friendly chat interface, conversation history, and model management. This is especially useful when a team shares the server. Also remember to periodically update the models themselves with the ollama pull command — developers regularly release improved versions.

Conclusion

Deploying your own language model on a remote server is easier than it seems: Ollama handles all the complexity of packaging dependencies, while a VPS provides stable resources and constant API access. You get full control over your data, no request limits, and the ability to use any open models without being tied to external services. Rent a suitable server on NodexGo, choose a model you like — and your personal AI assistant will be ready to work in less than an hour.