From Laptop to Cluster: Taking an AI Agent Live with vLLM on OpenShift

An agent running on a laptop is a dream for exactly one person: Qwen3 does the reasoning, Ollama does the serving, no message data ever leaves the machine, and the bill is zero. But the dream ends the moment the lid closes for the commute home, or a second person wants to tap the same mind; a one-machine, one-user rig collapses under multi-user reality. The presenter builds the whole question from that fracture point: how do we turn this quiet desktop setup into a service a crowd can use, without falling into the one-machine trap ?

The starting rig is lean: a three-tool agent carrying a calculator, a web searcher, and a GitHub MCP connection, with reasoning supplied by the Qwen3 open-weight model and serving handled by Ollama on the laptop. Because message data never leaves the box, there is no privacy headache and no cost line; trial-and-error freedom is total. The host frames this as one-person, one-machine comfort, then states the production question crisply: how does the same agent live on without depending on a laptop with its lid shut?

The talk's strongest idea lands in the first minute: nothing inside the agent changes at all — what changes is where the model gets served and where the agent runs. There are two moves: first Ollama gives way to an inference server tuned for concurrent traffic, then the whole rig is containerized and moved onto an OpenShift cluster. The same shape works with outside endpoints such as Claude or GPT; the agent does not care where reasoning comes from as long as it has an endpoint abstraction to talk to. The Unsloth docs describe servers like Ollama and vLLM connecting through the same OpenAI-compatible interface pattern, which is why the laptop-to-cluster difference shrinks to three lines in a .env file.

First move: the model layer

Move one is the model layer. The laptop's Ollama setup is replaced by vLLM serving through OpenShift AI; vLLM is a Berkeley-born open project that, according to the RedHat blog introduction, was built for accelerator-memory efficiency and speed, gathering more than 40,000 stars on GitHub. As TheNewStack's internals piece explains, the server carries stacked calls on one model efficiently through continuous batching and paged-attention techniques, supporting families from Llama and Mistral to Granite and DeepSeek. In the AI Hub flow the model is pulled from a HuggingFace URI address, accelerator resources and a runtime are picked, and the deployment produces an internal link for in-cluster access, an external link for outside access, plus an API key that authenticates requests. The .env diff is three lines: the cluster's vLLM service instead of localhost Ollama, the fresh key, and the model identity vLLM expects.

The mind in the machine is Qwen3. Per the official announcement published on GitHub, the flagship 235B-A22B trades blows with top-tier models like DeepSeek-R1 and Gemini-2.5-Pro on coding and math, while a small mixture-of-experts build such as 30B-A3B and dense builds from 0.6B to 32B ship as open weights under Apache 2.0. A long context window and a hybrid thinking mode make the family a natural fit for tool-calling agent workloads; the free laptop rig in the demo is the everyday face of that licensing openness.

Second move: container and cluster

Move two is the agent itself. First a login command is grabbed from the OpenShift console and the cluster is entered; then the image is built with Podman, shipped through Quay, and rolled out — Docker does the same job. Make commands in the repo compress the build, push, and deploy steps into single keystrokes. A minute later the model endpoint and the agent run as side-by-side pods in the same cluster; the pod is scheduled, the image is pulled, and the agent that lived on a laptop a minute ago now lives as a pod on OpenShift. Not a line of code changed — only the home address did.

In the live trial the agent is first probed with a web-search and an arithmetic question, then asked to read an issue on GitHub and open a new one, with the same MCP tool shouldering reads and writes from inside the cluster. Chat logs are watched in the RedHat OpenShift AI playground instead of a local terminal; everything from the model to the agent to the playground runs in the cluster, and the setup shifts to in-cluster log watching.

Deployed is not production-ready

The finale draws an honest boundary: the pod can read any file inside its own container and reach out to nearly anything it wants; behavior visibility, resource limits, safe multi-team use, permissions assigned to the agent, and behavior scoping are all absent. Under the zero trust lens of the CNCF piece, unrestricted network traffic and open endpoints in AI workloads on clusters are the expensive edition of classic problems — a stolen key is not a small leak but a swelling bill. The open Agent Sandbox design covered by InfoQ builds isolation with a gVisor curtain, while the ephemeral-sandbox direction Google presented at KubeCon targets the same gap. The repo and the manifest wait in the links; the lockdown work is left as the next video's subject.