Two VPSes, a GPU that wakes itself up

Two machines, two different jobs

The entire solveiAgents platform — the orchestrator, fourteen specialized agents and, on top of them, three vertical applications already in production (MediLog, Survey, Legal) — runs on two Oracle Cloud Infrastructure (OCI) virtual machines, not one. The split isn't cosmetic: it reflects two opposite needs.

The first VPS is an Ampere ARM64 Compute instance with 24 GB of RAM, in a public VCN subnet, always on. It hosts everything that has to be able to respond at any moment: the reverse proxy, the orchestrator, the databases, and most of the agents — started and stopped on demand by an idle watcher, but on the container host hosting them, which never sleeps. The second is an NVIDIA A10 GPU, in a private subnet with no public address at all, powered on only when the compute power the ARM64 lacks is genuinely needed.

Here's how the work actually splits, in detail:

What happens when an agent needs the GPU

No agent talks to OCI's APIs directly. When a task needs a model that doesn't run comfortably on the ARM64, it calls gpu_manager, or the orchestrator's equivalent service used for starting/stopping it independently of the frontends' GPU widget. Both authenticate to OCI with an Instance Principal: no API key stored anywhere — the public VM's own identity, authorized by a Dynamic Group and a dedicated IAM policy, is the credential.

Once the task is done, the GPU doesn't stay on "just in case": an idle watcher shuts it down on its own after a configurable period. And if even that mechanism were to fail — an agent restart that resets an in-memory timer, a bug not yet discovered — a second, independent check, a hard uptime cap (four hours by default), shuts it down anyway.

What OCI actually does (and doesn't)

It's worth being precise here, because it's easy to picture a cloud infrastructure as a bundle of managed services — object storage, an API gateway, native monitoring — when in reality most of what runs on these two machines isn't an OCI service at all: it's containerized open-source software we manage ourselves. The OCI services actually involved number only four:

Why the GPU has no public address

The private subnet isn't an implementation detail: it's the design choice that makes it acceptable to keep a machine running an LLM that processes real requests powered on, even for just a few minutes. No port is ever published to the internet on the GPU VPS; the only way to reach it is through the public VPS, over the VCN's private IP, and only with the right IAM credentials.

The same discipline applies to user authentication: Clerk, the identity provider used by every frontend, runs in production mode, with JWT verification centralized in a single shared module consumed by five different services.

Logs, traces, and metrics: knowing what's happening before it becomes a problem

Centralized logs

Every agent — not just the three or four "main" ones — writes its own logs to Loki through a shared module.

Distributed traces

OpenTelemetry, exporting to Tempo, is now active on fifteen agents: every request carries a traceparent propagated from the orchestrator all the way into the agent serving it, so a slow request can be followed end to end instead of having to cross-reference logs by hand, service by service.

Metrics and alerts

Prometheus "watches" every active agent; two metrics purpose-built for supervision — agent_restarts_total and agent_up — feed a dedicated alert rule (AgentFlapping, an agent restarting more than three times in ten minutes) alongside the existing one for a circuit breaker stuck open too long. Alertmanager closes the loop: before it existed, rules only "fired" visually inside Prometheus, without notifying anyone.

Operational reporting

In practice, whoever operates the platform looks at two places. The auto-provisioned Grafana dashboard Solvei Agents Overview, with throughput, p50/p95 latency, and error rate per service. And the /monitoring panel in the frontend, with three sections: LiveSection shows in-flight requests in real time (status "classifying" → "streaming" → "completed," or "abandoned" if the client disconnected halfway through); LogViewer queries Loki by service, level, and time range without leaving the browser; DeployPanel runs git pull, build, restart, and an nginx reload as a live stream, straight from the UI.

Backup and disaster recovery

A daily pg_dump backup covers the entire Postgres instance — schema by schema: platform, MediLog's medical record, FHIR data, Survey, Legal, Umami. Alongside it, read-only tar archives for the binary data that doesn't live in the database: MinIO, Orthanc (DICOM studies), uploaded paper questionnaires, generated wikis, blog posts, Open WebUI data. An offsite rsync destination is available as an option, not enabled by default. A runbook (scripts/restore.sh) walks through the restore step by step — component by component, not all at once — with a dry-run mode on by default that prints the commands before actually running them.

The backup job now runs on the scheduled-job queue (more on that in the next paragraph), rather than as a direct call inside the orchestrator process: if the queue doesn't respond, the old direct path stays as a safety net — for backups, the risk of "nobody ran it" outweighs the risk of occasionally bypassing the queue.

What keeps everything standing when nobody's watching

A handful of mechanisms, all independent from each other, keep the platform consistent with no manual intervention:

Three vertical applications, one single platform

Everything described so far isn't an infrastructure exercise for its own sake: it's the foundation three real vertical products run on, sharing the same orchestrator, the same Postgres, the same Clerk authentication, and the same observability stack.

None of the three required new infrastructure from scratch: plugging in means registering a new agent in the existing registry, adding a schema to the shared Postgres, and getting everything this article described for free — logging, tracing, backups, and watchdogs included. It's probably the most concrete proof that the two-VPS architecture isn't just economically sound — it's also what lets a new vertical product get added in weeks, not months.

You don't need a huge cloud — you need an honest one

This architecture's value isn't in how many managed OCI services it uses — deliberately few — but in how clearly it separates "always on" from "on demand," and in the work that keeps both halves consistent: a log that always reaches Loki, a trace that crosses every service, a backup that runs even if the queue meant to trigger it doesn't respond, an agent that shuts itself down even when someone turned it on by hand by mistake.

This infrastructure was conceived as a "personal technology gym," but built using sound practices drawn from more than 35 years of hands-on experience 🙂 — and it could make a solid base for other kinds of projects. Sensitive data never leaves the infrastructure and only ever touches local services. Either way, to shrink the hypothetical attack surface, individual services exposed to the internet can be turned on and off through Nginx Proxy Manager (open source).

To push privacy even further, my next step is probably setting up a private email and messaging service based on open-source solutions... I'd say it's about time to stop sharing private information with Big Tech 🙂.

I hope this article is useful if you're considering a similar architecture for your own projects.