A private AI platform,
built from bare metal up.

What you just interacted with isn’t a cloud service—it’s a platform I built and run in my homelab. It combines Kubernetes, AI inference, RAG, observability, and security automation into a single environment that I designed, deployed, and maintain myself. This page gives a quick overview of the architecture, capabilities, and some examples of what the platform can do.

// What it is

This project started as a way to learn modern AI infrastructure by building and operating a complete platform myself, end to end.

At the center of the platform is a multi-tier inference environment. A primary RTX 5060 Ti node handles most GPU-backed inference workloads, while a secondary GTX 1650 Super node provides additional capacity and redundancy. Requests are routed through LiteLLM, which presents a single OpenAI-compatible API while handling model routing and automatic fallback between GPU and CPU resources when needed.

The platform also includes a retrieval-augmented generation (RAG) layer built on Qdrant, allowing models to answer questions using private documentation and operational knowledge instead of relying solely on model training data. For broader research tasks, the platform can combine retrieval with live web search results.

Around the inference stack is the operational infrastructure needed to run and maintain the platform. Metrics, logs, traces, and hardware utilization are collected and visualized through a centralized observability stack. Changes are deployed through GitOps workflows using Argo CD, with all infrastructure and application configuration stored in version control and automatically reconciled to the desired state.

To improve operational visibility and security, I also built an automated vulnerability-management pipeline. A daily workflow collects vulnerability intelligence from multiple public sources, compares it against a live inventory of software running across hosts, containers, and Kubernetes workloads, generates remediation recommendations, and creates issues for review. Relevant findings are automatically ingested into the platform’s knowledge base, allowing the system to answer questions about its own security posture.

The result is a self-hosted AI platform that combines inference, retrieval, observability, automation, and security into a single environment that I designed, built, and operate myself.

// The pillars

// Architecture, in broad strokes

From the outside, the architecture is fairly simple. A public edge accepts HTTPS requests and securely forwards them into the homelab over a private encrypted connection. The cluster itself—including the AI services, databases, and supporting infrastructure—isn’t directly exposed to the internet. Keeping most of the platform private reduces complexity and limits the amount of infrastructure that needs to be publicly accessible.

Exact topology, addressing, and the security internals are intentionally left off this page.

Platform architecture diagram view full size →