Recorded platform demonstration · October 25, 2026

The platform, on the record.

These are saved prompts, model responses, observed timings, search results, and an infrastructure-inventory snapshot from my self-hosted AI platform. The demonstration covers primary and fallback GPUs, CPU inference, retrieval, live search, and gateway routing.

Recorded output — not a live test2026-10-25 23:31 UTCSecurity-sanitized
8recorded demo sections
985tracked inventory components
3inference compute tiers
4published model aliases

Model identities below reflect the actual configured platform models from the current project documentation. The demo script's stale model labels have been corrected; recorded answer text and timing measurements are retained. Exact software, container, and kernel versions, internal monitoring ports, and implementation identifiers that would increase reconnaissance value are withheld.

01 Platform introspection

Live software inventory

Prometheus snapshot

The inventory exporter tracks host packages, container images, and Kubernetes workloads. At recording time it reported 985 components. Workload names and roles are retained below; version tags, precise package releases, and the kernel build are omitted.

KindAreaComponentExact version
DaemonSetCluster serviceskube-vipWithheld
DaemonSetGPU managementNVIDIA device pluginWithheld
DaemonSetCluster servicesK3s service load balancerWithheld
DeploymentInferenceLiteLLMWithheld
DeploymentInferenceOllamaWithheld
DeploymentInferencevLLMWithheld
DeploymentRAGPython RAG serviceWithheld
DeploymentGitOpsDexWithheld
DeploymentGitOpsRedisWithheld
DeploymentGitOpsArgo CD serverWithheld
DeploymentCluster servicesLocal-path storage provisionerWithheld
DeploymentCluster servicesCoreDNSWithheld
DeploymentCluster servicesTraefikWithheld
DeploymentCluster servicesMetrics serverWithheld
DeploymentObservabilityOpenTelemetry CollectorWithheld
StatefulSetRAGQdrantWithheld
StatefulSetGitOpsArgo CD controllerWithheld
Host packages represented

Git · NVIDIA Container Toolkit · NVIDIA Container Toolkit Base · OpenSSL · sudo

Kernel inventory

One reporting system in this sample; exact kernel release and patch revision withheld.

02 Inference · primary GPU

vLLM · Qwen3-14B-AWQ

RTX 5060 Ti · 16 GB VRAM

The primary, high-capacity inference path running Qwen3-14B-AWQ on the homelab GPU.

Recorded prompt

Explain what a KV cache is in LLM inference and why it matters for GPU memory.

Recorded response

A KV cache (Key-Value cache)in LLM inference stores the key and value vectors generated during the attention mechanism of transformer models. These cached values are reused in subsequent steps of the autoregressive generation process to avoid redundant computation. This caching is crucial for efficiency, as it reduces the computational overhead of repeatedly calculating attention scores for previously processed tokens. However, it also consumes significant GPU memory, especially for long sequences, which can limit the maximum context length or require more powerful hardware. Managing the KV cache effectively is therefore essential for optimizing both speed and memory usage during inference.
2,981 msModel alias: qwen-gpuBackend: vLLM

03 Inference · secondary GPU

Ollama · Qwen3.5-9B

GTX 1650 Super · 4 GB VRAM + RAM

The secondary node runs a hybrid GPU/CPU model, trading latency for a capable fallback inference tier.

Recorded prompt

Explain the benefit of heterogeneous GPU scheduling in a self-hosted AI inference cluster.

Recorded response

Heterogeneous GPU schedulingmaximizes hardware utilization by dynamically offloading smaller inference workloads from the primary high-VRAM node to the secondary smaller node. This approach prevents the expensive primary GPU from sitting idle during requests that do not require its full memory capacity, thereby improving overall cluster throughput. By balancing compute load based on model size and request complexity, it significantly reduces latency and cloud-like cost inefficiencies inherent in homogeneous configurations. Ultimately, this strategy ensures a cost-effective, scalable inference service that adapts seamlessly to fluctuating traffic patterns without needing identical hardware everywhere.
23,846 msModel alias: qwen-outpostBackend: Ollama

04 Inference · large CPU model

Ollama · Qwen3.6-35B-A3B

CPU/RAM only · MoE

A larger, CPU-only mixture-of-experts model provides a slow quality-oriented path. This request took approximately 2 minutes 51 seconds.

Recorded prompt

Explain the CAP theorem and its practical implications for distributed database design.

Recorded response (excerpt as captured by the demo script)

The CAP theorem, proposed by Eric Brewer in 2000, states that a distributed data store can only guarantee two out of the following three properties simultaneously:

  1. Consistency (C): Every read receives the most recent write or an error. All nodes see the same data at the same time.
  2. Availability (A): Every request receives a non-error response, without the guarantee that it contains the most recent write. The system remains active even if some data is stale.
  3. Partition Tolerance (P): The system continues to operate despite an arbitrary number of messages being dropped or delayed by the network between nodes.

The Core Premise: In a distributed system, network partitions are inevitable. Therefore, you cannot build a true distributed database that avoids partition tolerance ($P$). When a network split occurs and communication is severed, the system must make a choice: either halt operations to maintain Consistency (CP) or keep responding with potentially outdated data to maintain Availability (AP).


The Combinations and Concrete Examples

Designers primarily choose between Consistency and Availability when a partition ($P$) occurs.

1. CP Systems (Consistency + Partition Tolerance)

When a partition occurs, the system prioritizes data integrity. If nodes cannot verify with others that data is safe to write, they reject the request.

  • Behavior: High consistency but potential downtime during network failures.
  • Concrete Example: ZooKeeper
    • Context: Used for configuration management and leader election.
    • Mechanism: It operates on a majority quorum (Paxos algorithm). If the [output cut off at 2000 tokens]
170,697 msModel alias: qwen-cpuBackend: CPU-only Ollama

05 Retrieval-augmented generation

RAG over the platform's own documents

Qdrant · embeddings · qwen-gpu

A custom retrieval service searches the private corpus and sends relevant context to the model. Both recorded queries are shown, including the one that did not produce an answer.

Recorded prompt · monitoring

What monitoring ports does the platform expose, and what does each one serve?

Recorded response · internal ports redacted

The platform exposes the following monitoring ports: 1. Port [redacted] — This port is used by the Blackbox Exporter container. It serves the purpose of exporting metrics for external monitoring, such as HTTP probe results, which are then collected by Prometheus [5]. 2. Port [redacted] — This port is used by the Prometheus server. It serves as the web interface for querying metrics and is also used for federation, allowing the home Prometheus instance to scrape metrics from the monitor's Prometheus instance [3].
5,125 msModel alias: infra-rag

Recorded prompt · CVE findings

What CVEs are currently flagged as actively exploited on this platform?

Recorded response

I could not find this in the platform docs.
2,698 msModel alias: infra-rag

The second result is intentionally shown as recorded. It indicates this particular query did not retrieve the requested findings; it is not a verified statement that the platform has no actively exploited vulnerabilities.

07 Gateway routing

Fallback routing demonstration

LiteLLM · qwen-outpost

A controlled availability test removes the secondary Ollama service, sends a request through its public alias, then restores the service.

STEP 01Make fallback tier unavailable

Outpost Ollama scaled to zero replicas.

STEP 02Send the test request

Request targets qwen-outpost, with qwen-gpu configured as fallback.

STEP 03Restore the tier

The secondary Ollama service is restored after the test.

Recorded response

A GPU (Graphics Processing Unit) is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images for display.
11,770 msRequested alias: qwen-outpost

The recorded result shows a successful response while the outpost service was stopped. The response alias is still qwen-outpost; this excerpt does not independently expose the specific upstream that answered.

08 Gateway model discovery

Published models

/v1/models

The gateway advertised these four aliases in the recorded model-list response. The concrete model names here reflect the current deployment documentation.

qwen-gpu

Qwen3-14B-AWQ · vLLM · primary RTX 5060 Ti

qwen-outpost

Qwen3.5-9B · Ollama · secondary GTX 1650 Super + RAM

qwen-cpu

Qwen3.6-35B-A3B · CPU-only Ollama on homelab

infra-rag

Custom RAG service · Qdrant retrieval + Qwen3-14B generation