This is the engineering blueprint for the air-gapped "military brain" Rafael floated in the Krakow meeting: a locked, self-hosted large language model with a retrieval knowledge base (RAG) fed with drone frequencies, drone behavior, test results and failure data, so a small R&D team can query prior experiments instead of burning cameras and parts re-discovering the same dead ends.
Picture this: it is Sunday morning, the coffee is hot, the family is still asleep, and Rafael is deciding not just whether to build this but how, what model is even legal to use for electronic-warfare work, what box it runs on for the money he hedged, and whether the data-feeding problem he himself called "hard to enforce" has any real answer. This page is what he reads to make that call without fooling himself.
The brain is the same RAG pattern Rafael already runs for two real businesses, pointed at military R&D data. He described the mechanism plainly: "you feed the, they call it RAC [RAG], you feed the knowledge base with that, and then basically the AI knows where to use it, and you can even ask questions." The military version swaps in "what frequencies are used for drones... how drones will fly and how it works" plus physics basics and test logs.
| Locked / air-gapped | No Google, no cloud. "It needs to be locked." A hosted model would leak or be blocked: "Once you put information there, God knows what happens... or sell the info." |
| Self-hosted open-weight | Off-the-shelf hosted models refuse military questions: "if you ask Lord [Claude] or Gemini any military related... they don't want to answer." So the build uses open-weight models on owned hardware. |
| Data-feeding crux | The unsolved problem, said twice. "The laboratories should be aware that they need to input something for it to go. That's hard to enforce." Section 05 is the answer. |
| Self-improving loop | Wanted, but Rafael named the weak step himself: you query, "you leave your desk and if it works or not the LLM stays there, you're not going back and give feedback." |
| Budget & load | "We need our own servers... 10, 20... maybe five, I don't know, I need to look because also RAM prices are like this." 2 to 3 simultaneous users at most. |
| Languages | Ukrainian primary, Russian for legacy and adversary material, English for standards and datasheets. |
Meta's Llama Acceptable Use Policy explicitly prohibits use for military, warfare, nuclear, espionage, ITAR-controlled items and weapon development (confirmed in the published llama.com use-policy). Meta added a national-security carve-out in November 2024, but it covers only US government and contractors. The frequently-cited "Five Eyes" extension (UK, Canada, Australia, New Zealand) comes from press reporting, not Meta's own announcement, which named only the US government. Either way, a Ukrainian commercial entity is not covered, and the base policy still bans military/EW use. The Open Source Initiative also confirmed in 2025 that Llama's license is not open source. Llama is legally inappropriate here. Use a permissive (Apache-2.0 / MIT) model with no field-of-use restriction instead.
| Candidate | License / field-of-use | UK / RU / EN | Size · VRAM (Q4) | Verdict |
|---|---|---|---|---|
| Qwen3-32B (dense) | Apache-2.0 no military restriction | 100+ langs incl. UK/RU/EN (verify UK quality on your own eval) | 32.8B · ~20-22 GB; fits a 48 GB card comfortably, tight on 24 GB | Primary pick. Best reasoning-per-VRAM with a clean license; vLLM/SGLang supported. |
| Qwen3-8B / Ministral 8B | Apache-2.0 both | UK/RU/EN supported | 8B · ~6-10 GB; single consumer GPU | Fast/cheap tier for the 2-3 user load; the PoC workhorse. |
| Mistral 3 / 14B | Apache-2.0 (Mistral 3 family, Dec 2025) | Ukrainian AND Russian listed by vendor | 14B · ~9-12 GB | Strong alternate; note older Mistral Large 2 used a non-commercial research license, Mistral 3 fixed this. |
| Lapa-12B / MamayLM-12B | Gemma Terms custom Google license | Best-in-class Ukrainian; propaganda-resistant | 12B · ~9-10 GB | Route Ukrainian-heavy queries here, but legal-review the Gemma Terms before defense use (not OSI-open). Ukraine's own national LLM is also Gemma-based. |
| DeepSeek R1 / V3 | mixed R1 weights MIT; V3 weights carry a custom agreement that bans military use | UK/RU/EN | 671B MoE · ~405 GB at Q4 (multi-GPU only) | Avoid as front-line. R1 has Chinese political censorship baked into the local weights, and DeepSeek-V3's model license itself prohibits military use. Too big and too constrained for v1. |
EW, frequency and weapon-adjacent prompts will trip built-in refusals on some models. The cheap, mature fix is an abliterated (refusal-removed) community variant of the chosen base, these exist for Qwen3, Gemma and others, and the "Heretic" tool automates the process. Treat abliterated weights as untrusted supply-chain artifacts: verify the underlying base license still applies and benchmark quality loss on real EW prompts.
But the stronger justification for going air-gapped is NOT "hosted models refuse." A 2025 military-doctrine benchmark found the best frontier hosted models actually had near-zero refusal rates. The load-bearing reason is data sovereignty: you cannot send classified RF signatures, drone failure logs and EW counter-tactics to any cloud regardless of whether it answers. The air-gap requirement, not the refusal behavior, is what makes self-hosting non-negotiable.
| Tier | GPU / box | VRAM math | Runs | 2026 price (est.) |
|---|---|---|---|---|
| PoC | Single used RTX 3090 (24 GB) or new RTX 5090 (32 GB) | 32B Q4 ~20-22 GB fits 24 GB tight; 8B FP16 comfortable | Qwen3-8B / Qwen3-32B Q4 for 1-2 users, short-to-mid context | ~$3-5k all-in (3090 used ~$700-1,000; 5090 street ~$3,500-4,000) |
| Pilot | One RTX 6000 Ada (48 GB ECC, 300W) preferred over 2x used 3090 | 70B Q4 ~42-45 GB fits one 48 GB card with KV headroom | Qwen3-32B at higher precision, or a 70B-class model, with real context | ~$8-12k all-in (RTX 6000 Ada ~$6,800; dual-3090 cheaper but no ECC) |
| Production | Single RTX PRO 6000 Blackwell (96 GB GDDR7 ECC, 600W) OR Mac Studio M3 Ultra (96 GB unified) | 96 GB holds 70B Q4 with huge context, or multiple resident models | The locked brain: 70B + embedder + room for the feedback loop | ~$11-16k for the box (RTX PRO 6000 ~$8,500 MSRP, street $8-9.2k+) |
RAG-Anything (LightRAG-based, already installed at ~/tools/RAG-Anything, lightrag-hku 1.4.13 + MinerU for multimodal PDF parsing) is the retrieval core, its knowledge-graph retrieval suits R&D data where failure modes, frequencies and behaviors interrelate. LanceDB (embedded, file-on-disk, no server to run) is the vector store, the cleanest fit for an air-gapped box. Whisper large-v3 stays the transcription model, but run it LOCALLY via faster-whisper, the repo's Groq Whisper is a CLOUD API and cannot transcribe classified audio inside the air gap.
| Layer | Pick | Why |
|---|---|---|
| Inference engine | vLLM (Ollama as low-effort fallback) | vLLM (PagedAttention + continuous batching) is the one engine that holds up under 2-3 concurrent users; Ollama is simplest to stand up but degrades sharply under concurrency. |
| RAG framework | RAG-Anything (LightRAG) | Already installed; graph-augmented multimodal ingestion of PDFs, images, tables. |
| Vector DB | LanceDB (Qdrant as escape hatch) | Embedded, no phone-home; Qdrant only if metadata filtering outgrows it. |
| Embeddings | BGE-M3 (MIT) | 100+ languages incl. UK/RU/EN, 1024-dim, returns dense + sparse + multi-vector from one model, hybrid retrieval matters for frequency numbers and part codes. |
| Transcription | faster-whisper large-v3 (local) | Same model weights as Groq, self-hosted; int8 runs on CPU. Force the language tag, Whisper mislabels Ukrainian as Russian on auto-detect. |
| Chat UI | Open WebUI (AnythingLLM fallback) | Mature multi-user RBAC, OIDC SSO, runs offline in Docker, points at the vLLM endpoint. |
| Eval / grounding gate | Ragas (local-judge) | Reference-free, runs offline using your own model as judge; measures Faithfulness, Answer Relevancy, Context Precision/Recall. |
The single highest-leverage move for a low-compliance frontline team is to meet operators where they already are. A self-hosted Telegram bot accepts a voice note or a forwarded document, runs local faster-whisper to transcribe (with an explicit UK/RU/EN language tag, and WhisperX + pyannote diarization for multi-speaker debriefs so the store knows who said what), asks 5-6 quick structured questions (or extracts them from the voice note with the local model), and auto-ingests into RAG-Anything. This is a proven off-the-shelf pattern; the only change for the air gap is swapping the cloud Whisper/LLM calls for local ones. It works because the operators already use Telegram daily, you add zero new friction.
"Now if you record the test, it could be one thing. It's small, but at least record microphones. It's easy to speak." (Rafael, in the meeting, already reaching for voice capture.)
There is no autonomous self-improvement. Once the model and retriever are fixed, no further learning happens on its own, exactly the weak link Rafael named. What "self-improving" really means is two human-gated mechanisms:
Public, non-sensitive enrichment (standards, datasheets, the open tier of the Brave1 catalog, which has no documented public API) is scraped on a SEPARATE internet-connected machine via Firecrawl/Brave, sanitized, and sneakernetted into the air-gapped brain. The brain itself never touches OpenRouter, Firecrawl, Brave or Groq. That separation is the whole point of "locked."
CONNECTED SIDE (separate machine) | AIR-GAPPED BRAIN (no NIC / Wi-Fi / BT)
-------------------------------- | +-------------------------------------------------+
Firecrawl / Brave -> public standards, | | |
datasheets, open Brave1 tier --[sneakernet | | [1] CAPTURE |
signed media]--->| Telegram bot <- voice note / doc forward |
| | | |
| | v |
| | [2] TRANSCRIBE / PARSE |
| | faster-whisper large-v3 (UK/RU/EN tag) |
| | WhisperX + pyannote (who said what) |
| | MinerU (PDF tables/images) |
| | + 5-6 structured intake questions |
| | | |
| | v |
| | [3] EMBED BGE-M3 (dense+sparse) |
| | | |
| | v |
| | [4] VECTOR STORE LanceDB + provenance |
| | | registry (SQLite/Postgres) |
| | v |
| | [5] RETRIEVE RAG-Anything / LightRAG graph |
| | | |
| | v |
| | [6] LOCAL LLM Qwen3-32B on vLLM |
| | | (abliterated if needed) |
| | v |
| | [7] CITED ANSWER must-cite + Ragas |
| | | faithfulness gate |
| | v (refuse/flag if ungrounded) |
| | Open WebUI -> engineer reads + thumbs |
| | | |
| | [8] FEEDBACK LOOP thumbs + test OUTCOME |
| | |___ re-ingest back into [4] __________|
| +-------------------------------------------------+
| Obstacle | Mitigation |
|---|---|
| Data-feeding (the crux) | Frictionless Telegram voice/doc capture, structured intake schema, a named data steward, and a combat-points-style incentive. Outcome-ingestion mandatory. |
| Security / air-gap | Physically air-gapped box; all enrichment on a separate machine; signed removable media only. Treat query and retrieval logs as classified too, they reveal what is being researched. |
| Model refusals | Deploy an abliterated (refusal-removed) variant of the chosen Apache/MIT base; validate on real EW prompts and verify the base license still applies. |
| Hallucination | Enforced must-cite (answers quote source doc + test ID), Ragas faithfulness gate that refuses or flags ungrounded answers, confidence gating, and a human engineer signs off before any number drives a build. |
| Multilingual quality | BGE-M3 hybrid retrieval; force the Whisper language tag; store original + machine-translated text per chunk so a UK query can hit a RU source. Cross-lingual retrieval still degrades 30-50 points, run a UK/RU EW-terminology eval before committing. |
| Currency of knowledge | Version-and-supersede entries, incremental re-embed on content-hash change, steward chases stale countermeasure data. |
| Cost | Tiered build (PoC reuses the existing stack cheaply); price 2026 GPUs as floors and re-quote; the Mac path lowers power and export friction. |
| GPU export controls | Ukraine is EAR Country Group B (trusted, not embargoed), so consumer/workstation GPUs ship lawfully. Avoid controlled H100/H200 datacenter cards (ECCN 3A090) and do not site hardware in EAR 746.6 covered regions. |
| Talent / maintenance | Rafael's AI/IoT team plus AJ build and host it; prefer ECC VRAM (cannot patch easily in the field); keep the stack to mature, well-documented offline tools (vLLM, Open WebUI, RAG-Anything). |
| Phase | Hardware | What it unlocks | Budget |
|---|---|---|---|
| PoC | Single used RTX 3090 (24 GB), 64 GB RAM, 2 TB NVMe | Reuse the existing stack (RAG-Anything + LanceDB + faster-whisper) with Qwen3-8B/32B Q4 and Ollama or vLLM on a tiny real dataset. Proves retrieval, citations and UK/RU quality before any big spend. | ~$3-5k |
| Pilot | RTX 6000 Ada (48 GB ECC), 128 GB RAM, 4 TB NVMe, 1000W + UPS | First tier that holds a 70B-class model at Q4 with real context; vLLM for true concurrency; Telegram capture, structured intake, provenance registry and the thumbs/outcome feedback loop go live. | ~$8-12k |
| Production | RTX PRO 6000 Blackwell (96 GB ECC) or Mac Studio M3 Ultra (96 GB) | The locked, always-on brain: large context, multiple resident models for the feedback loop, full air-gap hardening (LUKS2, egress-deny, auditd), data-steward operations. | ~$11-16k for the box |
Bottom line: the architecture is real and nationally validated (Ukraine's Palantir-backed Brave1 Dataroom, January 2026, and Helsing's productized air-gapped on-prem defense LLMs prove the pattern). The specific ROI, fewer burned cameras, is unproven, so sell it as engineering-hours saved and avoided repeat-experiments, not a per-component dollar figure. Build the cheap PoC first; let it earn the $5-20k box.