AI R&D Brain · Build Blueprint

AI R&D Brain - Build Blueprint

A buildable plan for the secure, air-gapped, on-prem LLM-plus-RAG "R&D brain" Rafael pitched in Krakow: which open-weight model, what hardware, what software, and the one thing that actually decides whether it works, how you feed it. Grounded in the meeting; corrected against a source check.
Scope  on-prem server, NOT the on-drone chip Users  2 to 3 concurrent Budget  ~$5-20k (hedged) Languages  UK primary · RU · EN

What it is

This is the engineering blueprint for the air-gapped "military brain" Rafael floated in the Krakow meeting: a locked, self-hosted large language model with a retrieval knowledge base (RAG) fed with drone frequencies, drone behavior, test results and failure data, so a small R&D team can query prior experiments instead of burning cameras and parts re-discovering the same dead ends.

Picture this: it is Sunday morning, the coffee is hot, the family is still asleep, and Rafael is deciding not just whether to build this but how, what model is even legal to use for electronic-warfare work, what box it runs on for the money he hedged, and whether the data-feeding problem he himself called "hard to enforce" has any real answer. This page is what he reads to make that call without fooling himself.

Honesty caveat: the brain is Rafael's pitch in the meeting, not a built product, and the transcript has no speaker diarization. Every dollar figure, VRAM number, license clause and benchmark below should be re-verified at build time, hardware prices and model licenses both move monthly. Where the underlying research could not be confirmed against a primary source, the claim is softened and flagged inline.

01 What it is, and what it answers

A grounded recap of the requirements this build has to satisfy, taken straight from the meeting.

The requirement, in Rafael's own words

The brain is the same RAG pattern Rafael already runs for two real businesses, pointed at military R&D data. He described the mechanism plainly: "you feed the, they call it RAC [RAG], you feed the knowledge base with that, and then basically the AI knows where to use it, and you can even ask questions." The military version swaps in "what frequencies are used for drones... how drones will fly and how it works" plus physics basics and test logs.

The hard constraints it must satisfy

Locked / air-gappedNo Google, no cloud. "It needs to be locked." A hosted model would leak or be blocked: "Once you put information there, God knows what happens... or sell the info."
Self-hosted open-weightOff-the-shelf hosted models refuse military questions: "if you ask Lord [Claude] or Gemini any military related... they don't want to answer." So the build uses open-weight models on owned hardware.
Data-feeding cruxThe unsolved problem, said twice. "The laboratories should be aware that they need to input something for it to go. That's hard to enforce." Section 05 is the answer.
Self-improving loopWanted, but Rafael named the weak step himself: you query, "you leave your desk and if it works or not the LLM stays there, you're not going back and give feedback."
Budget & load"We need our own servers... 10, 20... maybe five, I don't know, I need to look because also RAM prices are like this." 2 to 3 simultaneous users at most.
LanguagesUkrainian primary, Russian for legacy and adversary material, English for standards and datasheets.
Not this server: the Nvidia "nano" ~$200 chip Rafael also raised is a SEPARATE, on-drone idea (onboard return-to-base and targeting compute that flies on the airframe). It is not a candidate for this server room brain and is costed and discussed elsewhere in the debrief. This blueprint is only the ground-based R&D brain.

02 Recommended LLM and licensing

For a Ukrainian commercial entity doing electronic-warfare and drone R&D, the license is the first filter, not an afterthought. It rules out the most famous model outright.

The Llama trap, avoid Meta Llama entirely

Meta's Llama Acceptable Use Policy explicitly prohibits use for military, warfare, nuclear, espionage, ITAR-controlled items and weapon development (confirmed in the published llama.com use-policy). Meta added a national-security carve-out in November 2024, but it covers only US government and contractors. The frequently-cited "Five Eyes" extension (UK, Canada, Australia, New Zealand) comes from press reporting, not Meta's own announcement, which named only the US government. Either way, a Ukrainian commercial entity is not covered, and the base policy still bans military/EW use. The Open Source Initiative also confirmed in 2025 that Llama's license is not open source. Llama is legally inappropriate here. Use a permissive (Apache-2.0 / MIT) model with no field-of-use restriction instead.

CandidateLicense / field-of-useUK / RU / ENSize · VRAM (Q4)Verdict
Qwen3-32B (dense) Apache-2.0 no military restriction 100+ langs incl. UK/RU/EN (verify UK quality on your own eval) 32.8B · ~20-22 GB; fits a 48 GB card comfortably, tight on 24 GB Primary pick. Best reasoning-per-VRAM with a clean license; vLLM/SGLang supported.
Qwen3-8B / Ministral 8B Apache-2.0 both UK/RU/EN supported 8B · ~6-10 GB; single consumer GPU Fast/cheap tier for the 2-3 user load; the PoC workhorse.
Mistral 3 / 14B Apache-2.0 (Mistral 3 family, Dec 2025) Ukrainian AND Russian listed by vendor 14B · ~9-12 GB Strong alternate; note older Mistral Large 2 used a non-commercial research license, Mistral 3 fixed this.
Lapa-12B / MamayLM-12B Gemma Terms custom Google license Best-in-class Ukrainian; propaganda-resistant 12B · ~9-10 GB Route Ukrainian-heavy queries here, but legal-review the Gemma Terms before defense use (not OSI-open). Ukraine's own national LLM is also Gemma-based.
DeepSeek R1 / V3 mixed R1 weights MIT; V3 weights carry a custom agreement that bans military use UK/RU/EN 671B MoE · ~405 GB at Q4 (multi-GPU only) Avoid as front-line. R1 has Chinese political censorship baked into the local weights, and DeepSeek-V3's model license itself prohibits military use. Too big and too constrained for v1.
Two corrections applied: (1) earlier research claimed "DeepSeek V3 weights are MIT", a source check refuted this, DeepSeek-V3's LICENSE-MODEL is a custom agreement that explicitly forbids military use; only R1, V3.1 and V3.2 weights are MIT. (2) DeepSeek 671B at Q4 is roughly 405 GB, not the ~143 GB figure that floated around (143 GB is roughly Qwen3-235B). Either way it needs a multi-GPU rig above this budget. Reserve the giant MoEs (DeepSeek, Qwen3-235B, Mistral Large 3) for a later phase only if budget tops $20k.

On refusals, and the real reason to self-host

EW, frequency and weapon-adjacent prompts will trip built-in refusals on some models. The cheap, mature fix is an abliterated (refusal-removed) community variant of the chosen base, these exist for Qwen3, Gemma and others, and the "Heretic" tool automates the process. Treat abliterated weights as untrusted supply-chain artifacts: verify the underlying base license still applies and benchmark quality loss on real EW prompts.

But the stronger justification for going air-gapped is NOT "hosted models refuse." A 2025 military-doctrine benchmark found the best frontier hosted models actually had near-zero refusal rates. The load-bearing reason is data sovereignty: you cannot send classified RF signatures, drone failure logs and EW counter-tactics to any cloud regardless of whether it answers. The air-gap requirement, not the refusal behavior, is what makes self-hosting non-negotiable.

03 Machinery / hardware

Three tiers anchored to the ~$5-20k ballpark. Sizing rule: VRAM = Q4 weights + KV cache + ~1.2x overhead; budget the GPU to (weights + 8-12 GB headroom) so 32-64K context and 2-3 users fit.
TierGPU / boxVRAM mathRuns2026 price (est.)
PoC Single used RTX 3090 (24 GB) or new RTX 5090 (32 GB) 32B Q4 ~20-22 GB fits 24 GB tight; 8B FP16 comfortable Qwen3-8B / Qwen3-32B Q4 for 1-2 users, short-to-mid context ~$3-5k all-in (3090 used ~$700-1,000; 5090 street ~$3,500-4,000)
Pilot One RTX 6000 Ada (48 GB ECC, 300W) preferred over 2x used 3090 70B Q4 ~42-45 GB fits one 48 GB card with KV headroom Qwen3-32B at higher precision, or a 70B-class model, with real context ~$8-12k all-in (RTX 6000 Ada ~$6,800; dual-3090 cheaper but no ECC)
Production Single RTX PRO 6000 Blackwell (96 GB GDDR7 ECC, 600W) OR Mac Studio M3 Ultra (96 GB unified) 96 GB holds 70B Q4 with huge context, or multiple resident models The locked brain: 70B + embedder + room for the feedback loop ~$11-16k for the box (RTX PRO 6000 ~$8,500 MSRP, street $8-9.2k+)
2026 reality, build the prices in as floors. A structural DRAM/GDDR shortage has pushed high-VRAM GPU street prices 50-100% over MSRP, so re-quote before buying. Apple has discontinued the 512 GB and 256 GB Mac Studio configs; as of mid-2026 the M3 Ultra Studio is effectively limited to a single 96 GB config, so the "512 GB unified memory Mac runs a 671B model" play is NOT buyable new. Skip the NVIDIA DGX Spark as the production brain, its LPDDR5x memory is bandwidth-starved (community reports ~4.6 tok/s and ~133s time-to-first-token on 70-90B models), which is unusable for interactive Q&A. Skip H100/H200 too: correct but $25-45k each, over budget and overkill for 2-3 users. Add the supporting BOM to every tier: 64-128 GB system RAM, 2-4 TB Gen4 NVMe, an 850-1300W Platinum PSU, and a UPS sized to the GPU (the Mac's sub-200W draw makes UPS and air-gapping trivially easy, a genuine advantage in a lab or field setting).

04 Software stack

Everything below is open-weight, MIT/Apache-licensed, and runs with zero external network calls. It deliberately reuses what is already proven in this repo.

Reuse what is already proven here

RAG-Anything (LightRAG-based, already installed at ~/tools/RAG-Anything, lightrag-hku 1.4.13 + MinerU for multimodal PDF parsing) is the retrieval core, its knowledge-graph retrieval suits R&D data where failure modes, frequencies and behaviors interrelate. LanceDB (embedded, file-on-disk, no server to run) is the vector store, the cleanest fit for an air-gapped box. Whisper large-v3 stays the transcription model, but run it LOCALLY via faster-whisper, the repo's Groq Whisper is a CLOUD API and cannot transcribe classified audio inside the air gap.

LayerPickWhy
Inference enginevLLM (Ollama as low-effort fallback)vLLM (PagedAttention + continuous batching) is the one engine that holds up under 2-3 concurrent users; Ollama is simplest to stand up but degrades sharply under concurrency.
RAG frameworkRAG-Anything (LightRAG)Already installed; graph-augmented multimodal ingestion of PDFs, images, tables.
Vector DBLanceDB (Qdrant as escape hatch)Embedded, no phone-home; Qdrant only if metadata filtering outgrows it.
EmbeddingsBGE-M3 (MIT)100+ languages incl. UK/RU/EN, 1024-dim, returns dense + sparse + multi-vector from one model, hybrid retrieval matters for frequency numbers and part codes.
Transcriptionfaster-whisper large-v3 (local)Same model weights as Groq, self-hosted; int8 runs on CPU. Force the language tag, Whisper mislabels Ukrainian as Russian on auto-detect.
Chat UIOpen WebUI (AnythingLLM fallback)Mature multi-user RBAC, OIDC SSO, runs offline in Docker, points at the vLLM endpoint.
Eval / grounding gateRagas (local-judge)Reference-free, runs offline using your own model as judge; measures Faithfulness, Answer Relevancy, Context Precision/Recall.
Corrections applied: "Chunk Attribution" is not a Ragas metric (it belongs to other tools), dropped. The arXiv id 2510.12323 is the RAG-Anything paper, not LightRAG (LightRAG is arXiv 2410.05779), fixed in Sources. BGE-M3's dense output drops straight into RAG-Anything; its sparse/multi-vector hybrid may need custom wiring, verify the plug points.

05 How to feed it, the crux

This is the section that decides everything. Rafael said the data-feeding problem twice and called it "hard to enforce." Technology alone does not solve it, but a frictionless front door plus one accountable owner plus a working incentive does.

One frictionless front door: a Telegram bot

The single highest-leverage move for a low-compliance frontline team is to meet operators where they already are. A self-hosted Telegram bot accepts a voice note or a forwarded document, runs local faster-whisper to transcribe (with an explicit UK/RU/EN language tag, and WhisperX + pyannote diarization for multi-speaker debriefs so the store knows who said what), asks 5-6 quick structured questions (or extracts them from the voice note with the local model), and auto-ingests into RAG-Anything. This is a proven off-the-shelf pattern; the only change for the air gap is swapping the cloud Whisper/LLM calls for local ones. It works because the operators already use Telegram daily, you add zero new friction.

"Now if you record the test, it could be one thing. It's small, but at least record microphones. It's easy to speak." (Rafael, in the meeting, already reaching for voice capture.)

Structured intake beats free text

Provenance and versioning (cheap, mandatory)

The real feedback loop, reframed honestly

There is no autonomous self-improvement. Once the model and retriever are fixed, no further learning happens on its own, exactly the weak link Rafael named. What "self-improving" really means is two human-gated mechanisms:

Caveat: the often-quoted "accuracy went 81% to 92% over six months" figure is a single vendor case study with no primary source, treat it as illustrative, not a promise. Sell this as "gets better as you feed and correct it," never as "learns on its own."

Two organizational fixes that matter more than any model choice

Correction applied: an earlier claim that "240,000 drones were ordered via the points program" was refuted, 240,000 is the 2025 count of enemy personnel killed/wounded. Actual equipment delivered via the points program was roughly 71,000 units including about 22,000 FPV drones. The incentive mechanism is real; that specific number was wrong.

Keep enrichment outside the wire

Public, non-sensitive enrichment (standards, datasheets, the open tier of the Brave1 catalog, which has no documented public API) is scraped on a SEPARATE internet-connected machine via Firecrawl/Brave, sanitized, and sneakernetted into the air-gapped brain. The brain itself never touches OpenRouter, Firecrawl, Brave or Groq. That separation is the whole point of "locked."

06 Architecture

Capture to cited answer, with the feedback loop closing back into the store. Everything inside the dashed box is air-gapped.
  CONNECTED SIDE (separate machine)            |   AIR-GAPPED BRAIN  (no NIC / Wi-Fi / BT)
  --------------------------------             |   +-------------------------------------------------+
  Firecrawl / Brave  -> public standards,      |   |                                                 |
  datasheets, open Brave1 tier  --[sneakernet  |   |   [1] CAPTURE                                    |
                                  signed media]--->|     Telegram bot  <-  voice note / doc forward  |
                                             |   |          |                                      |
                                             |   |          v                                      |
                                             |   |   [2] TRANSCRIBE / PARSE                         |
                                             |   |     faster-whisper large-v3 (UK/RU/EN tag)      |
                                             |   |     WhisperX + pyannote (who said what)         |
                                             |   |     MinerU (PDF tables/images)                  |
                                             |   |     + 5-6 structured intake questions           |
                                             |   |          |                                      |
                                             |   |          v                                      |
                                             |   |   [3] EMBED        BGE-M3 (dense+sparse)         |
                                             |   |          |                                      |
                                             |   |          v                                      |
                                             |   |   [4] VECTOR STORE   LanceDB  + provenance      |
                                             |   |          |           registry (SQLite/Postgres) |
                                             |   |          v                                      |
                                             |   |   [5] RETRIEVE   RAG-Anything / LightRAG graph  |
                                             |   |          |                                      |
                                             |   |          v                                      |
                                             |   |   [6] LOCAL LLM   Qwen3-32B on vLLM             |
                                             |   |          |        (abliterated if needed)       |
                                             |   |          v                                      |
                                             |   |   [7] CITED ANSWER  must-cite + Ragas           |
                                             |   |          |          faithfulness gate           |
                                             |   |          v          (refuse/flag if ungrounded) |
                                             |   |     Open WebUI  ->  engineer reads + thumbs     |
                                             |   |          |                                      |
                                             |   |   [8] FEEDBACK LOOP  thumbs + test OUTCOME      |
                                             |   |          |___ re-ingest back into [4] __________|
                                             |   +-------------------------------------------------+
  

07 Obstacles and how to overcome them

The honest risk register, each with a concrete mitigation.
ObstacleMitigation
Data-feeding (the crux)Frictionless Telegram voice/doc capture, structured intake schema, a named data steward, and a combat-points-style incentive. Outcome-ingestion mandatory.
Security / air-gapPhysically air-gapped box; all enrichment on a separate machine; signed removable media only. Treat query and retrieval logs as classified too, they reveal what is being researched.
Model refusalsDeploy an abliterated (refusal-removed) variant of the chosen Apache/MIT base; validate on real EW prompts and verify the base license still applies.
HallucinationEnforced must-cite (answers quote source doc + test ID), Ragas faithfulness gate that refuses or flags ungrounded answers, confidence gating, and a human engineer signs off before any number drives a build.
Multilingual qualityBGE-M3 hybrid retrieval; force the Whisper language tag; store original + machine-translated text per chunk so a UK query can hit a RU source. Cross-lingual retrieval still degrades 30-50 points, run a UK/RU EW-terminology eval before committing.
Currency of knowledgeVersion-and-supersede entries, incremental re-embed on content-hash change, steward chases stale countermeasure data.
CostTiered build (PoC reuses the existing stack cheaply); price 2026 GPUs as floors and re-quote; the Mac path lowers power and export friction.
GPU export controlsUkraine is EAR Country Group B (trusted, not embargoed), so consumer/workstation GPUs ship lawfully. Avoid controlled H100/H200 datacenter cards (ECCN 3A090) and do not site hardware in EAR 746.6 covered regions.
Talent / maintenanceRafael's AI/IoT team plus AJ build and host it; prefer ECC VRAM (cannot patch easily in the field); keep the stack to mature, well-documented offline tools (vLLM, Open WebUI, RAG-Anything).

08 Security, air-gap and legal

The threat model near a front line is physical capture, not remote hacking. Build for that.

Air-gap vs isolated

Encryption, seizure and wipe

Export control and license reality

09 Phased build and budget

Walk it up in three steps. Each phase unlocks the next and keeps the cheap, falsifiable test first.
PhaseHardwareWhat it unlocksBudget
PoC Single used RTX 3090 (24 GB), 64 GB RAM, 2 TB NVMe Reuse the existing stack (RAG-Anything + LanceDB + faster-whisper) with Qwen3-8B/32B Q4 and Ollama or vLLM on a tiny real dataset. Proves retrieval, citations and UK/RU quality before any big spend. ~$3-5k
Pilot RTX 6000 Ada (48 GB ECC), 128 GB RAM, 4 TB NVMe, 1000W + UPS First tier that holds a 70B-class model at Q4 with real context; vLLM for true concurrency; Telegram capture, structured intake, provenance registry and the thumbs/outcome feedback loop go live. ~$8-12k
Production RTX PRO 6000 Blackwell (96 GB ECC) or Mac Studio M3 Ultra (96 GB) The locked, always-on brain: large context, multiple resident models for the feedback loop, full air-gap hardening (LUKS2, egress-deny, auditd), data-steward operations. ~$11-16k for the box

10 Bottom line and first step

What to do Monday, for the price of a weekend

  1. Stand up a free PoC on hardware Rafael may already have: install vLLM (or Ollama) with Qwen3-8B (Apache-2.0), point the already-installed RAG-Anything + LanceDB at a handful of real, sanitized test reports, and ask it three EW questions in Ukrainian. owner: Rafael + AJ · this week
  2. Wire a throwaway Telegram bot to local faster-whisper and auto-ingest one voice note end-to-end. This tests the feeding crux for almost nothing. owner: AJ · next 1-2 weeks
  3. Run a tiny UK/RU EW-terminology eval with Ragas to see if the answers are grounded and the language quality holds before committing to a 48 GB or 96 GB GPU. owner: Rafael + AJ
  4. Name the data steward and pick the incentive before buying anything, the organization decides this, not the model. owner: Rafael + partner

Bottom line: the architecture is real and nationally validated (Ukraine's Palantir-backed Brave1 Dataroom, January 2026, and Helsing's productized air-gapped on-prem defense LLMs prove the pattern). The specific ROI, fewer burned cameras, is unproven, so sell it as engineering-hours saved and avoided repeat-experiments, not a per-component dollar figure. Build the cheap PoC first; let it earn the $5-20k box.

11 Sources

The references behind the picks above. License, price and benchmark figures should be re-verified at build time.
  1. Meta Llama 4 Acceptable Use Policy — military/warfare/weapon prohibition.
  2. Meta makes Llama available for US national security applications (TechCrunch) — the carve-out scope.
  3. Meta's Llama license is still not open source (Open Source Initiative).
  4. Qwen3-32B model card and LICENSE (Apache-2.0) — 32.8B, 100+ languages, vLLM/SGLang serving.
  5. Introducing Mistral 3 (Apache-2.0 family, Dec 2025) — Ukrainian and Russian listed.
  6. MamayLM-Gemma-3-12B and Lapa LLM v0.1.2 (Gemma Terms) — best Ukrainian.
  7. DeepSeek-V3 LICENSE-MODEL (military-use restriction) and R1dacted: local censorship in DeepSeek R1.
  8. Heretic and Uncensor any LLM with abliteration (mlabonne).
  9. RTX PRO 6000 Blackwell pricing and specs (96 GB GDDR7 ECC).
  10. Used RTX 3090 price history and Mac Studio 512 GB option discontinued.
  11. Ollama vs vLLM concurrency benchmark (Red Hat).
  12. BGE-M3 embeddings (MIT, dense+sparse+multi-vector, UK/RU/EN).
  13. RAG-Anything technical report (arXiv 2510.12323) and LightRAG (arXiv 2410.05779).
  14. Ragas metrics (Faithfulness, Answer Relevancy, Context Precision/Recall).
  15. faster-whisper and WhisperX + pyannote community-1 diarization.
  16. Groq speech-to-text (cloud API, incompatible with air gap).
  17. Metadata-for-RAG precision lift (73% to 83%) and RAGShield (provenance vs KB poisoning).
  18. Brave1 Market launch and Brave1 Dataroom (Palantir, Jan 2026).
  19. Helsing + Mistral defense AI partnership (air-gapped on-prem deployment).
  20. BIS rescinds the AI Diffusion Rule and Ukraine moved to EAR Country Group B.
  21. cryptsetup luksErase (header-wipe seizure runbook) and vLLM usage-stats opt-out.
  22. FPV drone economics (~25% pad failure, ~43% on-target, ~31% jammed).