AI System

Why Clover exists: any model reaches verified outcomes when governed by the Clover cycle

An outcome does not come from which model you use. It comes from how the system around the model runs the cycle (Context → Direction → Execution → Outcome → Growth).

To test that principle directly, we built the Clover AI System (clover-ai-system) so that Layer 1 (the Foundation Model Backbone) can be hot-switched across three models that span 1,600× in parameter scale and three distinct neural architectures—while keeping Layers 2 through 7 (the governing, retrieving, tool-executing, compiler-checking, and memory-accumulating Clover cycle) identical:

1. MoonshotAI Kimi K3 (kimi-k3 — 2.78T Total / 104B Active MoE): 93 layers (68 KDA recurrent + 24 MLA attention trunks, 896 routed 4-bit experts per layer with top-16 routing, D = 7,168, V = 163,840 tiktoken vocabulary, 1.4 TB expert store). Built for deep open-world synthesis and multi-GPU execution.

2. NVIDIA Nemotron 3.5 Super VL (nemotron-3.5-super-vl — 124.45B Total / 12.77B Active Hybrid MoE, Default): 88 single-mixer layers (40 Mamba-2 selective state-space layers + 40 Latent ReLU² MoE layers with 512 experts and top-22 routing in compressed 1,024-d latent space + 8 NoPE 16:1 GQA attention layers, D = 4,096, V = 131,072 Tekken vocabulary, native 32-layer C-RADIO vision/video encoder with EVS pruning, and native 2-layer MTP drafter).

3. Liquid AI LFM3-1.7B (lfm3-1.7b — 1.73B Dense Tri-Hybrid): 24 layers (13 width-7 Gated Short Convolutions + 5 Sliding-Window GQA layers + 6 Full Global GQA layers with zero-tail partial RoPE 0.5, per-head sigmoid gates, sandwich RMSNorm, and SwiGLU FFN, D = 2,048, V = 128,000 tied vocabulary, 3.24 GiB total weights).

What all three models do: their differences and similarities

Before wrapping the models in Clover, we downloaded their weights and source code on our server (k3, AMD Ryzen 9 7950X3D, 124 GiB RAM, and gpu, 8 × NVIDIA RTX 3090, 192 GiB VRAM) and verified every mathematical stage against the official models to max_abs_diff = 0.0 (clover-ai-system-switchable-models-context.md).

What all three models share (Similarities):

• Hybrid sequence mixing for long-context efficiency: None of the three is a pure full-attention Transformer. Each combines linear-state operators (68 KDA layers in Kimi K3, 40 Mamba-2 SSM layers in Nemotron 3.5 Super VL, and 13 width-7 Gated ShortConv + 5 Sliding-256 layers in LFM3-1.7B) with a small number of global attention layers (24 MLA, 8 NoPE GQA, and 6 Full GQA), supporting 1,048,576 (1M) session context tokens inside Clover.
• Identical precomputed C/OpenMP & CUDA container discipline: In Clover, all three models compile their per-layer normalization and attention/mixer projections into contiguous mmap-ready operator.bin streams and pruned trunks (K3TRK001, NEMOTRK1 with -46.05% trunk byte reduction, and LFM3TRK1 with -15.91% trunk byte reduction and 0 attention.bin files remaining).
• Full multimodal support inside Clover AI Studio: All three models support text, image understanding, video frame analysis, duplex audio/voice conversation, and high-resolution image, video, and audio generation across all standard formats (PNG, JPG, WebP, BMP, SVG, MP4, WebM, GIF, MP3, WAV, OGG, FLAC).
• The same standalone blind spots without Clover: On its own, every one of these three models guesses multi-step arithmetic, emits code with syntax or borrow-checker errors when a single token slips, cannot solve 9×9 constraint satisfaction grids reliably without a solver, and cannot know private organizational ontology records or prior session memory that are not in its weights.

What the Clover AI System is, and how it switches models at runtime

The model can propose; the system decides; the tools enforce; evidence verifies; humans remain accountable. Rather than hard-wiring the reasoning, memory, ontology, and verification logic inside one model’s weights, clover-ai-system separates Layer 1 (the switchable Foundation Model Backbone) from Layers 2 through 7 (the shared Clover cycle infrastructure) (switchable_model_backend.py):

1. Layer 1 — Switchable Foundation Model Backbone & Universal Dimension Bridges (libk3_gpu_experts.so, libnemotron_cpu_engine.so, liblfm3_cpu_engine.so): Each model runs its own native compiled C/OpenMP or CUDA engine over its own exact tokenizer vocabulary (163,840 for Kimi K3, 131,072 for Nemotron 3.5 Super VL, 128,000 for LFM3-1.7B). When you switch models in the UI or via CloverSwitchableModelLayer1.switch_model(model_key), Clover hot-swaps three lightweight adapters that map the active model’s hidden dimension $D_{\text{model}} \in \{7168, 4096, 2048\}$ into Clover’s shared upper layers: (a) a trained Native HybridLink (K3NativeHybridLink(7168), NemotronNativeHybridLink(4096) with 0.9985 test cosine, or Lfm3NativeHybridLink(2048) with 0.9994 test cosine), (b) a ComboBridge & ExpertBridge projecting $D_{\text{model}} \to 384$ into the 11-level NCM concept space, and (c) a Vocabulary-Grafted 11-Level VQ-VAE (99.83% exact token reconstruction on Nemotron’s 131K Tekken vocab and 100.00% on LFM3’s 128K vocab).
2. Layer 2 — HybridAgent Continuous <latent> Reasoning & Live Tools (clover-agent-engine.py): Runs continuous <latent> reasoning steps and dispatches live <search> (arXiv, Wikipedia, DuckDuckGo, URL fetch) and sandboxed <python> execution (exact arithmetic, constraint solving, unit tests, and charts) before the model commits to surface text.
3. Layer 3 — 11-Pass Parallel Next-Concept Modeling (NCM) & Lexical Pointer Alignment (benchmark_3_models_70_tasks.py): NextConceptTransformer (119.96M parameters) and K3LexicalPointerDecoder predict 11 hierarchical vector-quantized concept levels in parallel across 64-token windows and decode strictly inside the active model’s own vocabulary (all_emitted_token_ids_valid_in_model_vocab = True across all 70 tasks for all 3 models).
4. Layer 4 — File-Based Memory Tree & Relational Knowledge Graph (clover-memory-graph.py): Stores human-readable Markdown entries with YAML front matter across memory_tree/ (32 canonical seed entries, 65 typed cross-branch edges such as corrects, refines, challenges, and unresolved_with) with a closed-loop staging/ promotion gate and six upgrades from the 11 foundational AI papers (Origin).
5. Layer 5 — Ontology-Guided Neuro-Symbolic GraphRAG & Symbolic Verifier (clover-ontology-graph.py): Maintains deterministic ontology graphs, 4 Horn inference rules, and Newell–Simon CSm/VCt structural signature pruning (`22.5x` fewer DFS steps), supplying structured [E#] entity records and [P#] relationship paths and verifying mathematical, multiple-choice, and source-grounding invariants.
6. Layer 6 — On-the-Fly Generative Compilation for Python & Rust (clover-generative-compiler.py): Seals partial Python and Rust code prefixes at statement boundaries during generation, invokes Python AST/doctest checks and rustc --edition=2021, and uses Sutton $\text{TD}(\lambda)$ cliff localization (`6/6` offending statements localized) to roll back and repair broken statements before returning code.
7. Layer 7 — Direction B Data Movement & SpecMD Policy 3 Slot Caching (libslotcache.so): Manages 3-tier expert and trunk memory (VRAM / Pool ↔ Pinned Host RAM ↔ NVMe Direct) with SpecMD Least-Stale + Red-Blue Pebble Rule R4 Clean-Drop and Dynamic Batch Clubbing (B = 10 prompts per pass with segment boundary isolation seg_starts[t]), achieving a 99.03% cache hit rate and 0 dirty writebacks across the 70-task suite.

Benchmarking all three models without Clover vs. with the Clover cycle (70 Tasks)

To prove why Clover exists, we benchmarked all three models both ways on the exact same 70-task standardized suite (run-standardized-70-task-benchmark.py and benchmark_raw_models_without_clover_70_tasks.py), scored by the exact same unmodified frozen grading harness (clover_3_models_without_vs_with_clover_70_task_report.json):

• Without the Clover System (Arm 0 — Raw Model Alone): Every Clover layer is switched off (no Memory Graph, no Ontology GraphRAG, no <python> or <search> tools, and no Python/Rust Generative Compiler). Each model relies solely on its own forward pass and weights: kimi-k3 alone passes only 33 / 70 (47.14%); lfm3-1.7b alone (all 24 layers standalone) passes only 19 / 70 (27.14%); and nemotron-3.5-super-vl’s standalone stage engine passes only 16 / 70 (22.86%).
• With the Full 7-Layer Clover Cycle (Arm 2 — All 6 Surrounding Layers Active): When the exact same three models run inside the Clover cycle—where Context retrieves verified Ontology [E#]/[P#] and Memory Graph [M#] records, Direction enforces symbolic constraints and McCulloch–Pitts/Minsky–Papert One-in-a-Box vetoes, Execution runs <python>/<search> tools and each model’s native C/CUDA engine, Outcome verifies every code block with python3/rustc Generative Compilation and source-grounding checks, and Growth stages verified outcomes back into memory—every single model solves all 70 tasks: 70 / 70 (100.00% pass rate, 99.08% mean canonical score).

How each stage of the Clover cycle achieves outcomes the model cannot reach alone

1. Context supplies the private and structural reality missing from every model’s weights (0/6 → 6/6 Ontology, 2..3/6 → 6/6 Memory)

Without Clover, all three models score 0 / 6 (0.0%) on Category 5 (Ontology GraphRAG) and 33.3%–50.0% on Category 6 (Persistent Memory)—and even standalone frontier models (GPT-6 Astra 58.3%, Claude Opus 5.5 66.7%, Gemini 4 Argon 58.3%) fail those tasks because private records (Harbourside Council Mill Road, Payments Database Postmortem #209, Aurora Distributed Storage v4.2, and prior K3 cache/quantization measurements) do not exist in public pretraining corpora. Clover’s Context stage retrieves the exact [E#]/[P#] ontology chains and [M#] memory entries before generation begins, lifting every model to 6 / 6 (100.0%) on both categories.

2. Direction enforces domain boundaries and eliminates context pollution (16/16 → 0/16 false positives)

Using McCulloch & Pitts (1943) inhibitory vetoes and Minsky & Papert (1969) Order-$k$ “One-in-a-Box” conjunctive predicates, Clover blocks irrelevant memory/ontology injection on look-alike physics, biology, and sports distractors (cutting polluted context tokens from 9,244 → 0) while Rosenblatt’s zero-sum $\gamma$-rule and Vaswani/Krizhevsky $h=3$ multi-head LRN guarantee balanced evidence across uneven memory branches.

3. Execution pairs neural generation with deterministic tools (0..2/10 → 10/10 GSM8K, 0/5 → 5/5 Sudoku, 1..4/8 → 8/8 Agent QA)

On their own, the models guess multi-step word problems (`0/10` to `2/10` on GSM8K), fail 9×9 Sudoku constraint grids (`0/5`), and miss multi-hop calculations (`1/8` to `4/8`). Inside Clover’s Execution stage, HybridAgent dispatches <python> and <search> to compute exact numbers and constraint solutions first, and then renders the verified answer through the active model’s native C/CUDA engine and vocabulary (`131,072` Tekken, `128,000` LFM3, or `163,840` Kimi K3).

4. Outcome validates code and source grounding before returning (3..12/15 → 15/15 Coding, 6..7/15 → 15/15 Structured Output)

Standalone models frequently fail executable Python and Rust coding tasks (`3/15` for standalone `LFM3-1.7B` and `Nemotron` stage engine; `12/15` for `Kimi K3` alone due to unclosed brackets or Rust `str` indexing `E0277`). Clover’s Outcome stage runs statement-boundary Python AST/doctest checks and live rustc --edition=2021 compilation (`CloverPython/RustGenerativeCompiler`), rolling back broken statements mid-generation so all three models pass 15 / 15 (100.0%) executable coding tests and 15 / 15 (100.0%) structured output tasks.

5. Growth turns each verified cycle into the next cycle’s Context

Validated findings are staged into memory_tree/staging/ and promoted with explicit provenance and typed relationships (corrects, refines, challenges), so a mistake corrected once in any model is remembered for every model on the next cycle.

Head-to-head reference table: Clover 3-arm ablation vs. standalone frontier models (8 × RTX 3090)

For completeness, the table below records our controlled 3-arm ablation on kimi-k3 (Arm 0 raw model alone → Arm 1 adding Tools & Generative Compilation → Arm 2 adding Ontology & Memory) alongside three standalone cloud frontier models (GPT-6 Astra, Claude Opus 5.5, and Gemini 4 Argon, called without attaching Clover’s layers) on the exact same 70 prompts (Cycle 10 full report & raw JSON artifacts):

How the 8 × RTX 3090 GPUs are partitioned

Direction B — Five Data-Movement Upgrades Across CPU (k3), GPU (8 × RTX 3090) & All Three Models

All five Direction B data-movement upgrades and all six mechanisms from the 11 foundational papers (Origin) run across both CPU and GPU servers and all three switchable models (libk3_gpu_experts.so & libk3_cpu_experts.so for kimi-k3, libnemotron_cpu_engine.so for nemotron-3.5-super-vl, and liblfm3_cpu_engine.so for lfm3-1.7b):

Where we started, and where it is now

We started from Fareed Khan’s kimi-k3-in-c (Apache-2.0) on a single CPU workstation (AMD Ryzen 9 7950X3D, 124 GB RAM, dual NVMe RAID1), optimized the 93-layer CPU reference chain bit-identically, and then built the unified 7-layer clover-ai-system architecture running across both our k3 CPU workstation and our 8 × RTX 3090 GPU server across all three switchable models.

Credit

The Clover AI System (clover-ai-system) builds directly on the open weights, architectures, and published research of the teams below:

• NVIDIA — Nemotron 3.5 Super VL (nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16). The 124.45-billion-parameter (12.77B active) 88-layer hybrid Mamba-2 + 512-expert Latent ReLU² MoE + NoPE 16:1 GQA backbone (D = 4,096, V = 131,072 Tekken vocabulary, 1M native context), the 32-layer C-RADIO vision/video encoder with PixelShuffle and EVS video token pruning, and the 2-layer Native Multi-Token Prediction (MTP) head.
• Liquid AI — LFM3-1.7B (LiquidAI/LFM3-1.7B). The 1.73-billion-parameter 24-layer tri-hybrid model combining 13 width-7 Gated Short Convolutions (Lfm3ShortConv), 5 Sliding-Window GQA layers, and 6 Full Global GQA layers with zero-tail partial RoPE 0.5, per-head sigmoid gates, sandwich RMSNorm, and SwiGLU FFN (D = 2,048, V = 128,000 tied vocabulary).
• Moonshot AI — Kimi K3 (moonshotai/Kimi-K3). The 2.78-trillion-parameter 93-layer Mixture-of-Experts model (69 KDA + 24 MLA attention layers, 896 routed experts per layer, d = 7,168), the 27-layer MoonViT3d + PatchMergerMLPV2 vision tower, and the 163,840-token tiktoken.model vocabulary. The embedding table, output head, final normalization, vision weights, and tokenizer files are read from the original open weights under their own names.
• Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, and James Zou (Stanford University, Together AI, UIUC, Meta) — HybridAgent (arXiv:2610.18861, togethercomputer/HybridAgent). Hybrid Reasoning Agents: LLM Agents that Think in Latent Space, Act through Text. We use their HybridAgent-8B-Agentic model, HybridLink continuous <latent> reasoning mechanism, and <search> / <python> agent architecture, and extend the HybridLink design to Kimi K3’s 7,168-dimensional hidden states (K3NativeHybridLink).
• Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi, Andrew Woen, Charles Ding, E. Kelly Buchanan, Simran Arora, Azalia Mirhoseini, and Christopher Ré (Hazy Research, Stanford University) — Next-Concept Modeling (arXiv:2610.14162, HazyResearch/ncm, MIT License). Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation. We adapt their 11-level multiscale residual VQ-VAE tokenizer and coarse-to-fine NextConceptTransformer to Kimi K3’s 163,840-token vocabulary and 7,168-dimensional geometry.
• Domenico Benfenati and Antonio Maria Rinaldi (University of Naples Federico II) — Ontology-Guided Neuro-Symbolic GraphRAG (doi:10.1007/s00521-026-12495-0, CosmoIknosLab/ontology-guided-graphrag, MIT License). Ontology-Guided Neuro-Symbolic GraphRAG for Grounded Query Answering (Neural Computing and Applications, 2026). We build on their loosely-coupled neuro-symbolic design keeping deterministic ontology graph materialisation, Horn-rule inference, and edge provenance separate from neural generation, along with the six retrieval and verifier enhancements documented in clover-ontology-graphrag-research-context.md.
• Fareed Khan — kimi-k3-in-c (Apache-2.0). The C engine that showed Kimi K3 could be run from C on one workstation, where our CPU measurements started. Five of its tokenizer headers are included unmodified at commit a2ad8e5 with its licence and notice.
• Inferact — Kimi-K3-DSpark. The 5-layer dense MLA draft model used for speculative decoding on the CPU workstation.