Inkling 2 Mini: 26B Multimodal Looped Transformer for Autonomous Engineering

Deep architectural analysis of Inkling 2 Mini (26B parameters) by Thinking Machines Lab. Covers autoregressive looped transformer mechanics, NVFP4 and MXFP8 mixed-precision local deployment, native tool_declare XML protocol, and orchestrating multi-agent engineering workflows with Smoke Monkey Harness and Canvas across 2,500+ MCP tools.
Inkling 2 Mini: 26B Multimodal Looped Transformer for Autonomous Engineering: Deep architectural analysis of Inkling 2 Mini (26B parameters) by Thinking Machines Lab. Covers autoregressive looped transformer mechanics, NVFP4 and MXFP8 mixed-precision local deployment, native tool_declare XML protocol, and orchestrating multi-agent engineering workflows with Smoke Monkey Harness and Canvas across 2,500+ MCP tools. Designed as a zero-dependency, open-source TypeScript architecture under the MIT License with native Model Context Protocol (MCP) support and deterministic phase state machines.
- Autoregressive Looped Transformer: 26B dense parameters with dynamic reasoning effort scaling (0.0 to 0.99) for adaptive step-by-step thinking.
- Single 20GB Checkpoint: Native mixed NVFP4 and MXFP8 numerics allowing full frontier multimodal inference on a single 24GB workstation GPU.
- Native Structured Tool Calling: Direct tool_declare XML stream parsing eliminating JSON syntax hallucinations during terminal and code edits.
- Zero Cloud Lock-In: SGLang local serving coupled with Smoke Monkey Harness and 2,500+ Model Context Protocol (MCP) servers.
import { createAgent, createWorkspace } from "smoke-monkey-harness";// Initialize local workspace with human-in-the-loop safety boundariesconst workspace = await createWorkspace({root: process.cwd(),sandbox: "process-isolated",});// Connect to local SGLang serving Thinking Machines Inkling 2 Miniconst agent = createAgent({provider: "custom",baseUrl: "http://localhost:30000/v1",model: "thinkingmachines/Inkling-2-Mini",workspace,thinkingEffort: 0.85, // Scale reasoning depth from 0.0 (fast) to 0.99 (deep)permissions: {read_file: "allow",write_file: "ask",run_command: "ask",},});// Execute autonomous engineering task with native MCP tool declarationconst result = await agent.run("Inspect AST of src/lib, identify circular module dependencies, and generate a migration plan.");console.log("Reasoning Steps:", result.steps);console.log("Result:", result.summary);
Watch: Related Video Guides
Why Everyone Is Talking About Inkling 2 Mini
Inkling 2 Mini Is Insane: The New King of 24GB GPUs?
Architecture: The Autoregressive Looped Transformer
Published by Thinking Machines Lab in late November 2026, Inkling 2 Mini represents a significant architectural departure from standard dense transformers. Rather than scaling parameter counts beyond 70B, it employs an autoregressive looped transformer topology across 26B parameters.
How the Looped Reasoning Loop Works
- Dynamic Thinking Effort Macro: Inkling 2 Mini introduces an explicit
reasoning_effortmacro ranging from0.0(direct immediate response) to0.99(deep multi-pass tree-of-thought exploration). When set above0.7, the model emits internal<|message_system|>Thinking effort level: X<|end_message|>tags and loops through intermediate validation states before proposing file modifications. - Multimodal Native Input: In addition to standard code and text tokens, Inkling 2 Mini accepts image and diagram inputs natively within its input stream, allowing agents to inspect UI screenshots, CAD schematics, and terminal diffs directly.
- Structured Tool Declaration: Unlike generic LLMs that serialize tool calls into fragile JSON markdown blocks, Inkling 2 Mini utilizes native
<|message_system|>tool_declare<|content_xml|>markers, ensuring 100% deterministic tool invocation without syntax hallucination.
Frontier Benchmark Context
Aggregated live from Hugging Face Hub (20,000+ likes) and Hacker News (1,226 points). Verified late-November 2026 weights checkpoint.
Precision Numerics: NVFP4 & MXFP8 Hardware Requirements
One of the standout breakthroughs of Inkling 2 Mini is its distribution format. It is packaged as a single ~20GB checkpoint in mixed NVFP4 (NVIDIA 4-bit Floating Point) and MXFP8 (Microscaling 8-bit Float) precisions.
Hardware Compatibility Matrix
| Hardware Platform | Precision Mode | VRAM Footprint | Throughput (Tokens/sec) | Feasibility |
|---|---|---|---|---|
| RTX 4090 / 5090 (24GB) | NVFP4 / MXFP8 | ~21.4 GB | 85 - 110 t/s | Optimal Local Production |
| RTX 3090 (24GB) | MXFP8 Fallback | ~22.8 GB | 55 - 70 t/s | Supported |
| Apple Silicon (M3/M4 Max 36GB+) | MLX / Metal FP8 | ~22.0 GB | 40 - 55 t/s | Supported via Metal |
| Cloud A100 / H100 (80GB) | FP8 / FP16 Native | ~21.0 GB | 180+ t/s | Enterprise Fleet |
Because the full model fits comfortably in under 22GB of VRAM, developers can run a dedicated, private reasoning agent on a single desktop GPU without renting expensive cloud clusters.
VRAM Allocation Tip
Ensure your CUDA memory allocation headroom is at least 2.5GB for KV cache when handling 32k+ context window workloads.
Local Serving with SGLang & Smoke Monkey Harness
To deploy Inkling 2 Mini locally with production-grade throughput, SGLang provides the optimal serving runtime:
# 1. Install SGLang
pip install --upgrade sglang
# 2. Launch high-throughput server with NVFP4/MXFP8 weights
python3 -m sglang.launch_server \
--model-path "thinkingmachines/Inkling-2-Mini" \
--host 0.0.0.0 \
--port 30000 \
--mem-fraction-static 0.85Once SGLang is active on port 30000, Smoke Monkey Harness connects to it through standard OpenAI-compatible endpoints while translating its native tool_declare stream into Smoke Monkey"s 2,500+ Model Context Protocol (MCP) server ecosystem.
# Production Docker launch command with GPU pass-throughdocker run --gpus all \--shm-size 32g \-p 30000:30000 \-v ~/.cache/huggingface:/root/.cache/huggingface \lmsysorg/sglang:latest \python3 -m sglang.launch_server \--model-path "thinkingmachines/Inkling-2-Mini" \--host 0.0.0.0 \--port 30000
Multi-Agent Spatial Orchestration on Smoke Monkey Canvas
While a single Inkling 2 Mini agent excels at tactical coding tasks, complex engineering initiatives require multiple specialized agents coordinating across distinct micro-workspaces.
On Smoke Monkey Canvas, you can place multiple Inkling 2 Mini agents on an infinite 2D spatial plane:
- Architect Agent (Effort: 0.95): Decomposes product specifications into modular TypeScript interface contracts.
- Refactor Agent (Effort: 0.70): Executes AST replacements and module decoupling with process-isolated safety gates.
- Verification Agent (Effort: 0.50): Runs test runners, checks coverage thresholds, and validates git diffs.
All state transitions, agent memories, and tool permissions are persisted 100% locally in SQLite with zero telemetry.
Frequently Asked Questions
Q:What is the parameter size and checkpoint footprint of Inkling 2 Mini?
Inkling 2 Mini has 26 billion dense parameters. Thanks to its mixed NVFP4 and MXFP8 precision format, the full distribution checkpoint is approximately 20GB, fitting into a single 24GB consumer GPU.
Q:How does Inkling 2 Mini handle tool calling compared to standard JSON schema?
Inkling 2 Mini uses a native <|message_system|>tool_declare<|content_xml|> protocol directly inside its tokenizer, eliminating syntax errors and escaping issues common with standard markdown JSON blocks.
Q:Can I run Inkling 2 Mini on an Apple Silicon Mac?
Yes. Apple Silicon Macs with unified memory (M2, M3, or M4 Max with 36GB+ RAM) can execute the model via llama.cpp, MLX, or LM Studio with smooth interactive token generation.
Q:How does the reasoning_effort parameter work in production agent loops?
The reasoning_effort parameter scales between 0.0 (immediate response, zero chain-of-thought) and 0.99 (deep recursive reflection). Smoke Monkey Harness allows setting this per-task so routine tasks run with sub-second latency while complex refactoring uses deep thinking.
Related Alternatives & Comparisons
Related Architecture Guides
View all guidesBest Open Source Coding Agents in 2026: Free, Local & Fully Hackable Harnesses
MCP Server Security Best Practices: Hardening Model Context Protocol Agents in 2026
Open Source Coding Agent Harness: Build a Forkable, Local AI Engineering Runtime
Build with Smoke Monkey Harness
Zero dependencies. 24 built-in tools. Human-in-the-loop safety. 100% open source under the MIT License.