Can You Self-Host Kimi K3? 1.4–2.8TB Weight Memory and GPU Rental Planning

Decision first: Kimi K3 is not a practical single-workstation model. Moonshot reports 2.8 trillion total parameters and plans to release the full weights by July 27, 2026; at this July 18 snapshot, the API is live but the weights are not yet available for a verified self-hosting test. A 4-bit weight-only estimate is roughly 1.4TB, while FP8 is roughly 2.8TB, before KV cache, runtime buffers, batching, and redundancy. Most teams should test the official API now. Infrastructure teams can use the GPU counts and ComputeUnion rental curves below for preliminary capacity planning, then revise the design after the weights, quantization artifacts, license, and supported inference stack are published.

The memory calculation: why 2.8T parameters changes the decision

The lower-bound formula is parameter count multiplied by bytes per stored weight. It is useful for rejecting impossible hardware plans, but it is not a complete deployment bill.

Weight format	Lower-bound calculation	Weight memory only	Decision use
4-bit	2.8T × 0.5 byte	about 1.4TB	Optimistic quantized planning floor
FP8 / 8-bit	2.8T × 1 byte	about 2.8TB	Higher-quality serving baseline
FP16 / BF16	2.8T × 2 bytes	about 5.6TB	Usually unrealistic for a cost-sensitive deployment

K3 is a mixture-of-experts model and activates 16 of 896 experts per token, which reduces active compute. It does not mean only the active experts need to be stored. Unless the released runtime supports a different placement strategy, the cluster must still plan for the full weight set.

Preliminary GPU count by memory capacity

GPU configuration	Memory per GPU	4-bit mathematical minimum	Preliminary planning range
NVIDIA H100 SXM 80GB	80GB	18	24+
NVIDIA H200 SXM 141GB	141GB	10	12–16
NVIDIA B200 SXM 180GB	180GB	8	10–16
AMD MI300X 192GB	192GB	8	10–12

The “mathematical minimum” fits only the estimated 4-bit weights. The planning range reserves room for serving overhead but is still not a tested K3 recipe. Million-token context, concurrent requests, expert routing, tensor/expert parallelism, communication buffers, and framework support can all move the number.

Use ComputeUnion price curves, not one hourly quote

GPU rental prices change by platform, region, availability, spot policy, and interconnect. The chart above reads ComputeUnion's recent daily on-demand lows for the four candidate GPU configurations. Use it to answer two questions: whether a capacity plan is available at all, and whether a temporary cluster is cheaper than committing to owned hardware. Open each GPU page for provider-level quotes and the longer price history.

• H100: widest ecosystem support, but more cards increase interconnect and coordination overhead.
• H200: more memory per GPU reduces the theoretical card count while staying in the NVIDIA software ecosystem.
• B200: high-memory current-generation option; validate regional inventory and multi-node topology before budgeting.
• MI300X: strong memory density; validate that the released K3 inference stack supports the required AMD runtime path.

What the cluster needs beyond GPU memory

1. High-bandwidth interconnect: multiple accelerators need fast collective communication; ordinary PCIe-only multi-node layouts can erase the compute advantage.
2. Host memory and storage: plan for weight staging, conversion, checkpoints, and at least one recoverable copy. Multi-terabyte NVMe capacity is a starting requirement, not an optimization.
3. Serving software: wait for confirmed support in the chosen inference framework and for an official or auditable quantization artifact.
4. Operational headroom: leave capacity for KV cache, batching, health checks, node failure, and rolling upgrades.
5. Validation workloads: benchmark your actual coding, agent, multimodal, and long-context tasks instead of relying on a peak throughput claim.

API or self-hosting: which should you buy now?

Situation	Recommended next step	Why
Evaluating K3 quality	Use the official API first	No cluster commitment; measure accepted-task cost now
Variable or low utilization	Keep API or short-term rental	A large idle cluster can cost more than token usage
High, stable, sensitive workload	Prepare a post-release proof of concept	Self-hosting may improve control only after runtime and utilization are known
Consumer workstation deployment	Do not plan for the full model	24GB or 48GB cards are far below the weight-memory floor

Clear conclusion

Buy API access for immediate evaluation; rent a short-lived multi-GPU cluster only after the released weights and serving stack can be tested. For full K3, the first question is not “which single GPU?” but “which 10–24+ accelerator topology can hold the model with enough runtime headroom?” ComputeUnion's advantage is connecting that capacity calculation to live rental markets, price history, provider availability, and the existing K3 API price page.

Sources and method

K3 parameter count, MoE activation, context, availability, numerical formats, and the July 27 weight-release plan come from the official Kimi K3 launch page. Hardware memory specifications come from the NVIDIA H100, NVIDIA H200, and AMD Instinct MI300X product pages; the B200 row uses the 180GB configuration tracked in ComputeUnion. GPU price curves and provider links come from ComputeUnion's own rental-price observations. Card-count ranges are capacity-planning estimates, not vendor-certified K3 deployment requirements.