UnverifiedRTX 4070 12GB (Ada sm_89) + RTX 5070 12GB (Blackwell sm_120), layer-split via llama.cpp on Ryzen 5800XT Zen 3, 128 GB DDR4, Docker Desktop WSL2CUDA

bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF :: nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf Q4_K_M @ RTX 4070 12GB (Ada sm_89) + RTX 5070 12GB (Blackwell sm_120), layer-split via llama.cpp on Ryzen 5800XT Zen 3, 128 GB DDR4, Docker Desktop WSL2

Name: bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF :: nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf (Q4_K_M) on RTX 4070 12GB (Ada sm_89) + RTX 5070 12GB (Blackwell sm_120), layer-split via llama.cpp on Ryzen 5800XT Zen 3, 128 GB DDR4, Docker Desktop WSL2 — inference benchmark
Creator: gotcontext.ai
License: https://creativecommons.org/licenses/by/4.0/

Submitted May 2026https://gotcontext.ai/benchmarks/runs/_JfuOCvvVlA

Decode

10.5tok/s

PP speed (tok/s)

—

Peak VRAM

8.7GB

Context

64ktokens

Batch

4096parallel

Top 63% for this config16 runs

Flash Attn

Raw benchmark output

llama-server --no-mmap -ngl 999 --cpu-moe --batch-size 4096 --ubatch-size 4096 --tensor-split 0.55,0.45 --parallel 1

Not enough data yet to compare this combination. Submit more runs for this hardware + model to see a comparison.

Build diagnostics

Your build is within the typical range for this hardware. No obvious configuration issues detected.

Reproducibility recipe

Copy this to repeat or submit a similar build:

model: "bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF :: nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf"
quant: "Q4_K_M"
hardware: "RTX 4070 12GB (Ada sm_89) + RTX 5070 12GB (Blackwell sm_120), layer-split via llama.cpp on Ryzen 5800XT Zen 3, 128 GB DDR4, Docker Desktop WSL2"
context_length: 65536
batch_size: 4096

Notes from submitter

Official seed run — gotcontext.ai lab rig (RTX 4070 Ada + RTX 5070 Blackwell, layer-split, Ryzen 5800XT, 128GB). Measured range 9-12 tok/s; reported value is the midpoint. Decode 11.95 tok/s, VRAM 4.4 + 4.5 = 8.9 GB at --cpu-moe sweet spot. Hybrid Mamba+Attention+MoE: 23 of 52 layers use recurrent KV (constant memory per token, doesn't grow with context — native 1M ctx feasible). All partial-offload attempts (--n-cpu-moe 15/18/20) FAILED with cudaMalloc compute buffer errors. Mamba SSM state + 6 GQA layers always on GPU regardless of --n-cpu-moe. Tool calling uses XML <tool_call><function=...> inside <tools> declarations — NOT OpenAI tools API format. First load 24.7 GB read into DRAM.

Discussion

No questions yet. Ask the first one.

See all bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF :: nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf benchmarks Submit your build

Those are our numbers. Get yours.

Paste one of your own docs below and see your real compression ratio live — free, no signup.

Your text

# Service Operations Runbook: Payments API

## Purpose and scope

This runbook covers the payments-api service: what it does, how it is deployed, what its dependencies are, and what to do when it misbehaves. It is written for the on-call engineer. Every procedure here assumes you have production read access and the ability to trigger a deploy through the standard pipeline. Nothing in this document requires direct database write access, and no procedure here should be improvised under pressure: if the situation is not covered, page the service owner rather than inventing a fix at 3am.

The payments-api accepts charge requests from the checkout frontend, validates them against the pricing catalog, forwards them to the payment processor, and records the outcome in the orders database. It is the only service permitted to talk to the processor. Average traffic is steady during business hours with a daily peak around 19:00 UTC and a weekly peak on Friday evenings.

## Architecture and dependencies

The service runs as three replicas behind the regional load balancer. Each replica is stateless; all persistent state lives in the orders database and the idempotency-key store. The service depends on four things: the orders database (primary and one read replica), the idempotency-key store, the pricing catalog service, and the external payment processor. Of these, only the processor is outside our control.

Dependency failure behavior is deliberate and asymmetric. If the pricing catalog is unreachable, the service serves prices from its local cache for up to ten minutes and emits a degraded-mode metric. If the idempotency store is unreachable, the service refuses new charges entirely, because accepting a charge without idempotency protection risks double-billing, and double-billing is strictly worse than downtime. If the processor times out, the charge is recorded as pending and a reconciliation job resolves it within the hour.

## Deployment

Deploys go through the standard pipeline: merge to main, automated tests, staging deploy, a thirty-minute soak with synthetic checkout traffic, then production rollout one replica at a time. The pipeline aborts automatically if the error rate on the new replica exceeds the old baseline. A full rollout takes about twenty minutes. Rollback is the same pipeline in reverse and takes about six minutes; the on-call engineer can trigger it without approval.

## Monitoring and alerts

Three alerts page the on-call engineer. High charge failure rate fires when more than two percent of charge attempts fail over five minutes; the usual causes are a processor incident or a bad deploy, in that order. Idempotency store unavailable fires immediately on connection failure. Reconciliation backlog fires when pending charges older than ninety minutes accumulate, which usually means the reconciliation job is stuck rather than the processor being slow.

2,912/12,000 chars

Compressed

Compressed text will appear here…