Stacking DRAM on top of the compute: Shenzhen's MetaChip lights up an edge AI coprocessor that runs Qwen3-8B at ~80 tok/s on 5 watts, but volume production waits until Q1 2027
For two years the story of running large models on-device has been stuck on a plain problem: the model fits, but the tokens come out too slowly. The limit is not compute — it is distance. Model weights live in memory, and generating each token means reading billions of parameters again, so the channel between memory and the compute units becomes the bottleneck. The industry calls it the memory wall.
Shenzhen-based MetaChip has an answer: stop putting memory at the far end of the board and stack it directly on top of the compute. On November 18, on the eve of the ICCAD-Expo 2026 show, the company held a launch event in Beijing's Yizhuang district for its MEGA188 chip — "lighting up," in semiconductor terms, means the silicon came back from the fab and ran correctly for the first time. MEGA188 pairs 3D DRAM with a distributed TPU in a stacked package. MetaChip quotes a 600 GB/s peak memory bandwidth, 5-watt-class power, and roughly 80 tokens per second running Qwen3-8B.
One caveat up front: every performance figure here comes from the company, either at the launch event or in its own published material. No independent third party has measured the chip. This article is compiled from multiple media reports, including MetaChip's official release, an in-depth interview published by ZhiDongXi/ChipInside, and AI123's summary of IT Home's coverage.
TL;DR: the short version
- What it is: an edge inference coprocessor — not a GPU, and not a conventional NPU. It has no host controller of its own; it attaches to an existing system over M.2 or PCIe and handles model inference plus weight residency.
- The technical bet: 3D stacking with hybrid bonding and TSVs, placing DRAM vertically on the logic die so the memory-to-compute path shrinks from board level to inside the package.
- Bandwidth and power as quoted: 600 GB/s peak bandwidth, a 5-watt low-power mode; the company separately cites roughly 10W for the whole chip, running stably with passive, fanless cooling.
- Performance as quoted: close to 80 tok/s on Qwen3-8B with sub-200 ms time to first token; about 293 tok/s on MiniCPM5-2B from ModelBest.
- Process: domestic Chinese 28nm, die area around 100 mm². The approach deliberately sidesteps advanced nodes and HBM.
- Availability: volume production is planned for Q1 2027. The chip is currently in sample delivery and engineering validation.
The real edge bottleneck is bandwidth, not compute
Start with the memory wall. A language model emits text one token at a time, and every token requires pulling the weights for the active layers out of memory and through the arithmetic units. However fast the compute, if the data cannot arrive quickly enough, it sits idle. What sets token throughput is how many bytes of model data can reach the compute units per second — not how many multiply-accumulates they can perform.
MetaChip offered a rule of thumb at the launch: getting a gigabyte-scale model to roughly 100 tok/s on a device requires about 600 GB/s of bandwidth. That is roughly desktop discrete-GPU memory bandwidth, a budget that laptops, AI boxes, and robots simply cannot support in power and thermal terms.
Conventional 2D designs connect logic and memory through package interfaces. Raising bandwidth that way means more interfaces and more surrounding circuitry, which pushes up both power and cost. 3D stacking takes a different route: place the memory on the logic die, drill through-silicon vias, and bond the two metal layers directly with hybrid bonding, so data never has to leave the chip.
The price is engineering difficulty. MetaChip founder Yu Hao told ZhiDongXi that hybrid bonding and TSVs both consume die area and complicate the place-and-route of the logic, requiring several rounds of physical-design iteration. MEGA188 keeps the die around 100 mm² with average power of 5–10W, which makes thermals, warpage, and yield comparatively manageable — and that is precisely why 28nm is viable: skip the leading edge and win on architecture instead.
How the compute side keeps up: a distributed systolic TPU
High bandwidth alone is not enough; the compute array has to be able to consume it. MEGA188 uses a homegrown "3D distributed TPU" architecture: processing elements connect directly to memory over 3D I/O in a meshed distributed-access structure, data is partitioned across the elements for parallel work, and results are summed.
The compute itself is a systolic tensor array that reuses data to avoid redundant reads — the same operand feeds many operations instead of being fetched repeatedly. For the matrix-multiply and matrix-vector work that dominates Transformers, this dataflow keeps bandwidth and throughput better matched.
MetaChip says the distributed systolic architecture achieves about 80% bandwidth utilization. Against a 600 GB/s peak, that works out to roughly 480 GB/s effective. Treat that as an arithmetic projection, not a measured figure.
The benchmark numbers, and where they stop
The figures below come from the launch event and the company's website. All are vendor-reported:
- Qwen3-8B: close to 80 tok/s in high-performance mode; about 26 tok/s in low-power mode; time to first token under 200 ms.
- MiniCPM5-2B (ModelBest): about 293 tok/s in high-performance mode; about 92 tok/s in low-power mode.
- Qwen3-VL-2B (vision-language): about 290 tok/s high-performance; about 32 tok/s low-power.
- Qwen3-30B-A3B (mixture-of-experts): about 19 tok/s on a single card; about 80 tok/s with multiple cards in parallel.
The boundaries matter. The company has not disclosed the host platform, memory configuration, batch size, or precision settings used for these runs, and no third party has reproduced them. For context, in 2024 MetaChip ran a 6-billion-parameter model at 70–80 tok/s on an FPGA platform rated at roughly 1 TOPS, with about 80% architecture utilization — so the 80 tok/s range is not out of nowhere, but FPGA validation and shipping ASIC silicon are different things.
The MoE result deserves its own note: about 19 tok/s for a 30B-class MoE on one card, rising to about 80 tok/s only with multiple cards. That tells you what "runs 3B to 120B models" actually means at the large end — it leans on multiple cards and external storage, not a single accelerator carrying the load.
It does not replace the host — it plugs into it
This is the most practical part of the product definition. Edge devices already have capable host processors — x86 or Arm, with operating systems and application ecosystems. MetaChip is not trying to replace any of that. MEGA188 attaches over standard high-speed interfaces such as M.2 and PCIe: the host handles system scheduling and sensor input, while MEGA188 concentrates on inference and keeping weights resident.
For existing hardware, that means an add-in card upgrade path without redesigning the motherboard. For the next generation of PCs and laptops, one or more MEGA188 dies can be integrated at board level. A PCIe switch extends a single card to two or four, scaling compute and bandwidth with the device tier.
There is also an SSD pairing. The host SSD stores the full set of MoE expert weights while 3D DRAM holds only the experts activated per layer; when a needed expert is missing from DRAM, the system reads it from the SSD and loads it before computing. DRAM acts as a high-bandwidth cache and the SSD supplies capacity, so the full parameter set never has to fit in expensive on-package storage. MetaChip says this single-card arrangement already runs 30B-class MoE models, while multi-card and distributed-SSD configurations for 120B and larger are still being validated.
The progress is real, but it stops at "lit up"
The timeline is worth laying out plainly:
- October 14, 2026: silicon returned from the fab; SoC bring-up completed in about half an hour that day.
- October 19, 2026: PCIe link training succeeded; the host could enumerate and communicate with the chip.
- Late October 2026: 3D DRAM read/write verified within the tested range; core operators and numerical accuracy passed.
- November 2, 2026: MEGA188 received a 2026 "China Chip" Rising Star product award (background).
- November 18, 2026: launch event in Beijing's Yizhuang district, first public model benchmarks; a strategic cooperation agreement signed with Beijing Bodawangtong.
- November 19, 2026, 18:00: official milestone post published; the same day ZhiDongXi ran its in-depth interview.
- Q1 2027: the company's stated target for volume production.
Be precise about what "lit up" means in this industry. It proves the chip powers on, communicates, computes, and reads and writes its stacked memory — a critical milestone after tape-out. It does not mean volume production, and it does not mean customers can reliably get parts. Reliability qualification, toolchain maturity, host adaptation, and yield ramp all still sit in between.
What is known, and what is not
Established:
- Architecture (3D DRAM plus a distributed systolic TPU), process (domestic 28nm), die area (~100 mm²), power (5–10W average, with a separate "about 10W whole chip" figure), memory options (2/4/8 GB), peak bandwidth (600 GB/s), and compute (10 TOPS at FP16/INT4).
- Interface (M.2/PCIe, scalable to four cards) and the SSD-paired MoE arrangement.
- Target deployments as described by the company: embodied intelligence, smart education, industrial edge computing, and automotive electronics.
- A customer-side cost figure: Yu Hao says adding roughly 1,000–2,000 yuan to a device's bill of materials is enough to give it local agent capability.
Not disclosed:
- Unit pricing, final single-card power specifications, production yield.
- Customer names and shipment volumes. The company says only that first customers have been onboarded; no named device maker has confirmed.
- Independent benchmarks, or full test conditions.
- Software stack maturity. The inference engine has to cover LLMs, VLMs, MoE, and VLA models while supporting distributed parallelism — work that is routinely underestimated.
Who it's for: good fit and bad fit
- Hardware teams building AI boxes, mini PCs, or NAS units: an M.2 or PCIe add-in card means no motherboard redesign, so small-batch validation is feasible.
- Product owners defining AI PCs and laptops: if 1,000–2,000 yuan of extra BOM buys local 8B-class inference, that is a differentiator you can put in a spreadsheet.
- Embodied-AI and industrial edge vendors: local inference, low power, and fanless operation matter more in these settings than peak TOPS.
- Anyone tracking China's domestic supply chain: 28nm domestic process, a RISC-V control core, and standard interfaces form a clear route around advanced nodes and HBM, and it is worth watching as a case study.
Where it does not fit
- Frontier model training: this is an inference coprocessor; training is out of scope.
- Running 70B+ dense models on one card: 8 GB of on-package storage will not hold them; that requires multi-card and SSD configurations that are still under validation.
- Projects that need shipping volume now: production is planned for Q1 2027, and there is no inventory today.
- Workloads tied to CUDA: this is a proprietary architecture with its own toolchain, so migration cost is yours to assess.
- Buyers who require third-party performance validation: every number so far is vendor self-reported.
How to use it: where to start
- First step: do the bandwidth math, not the TOPS math. Take your model's parameter count and target tok/s and work backward to the effective bandwidth required. An 8B model at 80 tok/s lands in the 500 GB/s range — that is what determines whether 3D stacking is even necessary.
- Confirm host and interface compatibility. M.2 lane count and PCIe generation directly cap the bandwidth you can actually reach. This is where selection most often goes wrong.
- Ask for benchmark reports with test conditions attached. Specifically: which host platform, what precision, what batch size, whether the figure is time to first token or steady-state tok/s, and whether it throttles after 30 minutes.
- Evaluate the toolchain as part of the chip. A chip that runs a model is not the same as a chip that runs your model — operator coverage, quantization support, and the compilation flow determine how long integration really takes.
- Start with a small batch. Get one or two cards for end-to-end validation before anything goes into a production bill of materials.
Six things easy to misread
- "Lit up" is not "in production." It shows the chip works; reliability, yield, and toolchain are still ahead.
- 80 tok/s is Qwen3-8B in high-performance mode. Low-power mode is about 26 tok/s. Do not mix the two when sizing a design.
- 600 GB/s is peak, not effective. At the stated 80% utilization that is about 480 GB/s, and real workloads take another discount.
- The "120B" in "runs 3B to 120B" depends on multiple cards and SSDs. A single card with 8 GB of on-package memory cannot hold it.
- 5W and 10W are different figures. 5W describes the low-power mode; "about 10W whole chip" and "5–10W average" also appear. Design thermals against the most conservative number.
- The 1,000–2,000 yuan figure is added system cost, not chip price. No unit price has been published.
Cost and timing expectations
- Timing: volume production is planned for Q1 2027. Going from bring-up to volume typically takes another two to four quarters of reliability and yield work — that range is industry convention, not a company commitment.
- Cost: the only public figure is the customer-side estimate of roughly 1,000–2,000 yuan added per device. Working back from that, a coprocessor plus power delivery, cooling, and PCB changes plausibly lands in that band, but chip price, sample pricing, and NRE are all undisclosed.
- Hidden cost: software adaptation. The inference engine must cover LLM, VLM, MoE, and VLA workloads with distributed parallelism; if your model uses custom operators, budget a separate integration schedule.
The bottom line
MEGA188's significance is not that it is faster than something else. It is that it demonstrates a route to high edge bandwidth without advanced process nodes or HBM: stack memory on top of the compute, hold costs down with 28nm, and sidestep host replacement by shipping as a coprocessor. It has now cleared the "it runs" gate. The gates still ahead are "it ships in volume" and "someone will pay for it" — and the company's own date for those is the first quarter of 2027.