TileDRAM: Compiler-Controlled Tile-to-DRAM Layouts for 3D DRAM GPGPUs

What it claims

TileDRAM makes DRAM placement a compiler decision for GPUs whose compute units (CUs) each have a private 3D DRAM stack. It does this without changing the DRAM address mapping, the memory controller, or the ISA. The authors attribute the whole gain to the compiler-selected layout. On 108 simulated MoE FFN cases, conventional placement produces a 75.7% request-weighted row-conflict rate; TileDRAM lowers it to 15.2%, with a 1.77x geomean kernel speedup. Retargeting six traces to six 3D DRAM organizations cuts row conflicts by 79.2%, with a 2.20x geomean speedup.

How it works

The target is a modeled CU-local organization. Each CU reaches its nearby stack through a local memory controller, using a software-managed aperture in a dedicated device-physical address space. Requests to this aperture skip virtual-address translation, warp-level coalescing and caching. All other addresses use the normal memory hierarchy. Because nothing filters the traffic, row-buffer locality of the raw request stream decides effective bandwidth.

Layouts are F2-linear maps that extend GPU Linear Layouts (as used in Triton) from threads and registers to DRAM placement coordinates. The unit of specialization is a "tile access signature", which combines three things:

• tile configuration (shape, staging, warps)
• operand properties (lhs/rhs, dtype, stride, row/column major)
• the ordered tile-local access sequence

Tiles at different origins, or from experts with different routed-token counts, share a signature and therefore share a layout. The mapping is stateless, but row hits depend on request history. An offline solver therefore simulates row-buffer state transitions for each candidate under each signature's access order, then picks the layout with the fewest conflicts. The compiler applies the chosen layout during address generation, and the runtime supplies each tile's placement. Retargeting to a new DRAM organization reruns layout selection only.

The authors argue against controller-side remapping. One global mapping must serve both the A and B operand streams of the same GEMM, which have different address-difference distributions. Per-pattern controller mappings would need multiple mapping contexts.

Evidence

• Workload and traces: NVBit traces of Triton/Gluon grouped-GEMM kernels, replayed in Accel-Sim with Ramulator2.
• Arithmetic intensity: the suite spans 55.4 to 123.0 FLOP/B ideal intensity. That is below conventional GDDR/HBM ridge points (153 to 295 FLOP/B) but above the local-memory ridge reference the authors derive (32.6 to 52.0 FLOP/B). This derivation uses Stratum's 19.01 to 30.34 TB/s in-stack bandwidth and H100 compute.
• Bandwidth motivation: the package-level endpoint (about 0.8 to 1.2 TB/s per HBM3/HBM3E stack) is cited as 16 to 38x narrower than the modeled internal path.
• Real hardware: an NVIDIA RTX 3090 was used only to measure the cost of address transformation, 0.17% geomean over identity across 108 kernel configurations.

Limits

• All speedups come from simulation of a modeled architecture, not real 3D DRAM hardware.
• The analysis covers one CU-stack pair at a time.
• How runtime scheduling interleaves CTAs on a CU is treated separately from each tile's local access pattern. Layouts are chosen per tile, not for the interleaved stream.
• Evaluation is limited to MoE FFN grouped GEMMs.
• The fetched text stops partway through Section 2.3. Solver details, ablations and the authors' stated limitations are not available here.
• The text does not mention a code release or licence.