MemT-1.0: Towards Memory-Native Evolution

What it claims

MemT-1.0 is a family of two memory foundation models from the MemTensor team, released October 2026. Each model stores past interactions in a fixed-size learned memory state instead of a growing KV cache.

• MemT-1.0-Nano has 36B parameters (3B activated) and 10M dynamic memory. It uses Qwen3.6-35B-A3B, as released, as its backbone.
• MemT-1.0-Flash has 288B parameters (17B activated) and 336M dynamic memory. Its backbone is DeepSeek-V4-Flash after further post-training.

The authors evaluate both models on 11 memory benchmarks. They also apply the models to coding agents that load retrieved procedural experience into native memory instead of the prompt.

How it works

Selected Transformer blocks get a native memory block. Its memory attention runs in parallel with the original attention, and the two outputs are summed. The backbone's pre-attention, attention and post-attention computations are left unchanged to keep the backbone's capabilities.

Each memory block has two parts:

• Local memory is a per-layer state of shape H_kv × d_k × d_v. It is set to zero at the start of each trajectory and updated online. Compared with Metis, the query-key normalization vector is removed and the state is split into multiple channels.
• Hyper memory holds the projection and gate parameters. These are trained offline and fixed during interaction.

Storage happens at the end of each turn. Keys (L2-normalized projections of the hidden states) and values are written token by token through a Gated DeltaNet-2 update, which uses erase, write and forget gates. This GDN-2 update replaces the token selection used in Metis. Appendix C describes a "symmetrization fix" that makes the update stable.

Retrieval projects an L2-normalized query and reads out from the memory matrix. The result passes through RMSNorm and a SiLU output gate with a soft cap. Mixture-of-Storage scales memory capacity, but the fetched text does not explain how it works.

flowchart LR
  H[Hidden states, turn t] --> OA[Original attention]
  H --> MA[Memory attention]
  M[(Memory state M_t)] --> MA
  OA --> S[Sum]
  MA --> S
  S --> Out[Next layers]
  H -->|turn end: GDN-2 gated write| M2[(M_t+1)]
  M2 -.next turn.-> M

Serving keeps the model parameters shared and the memory states separate for each trajectory. For a fixed memory configuration, the persistent state does not grow as interactions accumulate.

Evidence

Figure 3 gives end-to-end inference latency for Nano, full context, and Temp-LoRA:

At 128K this is the stated 1.86× speedup. At 256K the gap widens to about 4×.

Persistent storage figures from the same figure:

• Full-context KV cache: 5.061 GiB at 256K, and listed as "too large" at 1M.
• Temp-LoRA: 172.5 MiB at both 256K and 1M.
• Nano: 40.00 MiB at both 256K and 1M.

The memory benchmarks include LoCoMo, MemBench, MemoryAgentBench, PerMemSafe, ImplicitMemBench and PersonaMem.

• Parametric memory baselines: Doc-to-LoRA, SHINE, δ-mem and MemoryLLM.
• Textual memory baselines: MemOS, DenseRAG and Graphiti, each run on DeepSeek-V4-Flash-0731 and Qwen3.6-35B-A3B.

In the fetched text, the per-benchmark scores are garbled chart labels that cannot be matched to models. The stated result is only that MemT-1.0 beats the parametric baselines on average and is "competitive" with retrieval methods. General capabilities are "largely preserved" under a noise setting.

On SWE-bench Verified and SWE-bench Multilingual, internalized experience raises average task success and run-to-run consistency over agents with no experience. It performs about the same as giving the experience as explicit text. No numbers for this appear in the fetched text.

Figure 2 compares Flash's backbone, before native memory is added, with GLM-5.3 Flash, DeepSeek-V4 Pro, Kimi K3, GPT-5.6 Sol and Claude Opus 5 on agentic benchmarks. The extracted values cannot be reliably matched to models.

How to use it

The page links the code and the weights:

• Code: https://github.com/MemTensor/MemT-1.0
• Models: https://huggingface.co/collections/MemTensor/memt-1

The fetched text gives no install commands, API, licence or hardware requirements.

Limits

The fetched text stops at section 2.2.1. It does not include the Limitations section (section 8), the ablations, the scaling results, the training setup or any benchmark tables. Memory is written at turn end, so information from the current turn reaches memory only after that turn is complete. Retrieval baselines are described as competitive, not beaten. Check the licence on the repository or Hugging Face before adopting.