Skip to content

Back to the catalog

Samuel Zeng (@SamuelZengML) on X
x.comthread

Samuel Zeng (@SamuelZengML) on X

Samuel Zeng announces the open-sourcing of Edge0's cross-platform app layer (iOS, macOS, Android, Windows), building on Edge0's earlier on-device 35B language model that runs in only 1-2.5GB of peak memory. The release includes app code, local model/runtime integration, model management, and build/deployment guides on GitHub; the technique keeps the full model on local storage and streams in only the experts (MoE-style) needed per step, predicting and prefetching the next experts while running storage reads in parallel with compute. A Q4 2026 roadmap promises a unified inference API across platforms plus Python, hardware-adaptive runtime, and a CUDA backend.

TL;DR Edge0 open-sources its cross-platform app layer for running a 35B model on-device in 1-2.5GB of memory via on-demand expert loading and prefetching.
Open original ↗
Document

What it claims

Edge0 has open-sourced its app layer: one fully on-device app built for iOS, macOS, Android and Windows. The release follows Edge0's first version, published three weeks before this post, which the authors describe as "a 35B language model running on a local device, using only 1–2.5 GB of peak memory." The authors frame the change as a move from an on-device model to an on-device AI application. They say developers can build local assistants, creative tools and smart-hardware experiences on it.

How it works

The full model stays in local storage, and each step loads only what it needs:

  1. The experts for the current step are loaded into memory on demand.
  2. The experts for the next step are predicted and prepared in advance.
  3. Storage reads run in parallel with model computation.
  4. Memory is reused. The post is truncated at this point, so the reuse scheme is not described.

How to use it

The repository is at github.com/Edge0-AI/Edge0. According to the post, it contains:

  • the app code for all four platforms
  • the integration between the local model and its runtime
  • model management
  • build and deployment guides

The post gives no install commands, device requirements or model file names. You have to get those from the repo itself.

Limits

  • No performance evidence. The post gives no tokens/sec, latency or quality figures, and no list of tested devices. The 1–2.5 GB figure is peak memory only, with no throughput attached. Streaming experts from storage makes speed depend on storage, and the post does not measure that.
  • Model not identified. The 35B model is not named, and the post does not say which MoE architecture it uses.
  • No licence stated. The post does not give a licence for either the app code or the model weights. Check the repo before building on it.
  • Truncated text. Several parts of the thread are cut off at "Show more": the memory-reuse step, the full feature list and the final roadmap item (support for "next-generation model…").
  • Roadmap features not in this release. The following are announced for Q4 2026 and are not part of what was released:
  • the unified cross-platform and Python API
  • hardware-adaptive runtime
  • the CUDA backend

Organisations named here

  • Edge0-AI
  • Samuel Zeng

Technologies named here

  • Python
  • Edge0 app
  • Edge0
  • Edge0 unified inference framework
  • CUDA

Techniques named here

  • On-demand expert loading with predictive prefetch

Also named here

  • Mixture of Experts

Related in inference