Meta Releases the Weights of Its 816-Billion-Parameter Muse Spark Open Model

Meta made the weights of the Muse Spark Open model publicly available on October 15, 2026. According to the model card published by Meta Superintelligence Labs, the model has 816 billion total parameters and is natively multimodal: it accepts text, image, video, and audio inputs and generates text. The context window is 1 million tokens, and the number of active parameters is 42 billion. Meta describes the model as the most capable US open-weight model released to date, highlighting coding and long-horizon agentic tasks.

Architecture

Muse Spark Open consists of a Mixture-of-Experts Transformer and a Perception Encoder pair. The language model has 62 layers and a hidden size of 8192; there are 256 experts in total, and eight of them are activated for each token via top-k routing. The attention block uses gated attention with QK-norm: in a grouped-query configuration, there are 128 query heads and 16 key-value heads, with a head size of 64. The vocabulary consists of 202,048 tokens, and the input and output embeddings are untied. The Perception Encoder has 1.8 billion parameters across 50 layers; it uses a 14×14 patch size, and in sparse attention, every fourth layer and the final layer switch to full attention.

Reasoning budget and recommended settings

The model uses the reasoning_effort parameter to control how much it thinks before answering. The options are low, medium, high, and xhigh; the default is xhigh, and any other value causes an error. Meta recommends temperature 1.0, top_p 1.0, and top_k disabled for most setups; high or xhigh should be preferred for complex problem solving, coding, and agentic tasks. The training data consists of publicly available sources, web crawls, third-party data, Meta’s own products and services, and synthetic generation; it covers more than 100 languages.

Where to access it

The weights can be downloaded through Amazon SageMaker AI, Google Cloud Model Garden, Hugging Face, and Microsoft Foundry Models; on the API side, Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter, and Together AI are listed. For inference, SGLang, vLLM, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp, and Unsloth are supported; there are optimizations for AMD, Apple, Intel, and NVIDIA hardware, and PyTorch TorchTitan can be used for fine-tuning. The weights come in BF16 and NVFP4 versions.

Running it locally is still a heavy lift

For those who want to run the model on their own hardware, the threshold is high: the NVFP4 version requires at least 700 GB, and the BF16 version requires 2 TB of VRAM. According to Meta, Muse Spark Open is the first model in its class that can run on a desktop with two Apple M5 Ultra chips. In other words, open weights do not mean everyone will run it on their own computer; in practice, the first stop will still be cloud and API providers.

Source: Unite.AI