New SAM 3, Gemma 4 and GLM-5 have landed

Run any model,
on any backend

118 model families ported to pure Keras 3, with weights converted from the original checkpoints. The same code runs on JAX, PyTorch and TensorFlow, and nothing from transformers or torch is needed at run time.

Get started GitHub

Object detection DETR End-to-end, no anchors or NMS

SegFormer segmentation overlaid on a street corner at dusk

Segmentation SegFormer 13 variants, ADE20K and Cityscapes

Depth Anything V2 depth map of a tennis match

Depth estimation Depth Anything V2 Relative and metric depth

SAM 3 masks on two elephants, from the text prompt 'elephant'

Promptable masks SAM 3 Points, boxes and concepts

OWLv2 detecting a real bear from the prompt 'a photo of a real bear'

Open-vocabulary detection OWLv2 Text prompts, no fixed class list

whisper_base · 12.48 s clip He tells us that at this festive season of the year, with Christmas and roast beef looming before us…

Speech recognition Whisper Transcribe or translate

Two calls to a prediction¶

Build the model with from_weights, then feed it whatever its processor produces. Every model in the library follows this shape, so moving between a detector, a depth estimator and an LLM costs you nothing.

pip install -U zeromodels

Weights come from the zeromodels org on the Hub, and the same identifier builds both the model and its processor, so the resolution and normalization always match the checkpoint.

import os

os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.detr import DETRDetect, DETRImageProcessor

model = DETRDetect.from_weights("zeromodels/detr-resnet-50")
processor = DETRImageProcessor.from_weights("zeromodels/detr-resnet-50")

image = Image.open("photo.jpg").convert("RGB")
output = model(processor(image)["pixel_values"], training=False)

results = processor.post_process_object_detection(
    output, threshold=0.9, target_sizes=[(image.height, image.width)]
)[0]

One call, three sources¶

from_weights dispatches on what you hand it: a preconverted Keras repo on the Hub, a bare variant name that converts an upstream checkpoint on the fly, or any compatible Hugging Face repo behind the hf: prefix. Architecture details, including the class count of a fine-tune, are read from the repo config.

Loading weights · Main classes

from zeromodels.models.qwen3 import Qwen3TextGenerate
from zeromodels.models.segformer import SegFormerSemanticSegment

# Preconverted Keras weights (kf_config.json)
SegFormerSemanticSegment.from_weights("zeromodels/segformer_b0_ade_512")

# Bare variant: converted from upstream on the fly
Qwen3TextGenerate.from_weights("qwen3-8b")

# Any Hub repo with a matching model_type
SegFormerSemanticSegment.from_weights("hf:nvidia/segformer-b0-finetuned-ade-512-512")

# Architecture only, randomly initialized
SegFormerSemanticSegment.from_weights(
    "zeromodels/segformer_b0_ade_512", load_weights=False
)

Measured outputs, not illustrative ones¶

Every figure and every printed result on a model page comes from actually running the snippet beside it on the image or audio clip shown. Nothing is hand-written to look plausible, so what you read is what you get when you run it yourself.

Browse the model pages

SegFormer B5 on an open-plan kitchen and a herd in a field

Any backend, either data format¶

Set KERAS_BACKEND before importing Keras and the rest is unchanged. Models read keras.config.image_data_format() when they are constructed, so set that first too if you want channels_first; processors take a per-instance data_format argument.

Configuration · Utilities

import os

os.environ["KERAS_BACKEND"] = "jax"  # or "torch" / "tensorflow"

import keras

keras.config.set_image_data_format("channels_first")

Large checkpoints, as they ship¶

GPT-OSS 120B loads at bfloat16 with its MoE experts left packed in MXFP4 and dequantized on the fly, so it stays near 66 GB instead of the ~130 GB an fp32 expansion would cost.
Weight-only int8, int4, fp8 and mxfp4 are arguments to the same from_weights call, on any model.

Quantization · int8 · int4 · fp8 · mxfp4

from zeromodels.models.gpt_oss import GptOssTextGenerate

# Experts stay packed in MXFP4, dequantized in the expert layer's call
model = GptOssTextGenerate.from_weights("zeromodels/gpt-oss-120b")

# Quantize weight-only on the way in, for a smaller footprint again
model = GptOssTextGenerate.from_weights(
    "zeromodels/gpt-oss-120b", quantization="int8"
)