litert-community/SigLIP2-base-patch16-224

SigLIP 2 (ViT-B/16, 224) — LiteRT (TFLite) GPU

On-device LiteRT (.tflite) conversion of SigLIP 2 (Google 2025), a state-of-the-art CLIP-style image tower, converted from timm/vit_base_patch16_siglip_224.v2_webli (ViT-B/16, 93M params; the image tower of ViT-B-16-SigLIP2 / google/siglip2).

A single forward pass turns one RGB image into a 768-d L2-normalized image embedding for zero-shot classification, retrieval, and similarity — running fully on the LiteRT CompiledModel GPU accelerator (ML Drift): all ops are GPU-native (Replacing 809 out of 809 node(s) … LITERT_CL), no CPU fallback, no Flex ops, and the GPU output matches PyTorch (corr ≈ 1.0).

Files

File	Size	Description
siglip2_base_224_fp16.tflite	185 MB	FP16 single-graph model, GPU full-residency
convert_siglip2.py	—	Reproducible conversion script (timm → tflite)

I/O

• Input: [1, 3, 224, 224] float32, NCHW, RGB normalized to [-1, 1] ((pixel/255 - 0.5) / 0.5). Normalization is applied by the caller.
• Output: [1, 768] float32, L2-normalized image embedding.

For zero-shot classification, precompute text-label embeddings with the SigLIP 2 text tower (open_clip ViT-B-16-SigLIP2, prompt "This is a photo of {label}.") and take the dot product on device.

Usage (Android, LiteRT CompiledModel)

val model = CompiledModel.create(
    context.assets, "siglip2_base_224_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null
)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nchwFloatArray)        // [1,3,224,224], RGB scaled to [-1,1]
model.run(inputs, outputs)
val embedding = outputs[0].readFloat()      // [768], already L2-normalized

Performance

• ~60 ms / image steady-state on a Pixel 8a (Mali-G615) GPU (best ~9 ms), full GPU residency, FP16.

Conversion notes

Converted with litert-torch / ai-edge-torch. Making the ViT image tower run fully on the GPU delegate and produce correct output on device required three verbatim (weights-exact, corr ≈ 1.0) model-side rewrites (full GPU residency does not imply a correct result):

1. Fused-qkv → 4-D manual attention — the fused qkv reshape emits a 5-D head-split the delegate rejects; decompose into separate q/k/v. Self-attention uses scaled_dot_product_attention, whose lowering keeps the batch-matmul 3-D with a materialized transpose (both required for residency).
2. Attention-pool single-query attention → broadcast-multiply + reduce-sum — the pooling query is a constant latent, so a batch-matmul there is const @ non-const (rejected / mis-computed); express as (q·k).sum + softmax
   • (attn·v).sum.
3. Overflow-safe LayerNorm — the delegate computes the LayerNorm variance reduction in fp16 even for an fp32 graph; deep-ViT massive activations make sum((x-mean)²) exceed the fp16 max (65504), corrupting normalization (output correlation collapses with depth while still reporting full residency). Scaling by 1/32 before squaring keeps the sum in range.

Verified on a Pixel 8a GPU: zero banned ops, zero >4D tensors, full residency, and GPU-vs-PyTorch output correlation ≈ 1.0 (the on-device GPU result, not just the host CPU result).

Training data & PII

SigLIP 2 was pretrained by Google on the WebLI dataset (billions of web-crawled image–text pairs, multilingual, sigmoid contrastive objective). No new training was performed for this conversion — it is a weights-exact format change of the public timm checkpoint. Because the source data is web-scraped, it may incidentally contain people, faces, text, and other PII; no PII was deliberately collected, and this conversion adds none. Apply your own content/PII filtering as appropriate. See the original SigLIP 2 model card and paper for full dataset details.

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Runtime	Backend	Graph on GPU	Latency
TFLite benchmark_model (TfLiteGpuDelegateV2)	GPU (OpenCL)	809 / 1575	963.6 ms
TFLite benchmark_model	CPU (XNNPACK, 4 threads)	—	312.9 ms

Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.

On this delegate the CPU is the faster choice for (312.9 ms on CPU against 963.6 ms on GPU) — worth knowing before you reach for the GPU on a mid-range phone.

Note that the GPU does not take the whole graph here (809 of 1575); the remainder runs on the CPU and the split costs a per-partition round trip.

License & attribution

• Apache-2.0 (original SigLIP 2 / timm checkpoint).
• This is a format conversion; all credit to the original authors (Google).