---
title: Llm Moderation Testing
emoji: 🦀
colorFrom: gray
colorTo: blue
sdk: gradio
sdk_version: 6.0.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: A model to test different models assessing content policies
hf_oauth: true
hf_oauth_scopes:
 - read-repos
 - write-repos
 - manage-repos
 - inference-api
---

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

## Dedicated-endpoint classifier models

Most models run through the HF Inference Router (pay-per-token, nothing to host).
Three classifier models are **not** served by the router and each need their own HF
Inference Endpoint. They appear in the model dropdown but raise a clear error until
the endpoint + secrets are configured.

| Model | Serving | Recommended GPU | ~$/hr |
|---|---|---|---|
| PolicyLM-1.7B | Custom handler | T4 16 GB | ~$0.50 |
| Shieldstral-1.0-3B | vLLM (OpenAI-compatible) | L4 24 GB | ~$0.80 |
| CoPE-B-A4B | vLLM (OpenAI-compatible) | **A100 80 GB** | ~$4.00 |

> **Cost / teardown — read this.** Endpoints bill **per minute of uptime**, not per
> request. For a short event, deploy + smoke-test the day before, **Pause** all three
> (paused = no compute billing, weights stay cached), **Resume** ~30 min before, then
> **Pause or Delete immediately afterward**. A forgotten CoPE-B A100 is ~$4/hr ≈
> $96/day. Prices above are approximate HF list prices — confirm at
> https://huggingface.co/pricing.

### Shieldstral-1.0-3B (`mistralai/Shieldstral-1.0-3B`)

- Deploy to a GPU Inference Endpoint with the **vLLM** (OpenAI-compatible) container
  on an **L4 (24 GB)**. No custom handler needed.
- Set Space secrets:
  - `SHIELDSTRAL_ENDPOINT_URL` — the endpoint base URL ending in `/v1`
  - `SHIELDSTRAL_INFERENCE_TOKEN` — a token with access to the endpoint
  - `SHIELDSTRAL_SERVED_MODEL_NAME` — *(optional)* if the endpoint serves under a
    name other than `mistralai/Shieldstral-1.0-3B`
- The app calls chat completions with `max_tokens=1, logprobs=True, top_logprobs=20`,
  softmaxes the yes/no logprobs into a violation probability (threshold 0.5), and
  shows the score in the Reasoning Trace panel.

### PolicyLM-1.7B (`musubilabs/policylm-1.7b`)

- Not OpenAI/vLLM-compatible — deploy as a **Custom** Inference Endpoint on a
  **T4 (16 GB)** using the provided `policylm_handler.py` (rename to `handler.py` in
  the endpoint repo). See the deployment steps in that file's docstring.
- Set Space secrets:
  - `POLICYLM_ENDPOINT_URL` — the endpoint URL
  - `POLICYLM_INFERENCE_TOKEN` — a token with access to the endpoint
- The handler returns `{flagged, max_score, categories[]}`; the app maps `flagged`
  to the violation label and lists per-category scores in the Reasoning Trace panel.

### CoPE-B-A4B (`zentropi-ai/cope-b-a4b`)

- 25.2B-total MoE (~52 GB bf16) — deploy with the **vLLM** (OpenAI-compatible)
  container on an **A100 80 GB**. Container args: `--trust-remote-code` and
  `--max-model-len 16384`. First deploy downloads ~52 GB, so allow **15–30+ min** and
  do it ahead of time.
- Set Space secrets:
  - `COPE_B_ENDPOINT_URL` — the endpoint base URL ending in `/v1`
  - `COPE_B_INFERENCE_TOKEN` — a token with access to the endpoint
  - `COPE_B_SERVED_MODEL_NAME` — *(optional)* if served under a different name
- The app sends the INSTRUCTIONS/POLICY/CONTENT/ANSWER prompt with `max_tokens=1`,
  reads the `"0"`/`"1"` answer (1 == violation), and softmaxes the logprobs into a
  violation probability shown in the Reasoning Trace panel.