AK_QMUL at ArGuard Shared Tasks: A Single Fine-Tuned JAIS for Arabic Harmful-Prompt Detection

What it claims

AK_QMUL's ArGuard-2026 system shows that one recipe, JAIS-6.7B with 4-bit QLoRA and a linear head on the last-token state, wins both Arabic harmful-prompt tracks. The decisive B1 ingredient is AraSafe augmentation, not the dialect-factored head. Without the extra safe MSA prompts, B1 macro-F1 falls from 0.984 to 0.624. The B2 weakness on Bully is almost entirely an MSA problem: Bully F1 is 0.46 in MSA and 0.81 to 0.97 in the five dialects.

How it works

The shared setup loads JAIS-6.7B in 4-bit NF4 with double quantisation and bfloat16 compute. LoRA adapters sit on all linear projections (r=16, alpha=32, dropout 0.05), and the base weights are frozen. The pooled vector is the top-layer hidden state at the last non-padding token, passed in float32 to one linear layer. Training runs for 3 epochs with AdamW at learning rate 1e-4, linear schedule with 5% warmup, batch size 8 and max length 192. The loss is inverse-frequency class-weighted cross-entropy. Folds use stratified group k-fold (k=7), and the final models are refit on train plus dev.

For B1, labels are split into safe/unsafe × MSA/other. The argmax over four logits collapses to binary, with no dialect post-processing. The augmentation keeps 7,607 of AraSafe's 12,077 human-written prompts: non-harmful ones only, at least ten characters, with no dialectal markers, deduplicated against train, dev and test. All are labelled safe-MSA and added to the training fold only. For B2, five seeds (42 to 46) of a seven-way classifier are trained, their softmax outputs are averaged, and the argmax is taken. There is no stacking and no per-class thresholds.

Evidence

The B1 ablation is run on the official test set with one seed:

• A 2-class head with AraSafe scores 0.983.
• A 4-way head without AraSafe scores 0.624.
• A 4-way head with AraSafe scores 0.984.

The training pool of 19,236 prompts is about 80% unsafe, while the test set is near balanced. The un-augmented model therefore flags 6,426 of 7,809 test prompts as unsafe, against 3,851 truly unsafe. The submitted model predicts 3,888. Per-dialect B1 accuracy runs from 93.6% (Moroccan) to 99.8% (Egyptian).

On B2, single seeds score 0.779 to 0.785, and the five-seed average scores 0.790. Bully has precision 0.815 but recall 0.443, and its misses go mostly to harassment and harm-to-others. The class has 908 test examples, so data volume does not explain the gap. Of those 908, 719 are MSA. MSA macro-F1 is 0.70, against 0.88 to 0.95 for the dialects. Harm-to-others, self-harm and FraudandDeception follow the same MSA drop.

How to use it

The code and the train/dev splits are at https://github.com/alikhairallah/AK_QMUL-ArGuard-2026. The repository does not include test labels or new harmful content. The page gives no licence, run commands or released weights.

Limits

The authors compare no other backbones or encoders, so the results speak to this recipe only. The B1 ablation uses a single seed and fold. The 0.983 to 0.984 gain from the four-way head is within likely seed noise. The non-MSA per-class cells rest on 14 to 239 examples, and the authors call them indicative. The authors expect that fixing Bully in MSA needs finer annotation rather than more data.