On checking set*id() return values

What it is

A technical writeup on why SUID/SGID binaries must check the return value of setuid() / setreuid() / setresuid(). The failure mode: a program calls setuid(getuid()) to drop root, the call fails, the unchecked return leaves the process still privileged while the rest of the code assumes privileges are gone — a straightforward privilege-escalation primitive.

How it works

On Linux 2.6.0–3.0, set*uid() enforced RLIMIT_NPROC: if changing the real UID would push that UID's process count over the limit, the call returned -1 with EAGAIN. An attacker could deliberately exhaust the target UID's process count so the drop failed. The 2.6 check was itself added to stop daemons from bypassing per-user process limits (start as root, drop to a user). Since Linux 3.1 the set*uid() call no longer fails in this case — it changes the UID and sets an internal PF_NPROC_EXCEEDED flag instead. The limit is re-checked and enforced at the next execve(), which fails with EAGAIN if still over limit. The kernel comment in flag_nproc_exceeded() (kernel/sys.c) states this explicitly: they defer the failure "because too many poorly written programs don't check set\*uid() return code, assuming it never fails if called by root."

Affected

Any SUID/SGID binary that drops privileges via set*uid() and does not check the return value. On kernels ≥ 3.1 the historical RLIMIT_NPROC trigger no longer works against set*uid() itself — but the man page still advises checking, and the enforcement now surfaces at execve().

What to do

• Always check the return value of setuid(), setreuid(), setresuid(). Treat a non-zero return as fatal and abort — never continue as if privileges were dropped.
• Because 3.1+ defers RLIMIT_NPROC enforcement, also check execve() for EAGAIN in the fork() + set*uid() + execve() workflow; a successful UID drop no longer guarantees a successful exec.

Caveats

• The page states the classic RLIMIT_NPROC-based technique to force a setuid() failure does not work on modern (≥ 3.1) kernels; its own TL;DR calls such triggers "less likely than in the past" rather than impossible, and the author's included C proof-of-concept demonstrates only that this specific EAGAIN-on-setuid() path is closed — not that set*uid() can never fail. The guidance to always check is unconditional regardless of kernel version.