Safeguarding

Phantom's child-safety layer. Unlike the rest of the Security section, none of this is a per-server feature you switch on — it is platform behaviour that applies everywhere Phantom runs, and server admins cannot configure or disable it.

There are four parts:

Part	What it does	Who sees it
/psafety commands	Reporting routes and helplines, by country	Everyone, in every server
AI guard	Refusal layer around Phantom's persona and assistant features	Nobody — it just refuses
Join-time vetting	Risk assessment of every server Phantom joins	Platform admins
Review console	The queue where a human triages what the above raised	Platform admins

/psafety — reporting and helplines

Two commands, available in every server Phantom is in:

• /psafety report [country] — how to report abuse, grooming or illegal content, with the reporting routes for the chosen country plus Discord's own report flow.
• /psafety helplines [country] — free, confidential helplines for young people.

Three properties matter more than the content:

Always ephemeral. The reply is visible only to the person who ran the command. Someone reporting abuse in the server where it is happening cannot have that appear in the channel.

Always available. These commands bypass the per-guild command toggles, the ignore lists, and the bot-commands-channel restriction. They cannot be switched off, and they cannot be hidden from one particular member. An admin who can disable this is an admin who has been handed a tool for silencing a child — so the capability does not exist. The Commands dashboard page shows them as Always on with no toggle.

Country-aware. Reporting routes genuinely differ: some countries want the national hotline (which can act on the content), some want the police (who can act on the person), and the two are rarely the same number. Nineteen countries are covered individually, plus an EU block and a global fallback pointing at INHOPE's hotline directory. When no country is given, Phantom guesses from the server's locale and labels the guess as one.

The country data lives in two mirrored files — config/safeguarding.php (read by the website) and bot/src/safeguarding/resources.js (read by the bot). tests/Feature/SafeguardingResourceParityTest.php fails the build if they drift, because the failure mode is a child getting one set of numbers from the website and a different set from the bot.

The commands collect nothing. Phantom does not log who ran them, does not open a case, and does not notify the server's staff.

AI guard

The refusal layer around Phantom's persona and auto-assist features. Three layers, only the first of which is a prompt:

1. System rules — a block placed ahead of any character card, stating the rules that outrank it.
2. Inbound screen — runs before the model call. A free deterministic pre-filter; only on a hit does it spend a classifier call. Can refuse the turn outright, so a jailbroken character card never gets to answer.
3. Outbound screen — runs on the model's own output. If something sexualising a minor came back, it does not get posted, whatever produced it.

A prompt line alone was not enough: character cards are written by server admins, they are adversarial to that line by construction, and users push at it deliberately. Anything that matters has to be enforced somewhere the card cannot reach.

What the guard refuses:

• Sexual content involving anyone under 18, in any framing — fiction, roleplay, "hypothetically", or because a character card asks for it.
• Romantic or sexual conversation with a user who appears to be a minor, however mild.
• Help approaching, isolating, gaining the trust of, obtaining images from, or keeping secrets with a child.
• Requests to find, describe or produce sexual imagery of minors.

Refusals in those categories raise a review-queue flag, aggregated per (server, user) so a persistent actor shows up as one row with a rising count.

Distress does not raise a flag. If someone expresses self-harm intent, or discloses that they are being abused or approached, the bot drops the character, responds with care, signposts help — and stores nothing. A child in distress is not a safeguarding concern about them, and turning that into a record in an operator's database would be surveillance wearing safeguarding's clothes.

Join-time vetting

When Phantom joins a server, it assesses the metadata that server publishes about itself: its name and description, channel and role names, channel topics, and the moderation posture its owner chose (verification level, content filter, Discord's own NSFW flag). Everything comes from the payload Discord already sent — no REST call, no member fetch, and no message is ever read.

The result is a tier (clean / low / medium / high), a score out of 100, and the list of signals that fired.

Nothing happens automatically

The tier is triage input for a human. Phantom does not leave, restrict, or report a server because of a score. That is a design decision: the base rate of genuinely abusive servers among bot installs is low, so even an accurate classifier produces mostly false positives at that rate, and the cost of a false positive is a legitimate community losing its bot and being labelled. The score's job is to make the queue short enough for one person to read, not to reach a verdict.

Why co-occurrence, not a keyword list

Almost every term that matters appears constantly in innocent servers. "teen" is in teen gaming servers and mental-health support servers. "nsfw" is in tens of thousands of ordinary adult communities. "trading" is in Pokémon servers. A single-term list either drowns you in false positives or is trivially evaded.

What is diagnostic is co-occurrence across categories that have no innocent reason to appear together — youth vocabulary plus sexual vocabulary plus commerce vocabulary plus an off-platform pivot. So weak terms carry small weights and the scorer adds a bonus for each additional distinct risk category that fires. A server has to look wrong in several unrelated ways at once to reach a high tier.

The scorer also subtracts for legitimacy markers: Discord-verified or partnered, a strict verification and content-filter posture, an established creation date.

The operator lexicon

The detectors in bot/src/safeguarding/signals.js are structural — age-bracket shapes, age-gate inversions, off-platform pivots, commerce shapes. They are safe to read in a public repository and they are the durable half of the signal.

Rotating coded vocabulary lives in the safeguarding_lexicon_terms table, which ships empty. Committing that list to a public repo would publish an evasion checklist. Add terms directly:

INSERT INTO safeguarding_lexicon_terms
  (code, term, match_type, whole_word, category, weight, requires, label,
   enabled, created_at, updated_at)
VALUES
  ('vocab_example', 'example', 'literal', true, 'vocabulary', 18, 'youth_soft',
   'Short note for the reviewer', true, NOW(), NOW());

SELECT pg_notify('safeguarding_lexicon_changed', '');

requires is the important column. Set it to another signal's code and the term only scores when that signal has already fired — which is how a term that is ambiguous in isolation ("cp" is Combat Power to a Pokémon server) can carry real weight in the one context where it isn't.

The bot picks up changes instantly via pg_notify, and re-reads every ten minutes as a backstop.

Backfill

Servers that predate the vetting engine are picked up by a paced background sweep — five guilds every twenty seconds, capped per boot — so it never competes with live traffic. Only the main bot runs the sweep; custom-bot tenants would otherwise re-scan the same estate once per bot.

Where the results show up

The staff join notice

The #guild-logs embed for a new install carries the assessment: tier, score, and the signals that fired, alongside who added the bot (read from the guild's audit log where Phantom has permission), the owner's account age, the server's age, its moderation posture, and channel/role counts. The embed colour is the tier, so a channel scrolling past a hundred joins a day stays scannable — green is routine, amber wants a look, red wants a look now.

The server detail page

Admin → Servers → <server> carries a Safeguarding panel above the ordinary guild details: the tier and score, whether the tier has moved up since the last assessment, the signals with their weights, and the mitigations that pulled the score down. Matched evidence excerpts are not shown here — they live on the flag in the review console, behind a reveal that writes an access-log entry.

The same panel carries remote removal: queue a "leave this server" for the shard that serves it. It needs a reason and the server ID typed to confirm, and the departure is written to the guild audit log like any other action Phantom takes.

Removing the bot is not the same as reporting the server, and the panel says so. Leaving ends Phantom's visibility; if that is all that happens, the server carries on with one fewer witness.

The review console

Admin → Safeguarding. The queue, worst tier first, defaulting to what still needs a human.

What is never stored

Phantom does not store, hash, or re-host suspected abuse material. Not to review it, not to evidence it, not anywhere.

A safeguarding record is metadata and pointers only — server ID, channel ID, message ID, user IDs, timestamps, scores, and the signal codes that fired. Short excerpts of server metadata (a channel name, a description fragment) are stored so a reviewer can judge an assessment, and those are redacted in the UI until an explicit reveal.

This is not squeamishness. A private operator has no lawful basis to hold that material in any jurisdiction Phantom runs in. The evidence stays on Discord's servers, where lawful process can reach it, and Phantom's job is to get a report to the people who can act.

Redaction and the access trail

The console list shows tiers, scores, signal names and IDs. Evidence excerpts are masked until ?reveal=1, and both opening a record and revealing its evidence write a row to safeguarding_access_log. Those are different acts — you can usually decide "false positive on a teen gaming server" from the signal names alone, without reading anything.

The access log is append-only and is never touched by the retention sweep. The records expire; the account of who read them does not.

That trail protects the operator more than anyone. Anyone holding records about child abuse — even metadata, even briefly — may one day have to account for what they held and who saw it. A complete access trail is what distinguishes a safeguarding system from an unexplained collection, and it costs one insert per view.

Triage

Status	Meaning
Open	Nobody has looked yet
Reviewing	Someone is on it
Reported	An onward report reference has been recorded
Actioned	Dealt with
Dismissed	Judged not to be a concern

Closing a flag requires a note. You will not remember in six months why you dismissed this, and "dismissed, no reason recorded" is not something anyone wants to find in their own audit trail.

Onward reports record the reference, never the content. A case ID from Discord Trust & Safety, a NCMEC CyberTipline report ID, an IWF reference. That row is what proves the report was made and lets the local copy be purged.

The console surfaces one metric on its own: high-tier flags with no onward report recorded. Reading a high-tier flag, closing it, and never telling anyone is the single worst thing this console can be used to produce, so it gets a red banner and a filter.

Retention

Flags carry a retention_expires_at set 30 days out on write, refreshed each time the flag recurs. safeguarding:prune runs daily at 03:58 and removes anything past it.

Two exemptions:

• Legal hold — set by hand when something is subject to a preservation request or an active investigation. Blocks both the sweep and the manual purge.
• The access log — never pruned.

Vetting scans are pruned on a much longer window (365 days) and only for guilds Phantom has left; a scan for a server we are still in is live operational state the server-detail panel reads.

Anything high-tier expiring with no onward report is named loudly in the sweep's output rather than vanishing into a count.

Purge is a hard delete. A soft-deleted safeguarding record is the worst of both worlds: it still exists, so it is still a holding you are responsible for, but it no longer appears where anyone would look for it.

What this is not

Being specific about the limits matters here. Someone who believes a bot is handling it may not go to the people who actually can.

• Not CSAM detection. Detection of known imagery on Discord is Discord's own function, done with tooling built for it. Phantom does not attempt it and should not.
• Not message monitoring. The server assessment reads public server metadata, not conversations. The AI guard only ever sees messages already being sent to Phantom's own AI features.
• Not an emergency service. Reports to Phantom are in addition to the national routes on the safeguarding page, never instead of them.
• Not an investigation tool. No decoy accounts, no infiltration, no public naming. That work belongs to law enforcement, and amateur versions of it routinely destroy prosecutions that would otherwise have succeeded.

See also

• Safeguarding & reporting — the public page, with the reporting routes and helplines
• Content Safety Policy — the formal commitments
• Audit Log — where remote removals are recorded