GL-A-020Sheet 020FOR CONSTRUCTIONA · ARTIFICIAL INTELLIGENCE
Regex catches what it can name. NER catches what it can't.
SECTION · REGEX vs NER · REDACTION ENGINE · DETECTION
General notes
- 01An email has a shape. A card number has a checksum. A pattern finds them in any language, deterministically, in microseconds.
- 02A person's name in running prose has no shape. That needs a model. Keep the two apart: patterns in the hot path, the model loaded only where prose demands it.
Schedule of methodsPick by data shape, not by fashion
| Data | Method | Why |
|---|---|---|
| Email · phone · card · IBAN · keys | Validated patterns | Deterministic, ReDoS-safe, tiny |
| "Name: …" · "DOB: …" · JSON keys | Label-anchored patterns | The label gives the shape |
| Names, companies, places in prose | Named-entity model | No shape to match |
As posted
Regex catches what it can name. NER catches what it can't.
An email has a shape. A card number has a checksum. Patterns find them in any language, fast.
A name in prose has no shape. That needs a model. Models are heavy, so they load only where prose demands it.
Sheet 020.
Read with
- Sheet 019 · (415) 555-0142 is invisible to most redaction.