Pii

PII Detection and Anonymization

What is PII?
Personally Identifiable Information that can identify individuals: names, SSNs, addresses, phone numbers, etc.

PII Categories
| Category | Examples | Risk Level |
|----------|----------|------------|
| Direct identifiers | SSN, passport | High |
| Contact info | Email, phone, address | High |
| Financial | Credit card, bank account | High |
| Health | Medical records | High |
| Quasi-identifiers | Age, ZIP, occupation | Medium |

Detection Methods

Regex Patterns

PII_PATTERNS = {
    "ssn": r"\b\d{3}-\d{2}-\d{4}\b",
    "email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b",
    "phone": r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b",
    "credit_card": r"\b(?:\d{4}[-\s]?){3}\d{4}\b",
    "ip_address": r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b",
}

def detect_pii_regex(text):
    findings = []
    for pii_type, pattern in PII_PATTERNS.items():
        matches = re.finditer(pattern, text)
        for match in matches:
            findings.append({
                "type": pii_type,
                "value": match.group(),
                "start": match.start(),
                "end": match.end()
            })
    return findings

NER-Based Detection

import spacy

nlp = spacy.load("en_core_web_lg")

def detect_pii_ner(text):
    doc = nlp(text)
    pii_entities = []

    pii_labels = ["PERSON", "ORG", "GPE", "DATE", "MONEY"]
    for ent in doc.ents:
        if ent.label_ in pii_labels:
            pii_entities.append({
                "type": ent.label_,
                "value": ent.text,
                "start": ent.start_char,
                "end": ent.end_char
            })

    return pii_entities

Microsoft Presidio

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def anonymize_text(text):
    # Analyze
    results = analyzer.analyze(text=text, language="en")

    # Anonymize
    anonymized = anonymizer.anonymize(text=text, analyzer_results=results)

    return anonymized.text

Anonymization Strategies
| Strategy | Description | Example |
|----------|-------------|---------|
| Redaction | Remove entirely | [REDACTED] |
| Masking | Partial hide | *--1234 |
| Pseudonymization | Replace with fake | John Doe -> Person_1 |
| Generalization | Reduce precision | 94105 -> 941 |

Implementation

def anonymize(text, strategy="redact"):
    pii_findings = detect_pii(text)

    # Sort by position (reverse to preserve indices)
    pii_findings.sort(key=lambda x: x["start"], reverse=True)

    for pii in pii_findings:
        if strategy == "redact":
            replacement = f"[{pii["type"].upper()}]"
        elif strategy == "mask":
            replacement = mask_value(pii["value"], pii["type"])
        elif strategy == "pseudonymize":
            replacement = get_pseudonym(pii["value"], pii["type"])

        text = text[:pii["start"]] + replacement + text[pii["end"]:]

    return text

Best Practices
- Combine regex and NER for coverage
- Test with diverse PII formats
- Log anonymization for audit
- Consider context (names in quotes may be fictional)
- Use established libraries (Presidio)