Learn and improve

Problems surfaced before they reach contacts.

The platform monitors for compliance breaches, AI parse failures, delivery rate drops, queue depth spikes, and token budget thresholds. Friyay operators are alerted immediately on critical events — with enough information to act without logging in first.

See it in action Read the docs

Immediatecompliance false negatives — page fires before further messages send

1%delivery failure rate threshold over a 5-minute rolling window

80%token budget threshold — nightly alert at this level

5×per-turn token anomaly threshold — alerts on unusual cost per conversation

Alert severity levels

Not all alerts are equal. The platform knows the difference.

Alerts are classified by severity — critical (page immediately), high (investigate urgently), and medium (investigate within business hours). A compliance false negative is critical. An AI parse failure is high. Queue depth is medium. Different severities trigger different response protocols.

Critical — compliance false negative: message sent to opted-out contact
High — AI parse failure: fallback substituted on any non-zero rate
High — delivery failure rate: exceeds 1% over rolling 5-minute window
Medium — queue depth: any queue exceeds configured depth threshold
Nightly — token budget: campaign reaches 80% of monthly_token_budget
Per-turn — token anomaly: single conversation exceeds 5× campaign average cost

Alert severity tiers

1
Critical
Compliance false negative — page immediately, stop campaign
2
High
AI parse failure — investigate prompt or model drift
3
High
Delivery failure > 1% — investigate provider or number
4
Medium
Queue depth spike — investigate worker capacity
5
Nightly
Token budget at 80% — discuss with client before overage
6
Per-turn
5× token anomaly — possible injection or loop

What each alert catches

Operational problems before they become client problems.

Each alert type addresses a specific failure mode that would otherwise require manual discovery — either from a client complaint or from a data review. The platform surfaces them first.

Compliance false negative

If a message is sent to a contact after they have opted out, an immediate critical alert fires to Friyay operators. The CI compliance gate prevents this from deploying — but if a runtime failure escapes the gate, the alert catches it before further messages send.

How it works: The highest-severity alert in the system — paged immediately per the monitoring architecture. Distinct from the CI gate which prevents deployment.

AI parse failure

When the AI returns a response that fails schema validation and a fallback is substituted, a high-severity alert fires. Any non-zero parse failure rate warrants investigation — it may indicate model drift, a prompt change, or a schema mismatch.

How it works: Fallback flag is set on conversation_turns (tested in CI). The alert pathway from fallback=True to operator notification is the monitoring layer — verified in staging.

Token budget alerts

When a campaign's monthly token usage reaches 80% of its configured monthly_token_budget, a nightly alert fires to Friyay operators. This gives time to discuss with the client before costs exceed the agreed limit. Per-turn anomalies (5× average) alert immediately.

How it works: monthly_token_budget field and model_pricing table specified in platform architecture. Nightly budget check job fires alert at 80% threshold.

Queue and delivery monitoring

Delivery failure rates above 1% over a rolling 5-minute window indicate a systemic problem — not isolated retries. Queue depth alerts distinguish between a traffic spike and a worker outage. Both alert at different thresholds.

How it works: Queue depth observable via RQ Dashboard. Delivery failure rate tracked via delivery_status on outbound message rows — monitored at the infrastructure layer.

Want to see how operational
alerting is configured?

We'll walk through the alert architecture and show you what Friyay operators see on a critical event.

Book a discovery call Talk to the team