How Customer-Facing AI Agents Determine Trust

A customer-facing AI agent talks to two fundamentally different audiences:

1. Customers -- untrusted, unauthenticated, arriving via SMS or phone. They should be able to book appointments and ask questions, but not read the owner's email or resolve escalations.

2. The business owner -- trusted, managing their AI through a portal or their own phone. They should have full access to everything.

The agent is the same code, the same model, the same system prompt. The difference is which tools it has available. This is the trust boundary -- and how you determine trust matters more than most people think.

Phone Numbers as Identity

When a text message arrives, we compare the sender's phone number to the business owner's phone number. If they match, the agent gets full tool access. If not, tools are filtered to the customer level.

The obvious objection: "Can't someone spoof the phone number?"

In the Twilio ecosystem, not without compromising the Twilio account itself. Here's why:

Webhook signature validation. Every inbound request is validated against an HMAC signature using the account's auth token. Forging a request requires knowing the token -- which means you've already compromised the account.

Carrier-validated sender IDs. The From field in a Twilio webhook comes from the carrier network's signaling, not from the sender's device. You can't set an arbitrary From number on a text message the way you can spoof caller ID on some voice calls.

The attack surface is account security, not phone trust. If an attacker has your Twilio credentials, they can already send messages as your business, read your message logs, and reconfigure your webhooks. Phone number spoofing is a subset of a much larger compromise.

The real mitigation is infrastructure security: account 2FA, IP allowlisting, credential rotation. These protect against the actual attack vector (account compromise), not the downstream symptom.

This is the same trust model every phone-based system uses. When your bank sends you a text, they check your phone number. When your doctor's office calls, they verify by phone number and date of birth. Phone-number-as-identity is the baseline for telecommunications.

Portal: Authentication IS Authorization

When a business owner logs into the web portal, the trust context is owner. Always. No second verification step.

Why? Portal access requires authentication -- email and password or OAuth. The user must be associated with the business in the database. If you're authenticated and you're the business owner, you ARE the owner. There's no meaningful distinction between "authenticated owner" and "owner with extra verification."

This changes when you add team members with different permission levels. A receptionist who can view conversations but not edit the AI's instructions. An accountant who can see billing but not send messages. When those users exist, portal trust will map from role-based permissions.

But building role-based portal trust before having team members with different roles is speculative abstraction. Build it when the first client asks "can my assistant see the conversations but not change the instructions?"

API Keys for External Tools

Our MCP server (the API that lets Claude Desktop and other AI tools interact with the system) uses bearer token authentication. Each business has an API key. The key authenticates both identity and authorization.

MCP clients get the same capabilities as the portal -- full tool access. The API key IS the trust credential. Required for every request. Scoped to one business. No cross-client access possible.

What the Trust Boundary Actually Controls

The critical insight: the trust boundary doesn't restrict what the AI says. It restricts what actions the AI can take.

An untrusted customer's AI agent doesn't have access to read_inbox, resolve_escalation, or update_business_profile. These tools are physically absent from the API call. No amount of clever prompting can invoke a tool that isn't in the tool list.

This is enforcement at the API boundary, not at the content level. We don't try to detect malicious intent in messages. We restrict what actions are available based on who's calling.

The Design Principles

Enforce at the boundary, not in the content. Don't try to figure out if a message is "safe." Control what the agent can do based on who sent it.

Trust infrastructure over application logic. Twilio validates webhooks. The auth system validates sessions. We validate tool access. Each layer trusts the layer below it.

Build for actual users, not hypothetical attackers. The trust model serves real customers and real business owners. Elaborate role-based access comes when real team members need it.

Fail closed. Unknown tools default to the most restrictive level. Missing trust context defaults to untrusted. New channels start with no tools until explicitly configured.

When This Changes

The trust model evolves when the user base evolves:

• Team members with restricted roles -- portal trust maps from permissions
• Customer portals -- authenticated customers get a middle trust tier
• OAuth for MCP -- replaces API keys with scoped tokens
• Multi-tenant admin -- admin trust with cross-client access controls

Each is driven by a real user need, not by a security audit checklist. The current model is right-sized for the current users. And when it changes, the architecture supports it -- trust context is a single parameter that flows through the entire system.