Internal APIs Without Protobuf Overhead

Service-to-service APIs are the quiet bandwidth bill. Public REST endpoints stay on JSON for partners and browser clients. Internal calls — worker-to-worker, BFF-to-backend, sync jobs, bulk fetches — are where teams switch to MessagePack or gRPC+Protobuf to save bytes and parse time.

Twilic targets the MessagePack sweet spot: no schema required on day one, self-describing payloads, polyglot SDKs — with decisively smaller responses when services return lists, pages, or bulk records.

The Internal API Dilemma

Platform teams face a recurring trade-off:

Approach	Pros	Cons
JSON	Debuggable, universal	Verbose on lists; slow parse at volume
MessagePack	Schema-less; easy adoption	Key overhead on every object in a batch
Protobuf + gRPC	Smallest single records; mature	.proto governance; poor ad-hoc maps
Twilic Dynamic/Batch	Schema-less; smaller batches	Binary; newer ecosystem

Frameworks like MagicOnion demonstrate the MessagePack path: C# interfaces define the API, MessagePack serializes arguments and results on HTTP/2, teams skip .proto files. The gap appears when those results are arrays of objects — user lists, ledger entries, inventory rows.

Where Twilic Wins

Paginated list endpoints

A internal GET /internal/users?page=3 returning 50 records × 8 fields:

• JSON sends 50 × 8 field names
• MessagePack sends 50 × 8 field names (compact encoding, but still repeated keys)
• Twilic Batch sends field names once + interned repeated strings

Typical reduction: 40–60% vs JSON, 30–50% vs MessagePack on multi-record responses.

See Examples — API response for a runnable Hono server with size comparison.

Bulk RPC and job payloads

Data export, reconciliation, and ETL jobs often move thousands of same-shaped records in one call. Row batch (encodeBatch) or columnar batch (col_batch) matches the workload:

• Row batch: nested objects, moderate field count
• Columnar batch: tabular data, many rows, numeric columns

Gradual schema adoption

When a hot path stabilizes — payments, fraud checks, entitlement updates — promote to Bound Profile without changing transport:

Phase 1: Dynamic on all internal routes
Phase 2: Batch on list/bulk endpoints
Phase 3: Bound on 2–3 fixed-message hot paths

Protobuf forces phase 3 from day one. Twilic lets you earn it.

Integration Patterns

HTTP content negotiation

Serve Twilic on internal routes with a dedicated media type; keep JSON on public routes.

GET /internal/orders HTTP/1.1
Accept: application/vnd.twilic

The @twilic/hono and @twilic/fetch packages handle negotiation and decoding.

gRPC-like streams without Protobuf

For long-lived HTTP/2 or WebSocket streams between services, pair Stateful Profile with internal APIs when updates are incremental. See Real-Time Dashboards and Streaming.

Polyglot microservices

Twilic interoperability: a Rust producer and Go consumer exchange bytes without a shared schema file in Dynamic mode. Platform standards can mandate:

• Content-Type: application/vnd.twilic on internal east-west traffic
• Batch encoding for any response with ≥ N same-shape records
• JSON only at the north-south edge

Security Boundaries

Internal APIs are not automatically trusted:

• Service mesh identity proves caller, not payload safety
• Deserialization of bytes from a compromised peer is still an attack surface

Apply:

• Maximum payload size limits
• Reject typeless or dynamic map keys from untrusted peers where your SDK supports strict mode
• Keep public and partner APIs on JSON or governed Protobuf

Twilic belongs behind authentication and network policy — same as MessagePack.

Migration from JSON Internal APIs

A low-risk rollout:

1. Add Twilic encoder alongside existing JSON serializer
2. Dual-publish Accept header support on one read-heavy endpoint
3. Migrate internal clients one by one
4. Measure bandwidth on service mesh metrics or load balancer logs
5. Enable batch encoding when all clients on an endpoint support it

No flag day. JSON clients continue working until retired.

When to Use Protobuf Instead

Choose Protobuf + gRPC when:

• gRPC is already org-standard
• A single .proto is owned by a platform team with semver discipline
• You need generated stubs in many languages and maximum single-record density
• Long-lived external contracts cross company boundaries

Choose Twilic when:

• Teams need schema-less flexibility and faster iteration
• Bulk/list payloads dominate bandwidth
• You want to start without codegen and add schema later

Next Steps

• Migrating from Protobuf
• Web Integrations
• Cookbook — API Response
• Comparison — Twilic vs Protobuf
• Business Use Cases — Internal APIs