Is the cache hit rate different from connecting directly to the provider?
Yes, but the gap has narrowed significantly. Prompt Cache at mainstream providers takes effect per account. Poixe’s backend connects to multiple upstream accounts to ensure capacity and stability, so consecutive requests from the same user may be assigned to different accounts, preventing the cache from being reused. To solve this, Poixe has launched Cache Affinity Routing: within a certain time window, requests from the same user for the same model are pinned to the same channel, so the upstream cache can accumulate and be hit continuously.Cache Affinity Routing
Learn how affinity routing works, its scope, and its expiration conditions
How are cache hits billed?
Poixe does not change the providers’ cache billing rules:- When the upstream actually hits the cache, Poixe bills at the provider’s discounted cache price
- No extra configuration is needed; the system automatically detects cache hits and settles at the cache price
Scenarios where a cache miss can still occur
Even with affinity routing enabled, the cache may still miss in the following cases:Cache affinity routing is a best-effort optimization, not an absolute guarantee. However, for continuous-conversation scenarios such as coding assistants, the hit rate has improved significantly.
Upstream caching references
OpenAI Prompt Caching
OpenAI’s official caching rules and billing details
Anthropic Prompt Caching
Anthropic’s official caching rules and billing details