Posts tagged with "latency"

Claude message threads went into public beta on 2 November 2026, including on Microsoft Foundry: send the conversation once, then only new messages and an id, for about 35% lower time to first token on 500K to 1M token requests. Token charges and rate limits do not change, so caching remains the cost lever. The thread is a server-side copy of your conversation, held in memory for up to an hour and outside ZDR, which EU teams need to document.