What is prompt caching?

本站添加: 2026-05-06

[00:00:00]Today I learned about prompt caching. If you're going to have a dynamic variable in your prompt, try and put it at the end so that it can benefit the most from caching. If you see on this green chart cache hits, so if all of these tokens match exactly, they will benefit from caching. However, these last two right here, let's say they are the dynamic part of your prompt, then they won't, which is fine.

[00:00:22]Uh however, if you place that dynamic variable right at the beginning, none of it gets cached.