3 docs tagged with "cost"

Two independent caches matter in a LangChain app, and they solve different problems.

The training bill arrives, and most of it bought idle GPU time — a genuinely common outcome, and an entirely preventable one. Utilisation, not hourly rate, decides training cost: a half-idle expensive GPU beats neither a well-fed cheap one nor, often, a smaller model trained more efficiently.

An unbounded chat history eventually exceeds the model's context window, and even before that, it inflates every call's token cost and latency. Two standard strategies keep it in check.