3 docs tagged with "api"

langserve wraps a Runnable in a FastAPI app and gives you /invoke, /batch, /stream, and /stream_events endpoints, plus request/response schemas derived from the chain's input/output types — for free, with no route code of your own.

Choosing between an overnight batch job and a 50-millisecond synchronous API is not a minor implementation detail — it changes almost every other decision downstream, from infrastructure to model architecture to error handling. The serving pattern should be chosen deliberately, before the model is designed, not bolted on after.

One formatting engine, several entry points that differ only in where the characters land — pick