A pretrained model knows an enormous amount about language and often about the world — and none of it about how to behave. It has no notion that a question deserves a direct answer rather than a continuation of similar-looking text scraped from a forum. Fine-tuning is where capability turns into behaviour, and most complaints of "the model can't do X" are actually behaviour problems in disguise.
Full fine-tuning of a seven-billion-parameter model requires storing gradients and Adam's two moment buffers for every single one of those parameters — memory that a single consumer GPU simply doesn't have. Parameter-efficient fine-tuning methods sidestep this by training a tiny fraction of parameters instead, built on a striking empirical observation: the weight updates that fine-tuning actually needs are far lower-rank than the weight matrices themselves.
Almost nobody trains a vision model from scratch anymore, and there's a good structural reason: the early layers of any CNN trained on natural images learn something close to universal — edges, colours, simple textures — regardless of what specific objects the model was ultimately trained to recognise. That generic foundation is exactly what transfer learning reuses.