Foundation model: one learned base, several ways to adapt it

A foundation model is trained on broad data at scale and can be adapted to many downstream tasks. The term describes its role as a reusable base, not one particular architecture or output type.

A language model, a vision encoder or a multimodal model can provide that base. Its useful representations and capabilities are learned before the final application is specified.

Adaptation does not always mean changing weights

One application can freeze the model and train a small head. Another can fine-tune existing parameters. A third can condition it with instructions and demonstrations in context.

Flamingo illustrates composition: pretrained vision and language components can be connected into a model that handles interleaved images and text. Reuse can happen inside the architecture as well as between applications.

The shared foundation also carries shared limitations. A data gap or unreliable behaviour can propagate into many products built on the same base. Evaluate the adapted system on its actual task; the label “foundation” does not establish suitability.