Prompt tuning is a type of Parameter Efficient Fine Tuning (PEFT) method in the field of Natural Language Processing (NLP). It enables large language models (LLMs) to adapt to a wide range of tasks. Helps large language models (LLMs) generate more accurate responses.
In this technique a small set of trainable parameters, in the form of a prompt, is optimized while keeping the rest of the large language model (like GPT3 or GPT4) fixed. This approach allows for customizing the model’s responses to specific tasks or domains without the need to train or fine-tune the entire model, which can be computationally expensive and time-consuming. This technique involves introducing a small number of tunable tokens (soft prompts) that are prepended to the input text. These tokens are trained end-to-end to encapsulate task-specific information.
The advantages of prompt tuning include significantly reducing the number of parameters that need to be adjusted for each task, and allowing for the reuse of a single pre-trained model across multiple tasks. This approach is shown to be effective, particularly as the size of the language models increases, and has been compared favorably to traditional fine tuning methods, especially in terms of parameter efficiency and task adaptability.
Soft prompt methods keep the model architecture fixed and frozen, and focus on manipulating the input prompt to achieve better performance. This can be done by adding trainable parameters to the prompt embeddings or keeping the input fixed and retraining the embedding weights. Here is a conceptual way to think about this: when you enter a prompt, an automatic soft prompt is added to your prompt by the prompt tuned LLM itself.
References:
https://arxiv.org/abs/2104.08691
https://arxiv.org/abs/2303.15647
See Also: PETM, Prompt Tuning, Prompt tuning with soft prompts