Large Language Models

Source– This post on Large Language Models is based on the article “What is an LLM, the backbone of AI chatbots like ChatGPT, Gemini?” published in “Indian Express” on 27th February 2024.

Why in the News?

Large language model serve as the backbone of Artificial Intelligence Chat boxes like ChatGPT and Gemini.

What are Large Language Models

1. Description- Large language Models (LLMs) are large general-purpose language models that can be pre-trained and fine-tuned for specific purposes like text classification, question answering and document summarisation.

2. Use- Large Language Models enable the Generative AI models like ChatGPT and Gemini to “converse” with humans and predict the next word or sentence.

3. Features-
a. Large Data Sets- Large Language Models use extensive size of training data. They also have large parameter count.

Note- Parameters, also called hyperparameters, in machine learning represent the memories and knowledge acquired by a machine during model training. They determine the proficiency of the model in addressing a particular problem.

b. Use for General Purpose- This means the model is sufficient to solve general problems that are based on the commonality of human language regardless of specific tasks, and resource restrictions.

c. Tool to produce Human Language- It is a tool that helps computers understand and produce human language.

4. Types of LLMs:
a) On the basis of architecture- These are of 3 types, which are mentioned below-
i) Autoregressive model- They predict the next word in a sequence based on previous words. For ex- GPT-3

ii) Transformer-based model- They use a specific type of neural network architecture for language processing. For Ex- LaMDA or Gemini (formerly known as Bard)

iii) Encoder-decoder model- They encode input text into a representation and then decode it into another language or format. For Ex- T5, Bart, Pegasus, ProphetNet, Marge

b) On the basis of training data: There are three types of LLMs
i) Pretrained and fine-tuned- These language models are trained with multiple data set and are fine-tuned to provide accurate results.

ii) Multilingual Models- These LLMs can understand and generate text in multiple languages

iii) Domain-specific Models- These are trained on data related to specific domains such as legal, finance or healthcare.

c) Based on availability- They are categorised as open-source and closed-source.

i) Open Source- These use open source data from the web for training. For ex- LLaMA2, BlOOM, Google BERT, Falcon 180B, OPT-175 B.

ii) Closed Source- These models with close data sets. For ex- Claude 2, Bard, GPT-4, are some proprietary LLMs.

d) LLMs also vary based on their sizes. Large models require more computational resources but also offer better performance.

5) Working Methodology-

i) Deep learning is a key technique in training Large Language Models (LLMs). Deep learning involves using artificial neural networks inspired by the human brain.

ii) For LLMs, this neural network learns to predict the probability of a word or sequence of words by analysing the patterns and relationships between words in the data set used for training.

iii) Once trained, an LLM can predict the most likely next word or sequence of words based on inputs also known as prompts.

6) What can LLMs do

a) They generate text and are capable of producing human-like content for purposes ranging from stories to articles to poetry and songs.

b) They can strike up a conversation or function as virtual assistants.

c) They show proficiency in language understanding tasks, including sentiment analysis, language translation, and summarisation of dense texts.

d) LLMs engage with users providing information, answering questions, and maintaining context over multiple exchanges.

e) They can create content and personalise it, aiding in marketing strategies, offering personalised product recommendations, and tailoring content to specific target audiences.

6) Advantages of LLMs:

a)  Versatility: LLMs display a wide versatility as these can be applied to a broad range of tasks with just one model, due to their training on extensive datasets.

b) Efficiency with Limited Data: LLMs can perform effective tasks despite the availability of only small amounts of domain-specific data. This is because they utilize the extensive knowledge gained from their general language training.

c) Continuous Improvement: The performance of LLMs enhances as they are fed more data and parameters. This showcases a capacity for ongoing learning and development.

UPSC Syllabus- Science and Technology