Sayantan Das - Prem

Sign in Subscribe

Sayantan Das

RAG Strategies

The article "RAG Strategies" explores Retrieval-Augmented Generation (RAG) methods, detailing Naive RAG, Advanced RAG, and Modular RAG approaches. It introduces RAFT, a fine-tuning technique, and discusses optimizing large language models for RAG tasks.

Finetuning with LoRA and variants

Finetuning with LoRA and variants

The article "Finetuning with LoRA and Variants" discusses Low-Rank Adaptation (LoRA), a technique that efficiently fine-tunes large language models by adding a small number of trainable parameters, reducing computational costs. It also explores LoRA+ and other innovations enhancing model adaptation.

Mixture of Experts - Part 2

Mixture of Experts - Part 2

The article "Mixture of Experts - Part 2" explores the resurgence of Mixture of Experts (MoE) architectures in NLP, highlighting models like Mixtral 8x7B and DeepMind's Mixture-of-Depths. It discusses MoE's evolution since 1991, focusing on scalability and efficiency in large language models.

Model Alignment Process

Model Alignment Process

The article discusses methods to align large language models (LLMs) with human preferences, focusing on techniques like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO). It also introduces non-RL methods such as Kahneman-Tversky Optimization (KTO).

Providers Empirical Testing

Providers Empirical Testing

Prem's study evaluates how different providers implement Large Language Models (LLMs), affecting model quality. Using various prompts, the study compares providers like Anyscale, Replicate, FireworksAI, and Together, assessing parameters such as verbosity, correctness, and processing speed

LLM Datasets and Contamination

LLM Datasets and Contamination

Prem AI's post addresses dataset contamination in LLM training, where overlap between training and test sets inflates performance. It explores detection methods, data curation best practices, and ethical concerns around mislabeled or duplicated content.

Model Merging

Discover LLM model merging, a technique to combine multiple Large Language Models (LLMs) into a single, more powerful model without extra training. Explore top merging methods like Linear, SLERP, Task Arithmetic, TIES, and DARE, with YAML configs and practical use cases.

Emergent Capabilities of LLMs

Emergent Capabilities of LLMs

ChatGPT ha detto: ChatGPT The article explores how LLMs develop emergent abilities like few-shot prompting and chain-of-thought reasoning as they scale, highlighting key research findings and debates on predictability, potential misuse, and understanding their limitations.

Evaluation of LLMs - Part 2

Evaluation of LLMs - Part 2

The article explores using large language models (LLMs) as evaluators, addressing concerns about accuracy and inherent biases. It highlights the need for scalable meta-evaluation schemes and discusses fine-tuned evaluation models like Prometheus 13B, which aligns closely with human evaluators.

Evaluation of LLMs - Part 1

Evaluation of LLMs - Part 1

The article "Evaluation of LLMs - Part 1" delves into the rapid development of Large Language Models (LLMs) and the necessity for robust evaluation strategies. It examines traditional n-gram-based metrics like BLEU and ROUGE, discussing their roles and limitations in assessing LLM performance.