Only self-supervision
The video explains two main types of self-supervised learning: mask language modeling and autoregressive language modeling.
Training large language models to reason in a continuous latent space – COCONUT Paper explained
COCONUT trains language models to reason in a continuous latent space, improving efficiency and flexibility.
Why do we need Tokenizers for LLMs?
The video discusses how to represent text as vectors for language models using tokenization to handle new words and typos.
REPA Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You ...
The video discusses a paper introducing the REPA loss term for diffusion models, which enhances their ability to learn general-purpose image representations by leveraging pretrained models like DINOv2, resulting in faster and more effective training.