Advanced AI: LLM Research


- Cover new LLM techniques like FlashAttention, FasterTransformer, Lion optimizer, ALiBi, and QLoRA

- Compare and discuss LLM benchmarks like TriviaQA and HumanEval

- Practice fine-tuning foundational models (7B-70B) on cloud hardware

- Discuss in-context learning methods like summarization, chain of thought reasoning, and scratchpad memory

- Use and compare various self-correction methods

- Compare A100, H100, GH200, and other hardware

- Study MPT, LLaMA2, and Platypus in detail

- Gain a deep understanding of perplexity and explainability

- Cover the latest RLHF techniques

- Develop an understanding of the impact context length, dimensionality, and floating point precision have on inference cost and accuracy

- Hands-on experience using synthetic data and discussion of resulting issues like model collapse

- Use FPGA hardware to experiment with modifying the Transformer architecture

- Learn how to estimate costs of training and inference

- Use all of the above to develop experiments to reduce inference costs and improve real world accuracy on problems like code generation

- The latest papers related to the topics above will be incorporated into the class

Contact about the Course

If you have any questions about the course, please do not hesitate to contact me.