Dates and Times: TBD
Location: TBD
Tuition: $9500
If you are trying to keep up with the flood of new developments in LLMs, this class is for you.
- Cover new LLM techniques like FlashAttention, FasterTransformer, Lion optimizer, ALiBi, and QLoRA
- Compare and discuss LLM benchmarks like TriviaQA and HumanEval
- Practice fine-tuning foundational models (7B-70B) on cloud hardware
- Discuss in-context learning methods like summarization, chain of thought reasoning, and scratchpad memory
- Use and compare various self-correction methods
- Compare A100, H100, GH200, and other hardware
- Study MPT, LLaMA2, and Platypus in detail
- Gain a deep understanding of perplexity and explainability
- Cover the latest RLHF techniques
- Develop an understanding of the impact context length, dimensionality, and floating point precision have on inference cost and accuracy
- Hands-on experience using synthetic data and discussion of resulting issues like model collapse
- Use FPGA hardware to experiment with modifying the Transformer architecture
- Learn how to estimate costs of training and inference
- Use all of the above to develop experiments to reduce inference costs and improve real world accuracy on problems like code generation
- The latest papers related to the topics above will be incorporated into the class
If you have any questions about the course, please do not hesitate to contact me.
