TLDR Tech / Dev
Petals
Petals enables users to run LLMs like Llama 3.1, Mixtral, Falcon, and BLOOM at home by using a peer-to-peer network to access model parts using consumer-grade GPUs or Google Colab. It offers flexibility in fine-tuning and sampling methods, allowing for custom model paths and insights into hidden states.
Read the full story