TLDR Tech / AI
HRM-Text
HRM-Text is a 1B text generation model based on the HRM architecture. It can be trained with 130-600x less compute and 150-900x less data than foundation models, making foundation model pretraining accessible. The 0.6B parameter version of the model can be trained on 8 H100s on a single node in about 50 hours for around $800. The 1B parameter model can be trained on 16 H100s on two nodes in about 46 hours for around $1,472.
Read the full story