Russian tech giant Yandex has launched a new open source tool to help AI companies save money and resources when training new models.
In its announcement, Yandex says that the new YaFSDP tool can provide faster and more efficient training of large language models (LLM), and can help save up to 20% of GPU resources, saving money and time.
Yandex also published resources for testing YaFSDP on GitHub so anyone can check it out and experiment with it.
How to train your LLM
Yandex lays out plenty of technical details in its announcement post, saying the new open source tool is “the most effective publicly available tool for improving GPU communication and reducing memory usage in LLM training,” and can deliver speeds up to 26% faster. than FSDP, the version before YaFSDP.
“LLM training is a time-consuming and resource-intensive process,” says Yandex. “Machine learning engineers and companies developing their own LLMs spend a lot of time and GPU resources (which equates to money) training these models. The larger the model, the greater the time and expense associated with it.” with his training.
The company estimates that using YaFSDP to train a model with 70 billion parameters can save the resources of approximately 150 GPUs, which translates to around $0.5 to $1.5 million (depending on the virtual GPU vendor or platform). in monthly savings, if all goes well.
Using Meta's Llama models, which have been praised for being ahead of the curve while embracing open source, Yandex says the “ultimate speedup” is 21% and 26% on Llama 2 70B and Llama 3 70B, respectively.
YaFSDP is not Yandex's first foray into AI tools, having launched CatBoost, YTsaurus, AQLM, and Petals, all of which handle a different niche within AI development.
Yandex's move comes after it had to divest its company following the Russian invasion of Ukraine, selling its domestic business to its former management in the country.
Many big tech companies are also making AI core to their products; More recently, Apple launched its Apple Intelligence services as part of the upcoming iOS 18 update.
