Most of the groundbreaking results in the field of #MachineLearning that we have seen lately use enormous neural networks containing hundreds of millions of parameters. Training such a model is not only time-consuming but also has quite a negative environmental impact. Moreover, training #ChatGPT, for example, is estimated to have cost around 1 million dollars.
These issues make it impossible for an individual or even a smaller company to conduct research on such models, making the overall speed of technological advancement lower than it could be possible.
Therefore, at TechnoLynx, we believe that lowering the barrier of entry to these technologies is of utmost importance, which partly includes optimizing the training processes.
Luckily, research is being done on this topic, and we welcome #LiGO, a new method for learning by model growth. LiGO is shown to perform better than any previous model growth technique, let alone training from scratch. As a bonus, surprisingly, the network trained using LiGO is showing better results than traditional training techniques in many cases. In the article below, you can learn more about the way all this is achieved.
Full article here.