Making LLMs Environmentally (And Budget) Friendly
Red Hat News, Wednesday, August 21st, 2024
It seems that any given large language model (LLM) gets larger and larger with each release. This requires a large number of GPUs to train the model and more resources are needed throughout the lifecycle of these models for fine tuning, inferencing and so on.
There's a new Moore's law for these LLMs: The model size (measured by numbers of parameters) is doubling in size every four months.
LLMs are costly
It's expensive in terms of resources, time and money to train and operate an LLM.Resource requirements have a direct impact on companies deploying an LLM-it doesn't matter whether it's on their own infrastructure or using a hyperscaler. And LLMs are getting even bigger.