Training and running AI models is compute-intensive and costly – and does not compensate creators fairly for their work. Can decentralized technology provide a solution?
Decentralized technology for AI models creation
Many people think of all digital services and processes as essentially “free” in terms of energy consumption when compared with their real-world equivalents, but nothing could be further from the truth. Fingers are often pointed at cryptocurrency networks (particularly those dependent on proof of work consensus mechanisms) for the electricity they use, but critics frequently overlook the fact that every digital process needs compute power, whether this is the TV show you watch on Netflix or your card payment in the supermarket. The question is not necessarily how much power these services consume, but whether this is a price worth paying for the utility the consumer gains from the service.
The advent of cloud computing has removed the need for entrepreneurs to set up energy-hungry data centres, but all it means is that the costs are shifted to paying the likes of Microsoft Azure, Amazon Web Services or Google Cloud Platform for their services OpenAI CEO Sam Altman has not revealed the exact cost of running ChatGPT, given the huge demand for its services, other than a tweet in which he described it as “eye-watering”. However, independent estimates have placed the cost at somewhere in the region of $100,000 per day – in other words, upwards of $3 million per month.
It is also worth bearing in mind that the estimated daily cost of running ChatGPT does not include the energy expended on training the model in the first place – an energy-intensive process of ingesting huge amounts of data – nor the cost of further training on subsequent datasets. It is thus not surprising that Altman stressed the urgent need to begin monetising OpenAI’s creation as soon as possible. We have seen similar trajectories with AI-generated image services, such as OpenAI’s DALL-E, which gained initial traction from social media and then introduced a credit-based service, and Midjourney, which allows 25 free images before charges kick in.
All the costs we have considered so far are those that are transparently borne by the company responsible for the software, but there have also been dissenting voices from those who feel that the unwitting contributors of the training data that helped create these models should be compensated for their work.
Developers who wrote the code that was used to train GitHub’s CoPilot, artists who created the images fed into Midjourney or DALL-E and writers whose novels or Reddit contributions were used as the raw material for ChatGPT have all expressed their unhappiness with the fact that their copyrighted work has been used for commercial purposes with no payment. Legal action is currently being taken in a landmark case.
It is not simply the use of copyrighted material that is contentious – but the potential risk factors associated with incorrect facts in the training data. While ChatGPT communicates in a fluent fashion, it often makes statements that are factually incorrect, simply because it has been trained with data that has not been screened for accuracy. This is a risk for any software that consumes user-generated content for its inputs: Amazon’s Echo device was criticized for suggesting a lethal challenge to a child, which could have resulted in their electrocution.
Getting the incentives right
The above examples show that LLMs consume large amounts of compute power and data, which are currently not correctly incentivized. Over time, as the use of AI stops being a hype-driven novelty and becomes simply another part of business models, one would expect the pricing of both compute resources and verifiable data to move closer to their optimal level for each business case.
The question is how this resource allocation will be decided. An interesting paper by by Konstantinos Sgantzos and Ian Grigg points out that “One of the biggest challenges in data science today is the collection of a proper dataset, which can be utilized for training a neural network” and concludes that “Blockchains are therefore not only ideal storage for the data of deep learning, they include much data worth analyzing, and they are ideal storage for the trained frameworks themselves. In time we expect the discrimination between good and bad data to become easier based on pseudonyms and incentive models.”
In their paper, they also highlight the potential of blockchain-based automated payments to allow for a cheap and effective pay-per-use mechanism, both for data and execution of queries.
Similarly, Trent McConaghy of Ocean Protocol highlighted the potential of blockchain technology within AI to provide the following benefits:
- Leads to more data, and therefore better models.
- Leads to qualitatively new data, and therefore qualitatively new models.
- Allows for shared control of AI training data & models.
Desiring a world where providers of data and creators of content are compensated fairly for their efforts, and where entrepreneurs and technologists are able to attract investment to keep pushing the boundaries of innovation is not simply a matter of altruism.
The direction of travel in AI development is crucial to the future of humanity.We need to have serious conversations about improving the incentive model for developing, training and running these LLMs and applications built on top of them. Some are already openly questioning whether it is appropriate to have large tech companies control LLMs and the datasets that are used to train them. There have been proposals for publicly funded projects to compete with Big Tech in this space. In this situation, one might question whether datasets provided or endorsed by state interests are any more neutral than those used by large corporations, and whether this would simply skew the incentives further.
Organizations such as OpenAI are already openly thinking about these issues and on February 16 published a blog post explaining their plans for improving the behaviour of their software, and outlining a strategy for getting more public input into how it works.
Much has been written about the dangers of AI, and its potential to become sentient, creating a Skynet-type scenario where humanity will be wiped out. The real danger is probably more likely to be that poorly incentivized models trained on substandard data could end up as a deterrent to the widespread adoption of beneficial AI, thus delaying the very real benefits of this technological innovation by years, if not decades. As Sgantzos and Grigg suggest in their paper, “Blockchain’s immutability constructs a fruitful environment for creating high quality, permanent and growing datasets for deep learning”, and thus the fusion of blockchain technology with AI could be well worth exploring in order to create a properly incentivized environment for the evolution of LLMs.
