Here's a story from the "IT is different these days" department: graphics cards are now a massive business globally. The biggest name in the graphics business is American company Nvidia, which has grown into an enormous entity, valued as of writing at US$3.5 trillion.
Graphics cards started off as separate add-on boards with electronics that could accelerate video games, image work, computer assisted design and similar tasks. Also called graphics processing units (GPUs), they are used by crypto currency miners too.
What's really made them the hottest ticket in IT however, is artificial intelligence (AI).
Trans-Tasman IT services provider ASI Solutions shared a local perspective on the importance of GPUs for AI with Interest.co.nz. The company has partnered with global enterprise infrastructure firm VAST Data to create a secure and scalable GPU-as-a-Service (GPUaaS) offering for New Zealand businesses.
First, what is it about GPUs that make them so useful for AI? ASI Solutions country manager Lloyd Vickery explained that it is due to GPUs' very specialised design:
"GPUs excel at training AI models because they're engineered to handle numerous operations simultaneously, unlike CPUs [central processing units, the electronics that are the main brains of computers], which are optimised for sequential processing," Vickery said.
"This parallelism is crucial for AI tasks that require the simultaneous processing of vast amounts of data," he added.
In essence, it's the ability of GPUs to do many things roughly at the same time that makes them AI powerhouses.
"Imagine you have a massive library of books that need to be sorted. Using a CPU is like assigning the task to a single, highly skilled librarian who sorts each book one by one. In contrast, employing a GPU is akin to having an entire team of librarians, each handling multiple books at the same time," Vickery said.
"While the individual librarians (GPU cores) might not be as adept as the single expert (CPU core), their collective effort completes the sorting much faster due to their parallel approach," he said.
For AI model training, and statistical inference for them, having GPUs with thousands of smaller cores optimised for parallel workloads makes them particularly effective for concurrent computations, as opposed to CPUs that are designed for sequential tasks.
ASI's GPUaaS is a cloud platform, built together with CDC Data Centres, and local electronics distributor PB Technology. The company offers two types of GPUaaS, or instances as they're called in IT. The medium one is based on dual Nvidia L40S GPUs, whereas the small instance uses a single Nvidia L4 card.
The target market for the initial service offering are researchers, universities, the government and other organisations with data sovereignty requirements, Vickery explained.
Why use a cloud-based GPUaaS? Vickery said ASI's offering is hosted in data centres certified to run on 100 per cent renewable energy, and built to highest-grade international standards for reliability and energy, making the service more available and better for the planet.
"Organisations can make an AI model available to everyone in the company or to their customers. This makes it easier to share information across teams and securely give the AI access to customer data," Vickery said.
As a cloud-based offering, you can also scale the GPUaaS up and down according demand.
The AI boom shows few signs of abating, and there is plenty of demand for GPUs currently. Recently, the government of South Korea announced that it is looking to buy 10,000 GPUs for a national AI data centre.
That's a big number, but pales in comparison to other AI players such as Meta, whose chief executive Mark Zuckerberg in January last year said it wanted to buy 350,000 of Nvidia's top of the line H100 cards:
The United States recently severely tightened export restrictions on graphics cards as well. Furthermore, high-end Nvidia graphics cards with plenty of high-speed memory cost tens of thousands of dollars each, due to demand. Meta's graphics card order will run into billions of dollars.
Is it hard for ASI to find GPUs at the moment? Not in the amounts needed for NZ deployments, Vickery said.
"We're not finding any supply change issues at the moment. It might be hard to buy 10,000 cards but it's relatively easy to buy 10 or 100, which is more inline with NZ scales," he said.
Big AI deployments and using them for training, like overseas companies do, is also very expensive, and Vickery doesn't see the need for us to do so yet.
"I think having the ability to run open source AI models like Llama in country is important, especially as it gets easier to fine tune them for our specific requirements," he said.
"I don't as yet see the need for us to train our own models from scratch, this is incredibly expensive - currently in the billions of dollars to create a leading model. Better if we can leverage the investment from these companies spending billions, and then build on top of it to create value for New Zealand, Vickery added.
We welcome your comments below. If you are not already registered, please register to comment.
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.