Releasing hardware and development tools for artificial intelligence like US$3 trillion company Nvidia does is a completely different thing to being at the coalface of producing actually useful examples of the technology to wow users.
This particularly so for generative AI, a field in which both Google and Microsoft have recently stepped on launch landmines as the technology and its implementations can produce unpredictable results and consequences.
Now it’s Apple’s turn to have a crack at GenAI, at its annual Worldwide Developer Conference 2024 in Cupertino, California. If you’re the impatient sort, it’s sad trombone time because despite developer betas (early previews of software for bug and feature testing), we’ll have to wait a few more months to actually try out the new technology. That’s when we’ll get new betas that incorporate the technology.
Apple is no doubt aware of the risks that large language models for AI bring, like “hallucinations” or making up results that are either subtly wrong which can be hard to spot, or completely insane. Given the AI will be embedded deep into premium products destined for some of the most demanding and toughest users there are, it’s not inconceivable that Apple feels just a tad nervous about the whole thing.
In typical Apple style, the WWDC presentation of the tech, named Apple Intelligence, boldly co-opting the AI acronym, was super slick. Long story short, GenAI will be embedded in the operating systems (that’s the software that runs computers, and provides various tools and utilities for them) for iPhones, iPads and Macs.
Starting with iOS 18 (iPhones), iPadOS 18 (iPads) and macOS 15 Sequoia (Macs), users will have AI and use machine learning for automation and assistance.
Some caveats: at first, the AI will be in US English only 😬, and it’ll only run on new hardware, namely:
- Phone 15 Pro and Pro Max with the A17 Pro chip set.
- iPad Pro and iPad Air with M1 and later.
- MacBook Air and MacBook Pro with M1 and later
- iMac M1 and later.
- Mac mini M1 and later.
- Mac Studio M1 Max and later.
- Mac Pro M2 Ultra.
Apple isn’t a newcomer to AI. The company has put in specialised go-faster hardware into its in-house designed systems-on-chips since 2017, the Neural Engine accelerators which you might have used for cutting out subjects from images, and recognising text in pictures to give a couple of examples.
Nevertheless, Apple which normally prefers going on its own, having been burnt by technology partners in the past, hasn’t done so this time: Microsoft backed OpenAI and its ChatGPT is as expected a prominent player in Apple’s AI game. OpenAI’s reinstated chief executive Sam Altman seemed pleased about that, in all lower case as is his wont.
very happy to be partnering with apple to integrate chatgpt into their devices later this year!
— Sam Altman (@sama) June 10, 2024
think you will really like it.
This is what ChatGPT will do, as cribbed from Apple’s media release:
Apple is integrating ChatGPT access into experiences within iOS 18, iPadOS 18, and macOS Sequoia, allowing users to access its expertise — as well as its image- and document-understanding capabilities — without needing to jump between tools.
Siri can tap into ChatGPT’s expertise when helpful. Users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.
Additionally, ChatGPT will be available in Apple’s systemwide Writing Tools, which help users generate content for anything they are writing about. With Compose, users can also access ChatGPT image tools to generate images in a wide variety of styles to complement what they are writing.
Privacy protections are built in for users who access ChatGPT — their IP addresses are obscured, and OpenAI won’t store requests. ChatGPT’s data-use policies apply for users who choose to connect their account.
ChatGPT will come to iOS 18, iPadOS 18, and macOS Sequoia later this year, powered by GPT-4o. Users can access it for free without creating an account, and ChatGPT subscribers can connect their accounts and access paid features right from these experiences.
It does look like the privacy protections are robust, and that OpenAI won’t even be able to snag users’ metadata. Apple's philosophy here is that AI and ML will be useful with your own content, but that material ain't leaving your device and it's more than just a text box to type stuff into, like Google Gemini and OpenAI ChatGPT on the web.
Security researchers are very likely to have a go at ChatGPT enhanced Siri with prompt injection attacks, so let’s see how well Apple manages to defend against those.
One of the more impressive demos at WWDC of what AI can do involved handwriting recognition on an iPad. That is, the handwriting is recognised, tidied up, still looking like your scribblings, and fully editable. Copy and paste, spelling checking, proof reading etc. As someone whose handwriting is severely damaged from keyboarding much too much, I look forward to challenging the AI with my chicken scratchings.
Everyday apps like Mail, Notes, and Apple’s productivity programs like Pages, Numbers, Keynote will get generative Writing Tools support which might just drive non-US English speakers around the bend, but let’s see when they come out. Mail will also organise your inbox with AI, triaging messages and providing summaries of them. Safari will also have summaries and a redesigned Reader feature.
The Image Playground generator for pictures and animation with three styles for Notes and other apps looks potentially useful as a clipart replacement, but I dread to think what the text-to-emoji Genmojis will look like.
The AI announcement was chunky enough in terms of detail. Apple wasn’t content with that, and showed off heaps more changes and improvements to its operating systems at WWDC 2024. Here are some things that stuck to mind at 5 am:
Calculators on iPads! What won’t Apple think of next?
Yes, finally, after how many years, etc: a calculator app for iPads. The actual reason why there’s not been one is lost in the swirling mists of time (rumour has it that Steve Jobs thought the iPhone version ported to iPads looked awful and nixed it), but iPadOS 18 fixes that.
It was worth the wait. The iPadOS calculator app has the usual basic and scientific modes, but the Maths Notes that’s part of it is something else. This works with the Notes app, and lets you handwrite formulae and equations that are then calculated in real time.
Watch that health data and your fitness
Not much AI as such for the Apple Watch, but the watchOS 11 update has a slew of new features for health and fitness tracking that look good. This includes a new Vitals app to pick up health metrics when you wear the Watch at sleep, and improved pregnancy support. Will be great to try out, although maybe not the pregnancy stuff because kids are such an expensive hobby.
Hi res virtual screens but no Vision Pro for NZ
The visionOS 2 upgrade brings support for two 4K virtual screens that wrap around the user, if you have a Vision Pro headset that is. Which is not only very expensive, but also not going to be available in New Zealand although the Aussies will have it from the end of June.
Photos fix up
This looks like a good one, as anything that can tame and organise multi-gigabyte Photos libraries with pictures and videos is badly needed. On-device AI promises to help find pictures as well, in the redesigned Photos which has had what Apple says is its biggest-ever redesign.
Passwords get their own app
Right, passwords are meant to be dead but on the updated Apple operating systems, they get to live in their own app. It looks tidy and useful.
iMessages over satellites and RCS texting to the non-iPhone crowd
With an iPhone 14 and newer device, the Messages app can now hook up to satellite service when there’s no terrestrial mobile or Wi-Fi connections for the phone. Messages sent that way are end-to-end encrypted for privacy, and you can send and receive texts, emojis, and tapbacks.
Due to pressure from European Union regulators, Apple has had to introduce interoperability with the Rich Communications Services (RCS) standard. As the name suggests, RCS allows for fancier messages than ye olde Short Messaging Service (SMS) texts. If you thought that meant an end to the Blue vs Green Bubble Wars, sorry, it doesn’t. RCS messages are still green bubbles, but you can see in the text field now which type of service you’re using.
Tap to cash
Nothing to do with loud dancing, instead it's a new feature for the iOS Wallet that lets you send money to friends by tapping phones. Looks dead simple to use, but it's for the US only currently.
11 Comments
To what? The other main players all offer some form of AI in their OS. Apple provides the best development platform which in turn leads to better third party products.
3 trillion worth, most popular brand with entry devices costing 1k plus. I think they’ll be ok missing out on buyers from some of the commentators on interest.co
Apple has been using ML and AI for a very long time. Siri, predictive text, Apple Music etc etc. This is nothing new, if anything it's a signal to investors to get on the AI hype train. OpenAI and other LLM and FMs are far more performant and provide some very very clever services, so it's similar to outsourcing hardware rather than building their own, which they already do.
The real concern is what data is being sent to whom as part of this deal, and can you opt out? But that is becoming a concern with almost every piece of new software.
Just think how easy it will be for Governments and the like to disseminate misinformation and disinformation to half the people on the planet now (all of them presumably once Google does the same with its Android OS). You will only ever be told what those organisations in control of the information want you to believe.
Then there is the exercise of control over your behaviours. The next time you try to book a flight for a holiday, the AI will pop up and ask you if you really want to do that because your carbon miles will be xxx and you will be responsible for xxx amount of destruction of the rainforest and xxx amount of ocean water warming and ice melt. Are you still sure you want to go on holiday? And if you click yes, that information will be sent to the Govt so you can be charged a carbon tax and your social credit score reduced by an appropriate amount.
We welcome your comments below. If you are not already registered, please register to comment.
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.