They are using Google reviews as proxy for customer satisfaction. I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
Sure, 99% satisfaction rate is pretty much impossible, but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.
I'm quoted in the post. We're using Google reviews as one of several imperfect comparison points to Apple's 99% claim. You're absolutely right about a key downside to Google reviews. The limited information 451 provides on their survey methodology shows that they aren't asking about satisfaction as a yes-or-no style question.
If you want to go really deep on online review manipulation, I'm co-author of this research paper on ratings inflation in online marketplaces: https://pubsonline.informs.org/doi/abs/10.1287/mksc.2022.135.... If you get in touch, I can send you more papers on fake reviews that I like.
> America’s favorite holiday, Thanksgiving, enjoys 82% popularity according to a 2023 survey by Statista. Memorial Day comes in second at 77%.
> The three most popular movies of all time on IMDB are The Shawshank Redemption, The Godfather and The Dark Knight. They have ratings of 9.2, 9.2 and 9.0 respectively.
These examples are so orange and apples it’s baffling the authors even came up with them.
You compare the most advanced personal pocket computer of all time with a patriotic holiday (assuming patriotism as an absolute value) or a movie about inmates escaping a prison.
Like, really?!
Better would be to ask everybody who watched Shawshank Redemption: ‘were you satisfied with the movie?’. A 9.2 on imdb is not equal to 92% satisfaction due to granularity in the answer options
> I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
> but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.
"and only surveyed iPhone 14 owners"
451 Research's own statement on their survey mentions _nothing_ about owning the phone:
> The findings are based on a December 14th – January 22nd survey of 3,568 primarily North American consumers from 451 Research’s user insight service, Voice of the Connected User Landscape and its Leading Indicator panel of 25,000 accredited business and technology professionals whose application for inclusion identifies them as having a high wallet share being used for personal technologies, and a high readiness to try new products and services. The service captures consumer and business spending through weekly tracking studies, delivering a continuous view of user perceptions and purchase activity as new products and services enter the market.
And also, how valid would you consider a "satisfied" rating from an iPhone 14 owner who'd never owned anything but an iPhone? There's value in their ongoing satisfaction, but relatively speaking what does it mean? It could just as easily be interpreted as "not annoyed enough to change ecosystems".
> And also, how valid would you consider a "satisfied" rating from an iPhone 14 owner who'd never owned anything but an iPhone?
Why wouldn’t it be valid? If somebody says they’re satisfied with something, take them at their word. There’s no shortage of people who complain about anything and everything, especially if they paid money for the thing in question, and especially if they paid a lot of money.
Apple is a single vendor that makes a very small number of models and earned $205.49 billion on iPhones alone in 2022. This excludes a lot of their “services” revenue of which I’m convinced the Lion’s share and then some is also basically just iPhone revenue that wouldn’t exist without specifically the iPhones tied to them. You don’t get anywhere near that for one company on one hardware product division in one year without some seriously high customer satisfaction and repeat buyers.
Yeah, especially when you consider that Apple likely uses NetPromoter as its primary metric for customer satisfaction. NetPromoter only sends out surveys to verified purchasers and the question is a 1-10 scale for satisfaction. Anything 8-10 is “satisfactory”. This is not only plausible but likely and is why the rating is so high. There wouldn’t be very many anomalous responses of people not understanding the question because the question is literally “Would you recommend the iPhone (whatever model) to friends and family?”. That’s not a hard question to understand.
> Apple likely uses NetPromoter as its primary metric for customer satisfaction
The article is entirely about a completely different firm, 451 Research, who actually is not and does not use NetPromoter. They state that they use a pool of "high wallet spend early adopters and technical people".
(For added irony, 451 Research says it's own customers are 98%+ satisfied with them...)
Apple may use NetPromoter internally, but on the earnings calls they are citing satisfaction surveys from 451 Research.
In 2018, they shared some of the response categories from the survey. They are customer satisfaction questions "Very Satisfied, Somewhat satisfied." Not recommendation questions.
Hey - I'm the author of the post. Thanks for reading.
I included the Google reviews as a secondary example since it is public information. Google reviews are actually more positive on the iPhone than the third party surveys, which found satisfaction rates in the low 80s for 5G iPhones.
> A sample limited to accredited business and technology professionals who consider themselves early adopters and spend a lot on gadgets is, of course, not representative of iPhone purchasers generally.
1. Granurality: as OC mentioned, w/ an overall satisfaction "yes or no" method can reach there.
2. Bias: Pretty sure they did the research before releasing the product to get feedback from a loyal subset base/target customers.
Why would a study include non-Apple users if they have never used or refuse to use Apple products? Wouldn't this be biased if they are included to be "representative"?
I know people who don't use Apple products yet are vocal critics...humans can be highly ignorant creatures
They mention in TFA that even surveying anomalies (people accidentally answering negatively) can’t be represented by a 99% figure.
That said, if I had a survey that asked a couple times in different ways if someone was satisfied with the iPhone and they gave conflicting answers, I’d probably simply throw their answer out for being nonsensical. Maybe that’s what’s going on here: they’re teasing out survey anomalies.
I'm the post author. I suspect aside from conducting extremely unrepresentative surveys, they are also aggressively filtering out respondents. I didn't include that since it is purely speculative on my part, but I'd guess that is part of what is happening.
I'd note that other seemingly innocuous things could have an impact too. For instance, if they surveyed only people who've owned their phone for 90 days or whatnot, you would eliminate the vast majority of people who didn't like it and returned it from the sample.
99% is an extraordinary claim, which is why we think Apple should be more transparent about their methodology.
Thanks for weighing in and your decisions in writing the article make sense to me.
I think it’s important (and probably simply ethical) for people and companies who make such extraordinary claims to explain how they obtained them, but having taken a stats class in college and doing a lot of growth-related work in my professional career I know how frustrating it is to get useful data, especially out of surveys.
> I really can't take rest of the article seriously after that, knowing anyone can leave a review at Google and there's no guarantee they own the phone.
Do you mean you can't take Apple seriously? The article is calling out this issue, not engaging in it.
> I can easily see the number being 99%.
No, it can't. It's utterly impossible for it to be 99% unless it's fake data.
iPhone has been around for 15 years so there almost isn't a person on this planet who doesn't know (a) what it is, (b) what the competition is and (c) whether it's a right fit for them.
So if you're surveying existing users you're talking to people who knew before they purchased the iPhone that it was going to be a suitable fit for them. So of course the survey is going to be highly biased in favour of a positive rating.
It seems very obvious to me that a 99% satisfaction rating for an electronics product is extremely unlikely, even given the factors that you noted. Not impossible like the other poster is indicating, obviously it's impossible to prove that claim, but very unlikely.
If you reflect on it and genuinely believe that 99% seems realistic and even "very likely", well, we'll have to agree to disagree.
People buy things all the time that are not a perfect fit, just the best they are able to get at that time. Or they are forced into it for other reason.
99% is not possible. Nothing on Earth has a 99% satisfaction rating - not even breathing.
Do you think that (even if that was the survey) it is an honest survey? Do you not care about the richest company in the world being so blatantly dishonest?
So obviously 99% is impossibly high, but iPhones (and well made flagship phones in general) being some of the most popular things ever made wouldn’t be very surprising to me. People’s lives revolve around their phones. They’re probably people’s most used item after their bed. And for the most part they work really well. That 20% of people don’t like Thanksgiving and the Shawshank redemption isn’t surprising to me, and I don’t think that means the iPhone couldn’t have higher than 90% approval rating.
(Especially since a lot of people who don’t like iPhones would have gone to Androids at this point anyway, so current iPhone customers is a somewhat self selecting demographic)
My thinking is maybe iPhone 14 customers are satisfied by it, but I'm satisfied by my phone, not because it's a great phone but because any modern phone has the bare minimum I need to get satisfied by it.
Those people aren't using other phones so they wouldn't know if they'd be satisfied by them.
> I don’t think that means the iPhone couldn’t have higher than 90% approval rating.
I would be surprised if iPhone didn't have 90%+ customer satisfaction narrowly scoped to usage of the phone.
The product as hardware combined with software and supporting services are the best money can buy. Apple regularly pushes the smartphone category forward on fronts neglected by other companies.
That said, consumers seem to think more broadly about their experience than just the phone. For example an unexpected failure of an official or 3rd party case might influence overall satisfaction. The article mentions a 3% difference based on 5G availability, as another.
So without an explanation of the carve out, 99% does seem a bit too good to be true.
Apple should publish how these numbers are arrived at. If its a decent measurement it could help set an evolving standard for the category.
> If Apple’s claims of 99% satisfaction were true, it would mean the iPhone 14 was not only the most popular mass market consumer product of all time, but it would also probably make it the most popular anything of all time.
Is it not the most popular mass market consumer product of all time? People spend their lives on their phone these days, and I've never heard anyone say anything about their iPhone. I have one and it's fine. I'm satisfied. I have end-to-end encrypted backups, the battery lasts long enough, the screen is nice to look at, etc. If I like some sort of tech product, it's probably hard to find people that don't.
I'm quoted in the post. I agree, it is plausible that iPhones have a 90%+ satisfaction rate. Apple's claim isn't 90%, or 92% or even 95%, but 99%. I agree that 99% is impossibly high.
> Trader Joe’s fans will stand in line for hours for groceries but still give the store just an 84% satisfaction rating, according to a survey by the American Customer Satisfaction Institute.
It’s tough competition, but I think this is the most absurd sentence in the article. Nevermind the difference between a premium consumer good and a grocery store, my takeaway is that reported satisfaction could be much higher among TJ shoppers without the hours long waits.
Apple's Protect Mail Activity feature, which is on by default, "opens" all emails received on a server, to hide the user's IP address. This causes emails with tracker pixels to be falsely reported as read.
For a while, I was confused why all of my GitHub notifications were being marked as read!
Later on they said they use Mailtrack [0]. I looked at their FAQ and they just use a tracking pixel. I don't really think that mail clients in a coorporate environment (i.e. Apple) download external images automatically. Mailtrack seems to be confident, that only 5-10% of recipients aren't "trackable" [1], but I haven't seen a mail client that by default downloads external media in a while.
See my peer comment: Apple Mail really does download external images automatically, on a server to hide your IP address. It then rewrites those emails to reference the cached images from the server. Every tracker pixel sent to a mail account with this feature will be counted as read, whether or not the user opens it.
Ah I didn't see that one, they mention this on another page [0] (scroll a bit down). But they mention also that it still reliably detects views and that google released a new feature a while ago "dynamic email", which automatically loads but can also be disabled.
Maybe my knowledge about Mail clients is a bit skewed, because I'm using open source Mail clients on every platform (Thunderbird, K-9 Mail, Evolution), and they always block External content by default. But I thought this was the standard today, but I guess loading them on the server is a bit more UX friendly instead of blocking remote content, although I wouldn't want it.
I am totally out of the loop when it comes to these things. My client always opens in plain text or html render with JS and external content disabled.
I believe this is true for most clients. On the other hand security scanners, browsers engines and other Middleware stuff can trigger such a counter without any user interaction if I am not mistaken.
I have received Apple satisfaction surveys before and the options are basically this:
How satisfied are you with your new iPhone?
* Insanely satisfied
* Extremely satisfied
* Very very satisfied
* Quite satisfied
* Satisfied
* Mostly satisfied
* Somewhat satisfied
* Neither satisfied or dissatisfied
* Neutral
* No opinion
* Dissatisfied
I’m exaggerating a little to make my point, but I suspect that’s more or less how they get these insanely high scores. Also I’m not sure they send the satisfaction surveys to everyone. Probably you have to have a history of purchasing Apple products.
I don’t think they are manipulating the data itself as this article seems to suggest though. I think the survey methodology itself is skewed.
Thank you for reading! I don't think they are manipulating the data, rather that they are surveying an unrepresentative population of tech enthusiasts, and presenting it as if representative of the entire customer base.
They don’t present it as if it’s representative of the entire base, nor do they ever compare it to anyone else’s metrics, and your claim is based on information openly disclosed about how they create their metric.
The piece is just clickbait.
Your final paragraph is this decontextualized “Quote”:
> “We’re not calling on the SEC to investigate these customer satisfaction claims, since they have more important cases to go after,” Golden said. “But extraordinary claims require extraordinary evidence, and we believe Apple should review the survey results they trumpet on every earnings call. If they truly think they are valid, they should publish their methodology, and if not, they should issue retractions.”
Which is inherently disingenuous in the way it is written. (We’re not threatening you..) but worse, it’s a quote from your own CEO.
The piece is in far worse faith than anything Apple is doing. Please take it down.
It's one of multiple quotes from the same CEO, who is also posting in this comment section saying "I'm quoted in the post" as though having a blog post on your own company's website where someone you pay does what you told them gives you more credibility
I can't remember what it's called (Edit: Lizardman's Constant) but around 4 to 20% of people will give nonsense replies to any survey.
Like if you asked Are you Human? 4% (or more) of people will reply "No" just for fun, or because they don't read.
So you can try to adjust for this by asking a silly question, and if they reply that way ignore their entire survey, but that could lead to issues where they reply correctly on most of it, but false on the "fun" question.
I've yet to use a single iphone that I was satisfied with.
My last experience with an iphone was when my GPS began to malfunction and Apple's proposed solution was to have me buy a brand new iphone at no discount.
For what alternative? My 14 Pro is way too big, and the camera software is meh, but the lower models have really nerfed cameras and I'm too entangled in the Apple ecosystem to switch to Android even if there were better options. 80% satisfaction seems reasonable, most people seem to like the big phones, and 20% of people with some major complaint or other sounds plausible.
Prime Day I bought my first iPhone, iPhone 14, with a good discount and cashback, I was between it and Zenfone 9. Not sure what to expect in general to be honest, but I expect a good battery life and updates, these were two factors that contributed to my final choice.
I'll sure miss sideloading certain apps, but that's the price for a walled garden and security (I guess?).
The original title, "Apple claims 99% iPhone 14 satisfaction. Is that possible? (No). We investigate" (which this was submitted under originally) is less click-baity than the current version: "Is Apple making implausible iPhone satisfaction claims?"
I pinged dang to ask him to change the title to match the title of the actual article, because it was obvious from the rate of upvotes and the quality of comments that the title as submitted was inflamatory.
I can't measure clicks, but the pace of upvotes on this submission slowed dramatically after dang changed it: it had reached ~50 upvotes in about half an hour (~100 upvotes an hour), and now it's been here for another hour with only ~30 more upvotes (~15 upvotes an hour).
It’s not a survey of the general public, though. That’s why the OP article is nonsense. This is most-likely a NetPromoter survey. It is a survey of verified iPhone purchasers/owners just after the 14 day return window who decided to keep it. It’s self-selective but it’s not unattainable like the article claims.
Any situation where a company can be put in a better position with statistics will 99% of the time lead to the company hiring a statistics firm that favours them.
Is there a legal limit how much you can fake the numbers before you are held accountable? Seems like Apple is pulling a lot of numbers out of their rear end. See also LTT's recent video about Apple quite clearly faking Mac Studio performance numbers: https://www.youtube.com/watch?v=buLyy7x2dcQ
At what age is it reasonable to assume that a modern western consumer understands that Marketing routinely uses some torturously contorted standards for "truth"?
Family lore: My kid brother was ~5 years old when he bit into a spoonful of a breakfast cereal that he had asked for after having seen it advertised - and unhappily blurted out "it tasted good on TV!".
In short, it looks like potential-respondents first had to apply (e.g., in response to [this Facebook post](https://www.facebook.com/Wristly.co/posts/apple-watch-owner-... )), and then they pre-screened potential-respondents.. and it sounds like respondents may have had some sort of a weekly commitment?, as the document includes:
> To finish, a big thank you to the 1,100+ strong members of the Wristly Inner Circle- we wouldn’t be able to learn so much so fast about the Apple Watch without your weekly contribution.
So it doesn't sound like a random sampling of Apple-product users.
Anyway, then apparently they ask if a respondent's satisfied with a product, giving 4 options: "Very Satisfied/Delighted", "Somewhat Satisfied","Neither Satisfied or Dissatisfied", and "Somewhat Dissatisfied".
Then, they add up the first 2 categories as their "key metric of customer satisfaction".
> When it came to overall customer satisfaction, iPhone X owners in our study gave the product an overall 97% customer satisfaction. [...] Just to contrast that with the original Apple Watch research with Wristly I was involved in, 66% of Apple Watch owners indicated they were very satisfied with Apple Watch, a product which also ranked a 97% customer satisfaction number in the first Apple Watch study we did.
Point being that, while the above PDF appears to talk about the Apple Watch, it sounds like they're strongly implying that the customer-satisfaction figures for the iPhone were largely done in a similar manner -- and, at least in the case of the above-quoted speaker, by some of the same people.
> Hey - I'm the post author. Thank you sharing these details on the Watch survey. I wonder if 451 Research is doing something similar with their iPhone survey.
Isn't that something you should've researched before writing the post? Methodology seems like an important factor here.
I'm curious about the stat of the email being opened 73 times. I'm wondering if someone at apple who has access to that contact email address has the option "Protect Mail Activity" turned off on their Apple Mail client or if this is a bug.
Seem to be a common feature on most phones. Don’t like it? Bug your government to outlaw the use of the feature like “Canada,[5] Chile,[6] China, Israel,[7] and Singapore”.
iTunes backups my photos. No idea how to access them though - apart from restoring to a working phone.
> iTunes backups my photos. No idea how to access them though - apart from restoring to a working phone.
You should be able to go to iCloud.com and log in. If you have the E2E encryption enabled, it will ask you to give iCloud the decryption key, so the webserver can serve you the decrypted files. (This ability can be turned off if you're really paranoid.)
that all has worked in the US for 15 years or more, i'm one who has done that at least as far back as 18 years ago, though it might not be what everyone does.
possibly one can get a phone subsidized and locked until an extortion fee is paid, though, or get a phone from work that an employer didn't pay to have unlocked.
I think it is, but it's not like you can't get unlocked iPhones in the US. I think carriers also are required to unlock your phone if you ask them, but I may be imagining that.
Give me an iPhone 5 that’s fast enough to run an os that has all the software bloat we have today. Then, stop adding more bloat to force us to buy newer hardware
What are you talking about? Computers are thousands of times faster than they used to be, but performance is like half. Look up videos of people running old operating systems on old hardware, how everything happens instantly, even without ssds. The only improvement we've had is in games and web apps, and web apps are a mess
There are people who practically worship apple. A new iphone could ship broken and a certain percentage of buyers would praise it as being brilliant. They'll never be unsatisfied with an apple product. There's another group whose satisfaction has nothing to do with the phone itself and everything to do with knowing that it was new and highly expensive and out of reach for many. As long as they can pull it out of their pocket and show it off they'll be happy.
Adding to that, people already have a tendency to convince themselves after the fact that an expensive purchase was a good deal, and that companies pay for fake reviews and I'm not surprised iphone reviews seem inflated.
Sure, 99% satisfaction rate is pretty much impossible, but if all Apple asked was "are you satisfied with iPhone 14" with "yes" & "not at all" options and only surveyed iPhone 14 owners, I can easily see the number being 99%.