Generative AI Is Having Its Herbalife Moment

How predatory vibe coding startups are selling false hope to young people

Photo by Hitesh Choudhary on Unsplash

Over the past few weeks, I’ve noticed that TikTok has been serving me ads for Replit — one of the many vibe-coding startups that have emerged in the past couple of years, and that serve as a glorified wrapper for models from Anthropic, OpenAI, and Google.

Now, TikTok can be a weird place at times. It’s full of companies trying to sell things to an audience that can not — and will not — ever buy them. Things like industrial-grade glycine, graphite cubes, and lightly-used oil tankers.

I usually just laugh at those ads and move on with my day. And, in comparison, Replit isn’t as outlandishly batshit as the idea of someone trying to sell a graphite cube — the kind that looks like it might feel at home in a Soviet RBMK nuclear reactor — to a bunch of hyper-online gen-Z.

The ads — many of which were produced by influencers with brand deals, and can be found by searching “replit” and “#ad,” or #replitpartner — typically follow the same format. You have a beautiful person going about their day, having an idea, and then prompting it into existence with Replit.

The message is simple — you, yes you, can make software, and that software might make your life some level of better, or more convenient.

Or, that software might become the side-hustle that helps pay your rent, or even, if you’re lucky enough, makes you rich.

Where have we heard this before?

History Rhymes

Around the 1920s, the world was introduced to the noxious and hateful idea of multi-level marketing. The way it works is simple — a company will have a product, like health supplements or plastic food containers, and they’ll get ordinary people to act as their marketers and salespeople.

There’s very little money in actually selling product. If you want to make real cash, you need to recruit other salespeople. Every new recruit — often referred to as the “downline” typically brings in a one-time bonus, as well as a cut of every bit of product they sell.

Obviously, those downlines now need their own downlines. And those third-generation downlines need their own downlines.

Obviously, this isn’t sustainable. Eventually, you run out of people. Kind of like a pyramid scheme.

Over the following century, the multi-level marketing industry changed. It globalized. We saw the emergence of new MLM companies spruiking wellness drinks with dubious health benefits, shampoos that make you go bald, and more. But while the product (which was never the point) changed, the basic model didn’t. Neither did the incentives for actually joining them.

Have you noticed that it’s never rich people that join MLM schemes? It’s always those on the bones of their arse. It’s why a lot of the people joining them are (at least, in the US) undocumented migrants who are otherwise locked out of the formal economy.

Nobody joins an MLM because they’re passionate about fucking tupperware or protein shakes, or the company manufacturing them. They join them because they believe that if they work hard enough, they can find the economic stability that otherwise eluded them, and perhaps something more.

Of course, only a small percentage of people who actually join them actually achieve that economic stability, and a smaller number still that “something more.”

So long as times are tough — that there’s people who feel alienated from the economic system — there will be people looking to take advantage of that alienation.

Jesus, just look at the crypto boom of the 2010s. I had a front-row seat for much of it, with my time at The Next Web coinciding with the frothiest, dumbest part of the era. Each day, I’d get around 250 pitches from founders and PR people in my inbox, all asking me to write about their clients or companies, and I’d wager that around 200 of them (or four-fifths) had something to do with crypto.

Tell me, do you think that the crypto boom would have happened if not for the fact that people saw a material decline in their economic fortunes and living standards after the global financial crisis, and that they didn’t see any hope of that changing?

FTX pitched itself to a mainstream audience, buying the naming rights to sports stadiums and spending big on a marketing campaign that targeted normal people, with Superbowl ads fronted with recognizable celebrities like Larry David.

Do you think that mainstream appeal would have been there — or would have been quite as strong — if not for the fact that people were broke, with their spending power eroded year-after-year as inflation chipped away at their stagnant wages? Would Paris Hilton have been invited to talk about NFTs on the Jimmy Fallon show, if not for the fact that the dire economic situation had created an opening for NFTs in the first place?

The Vibe Coding Scam

Things are fucked. More fucked than they’ve been in a long, long time.

What makes this moment even more dangerous than, say, the pandemic era, or the period after the global financial crisis, is that at least people believed that this too shall pass. That with vaccines and natural immunity, and sensible policy decisions, life will return to normal. That eventually, the banks will right themselves.

Right now, people — especially young people — are looking down the barrel of a stagnant job market that has decided it doesn’t need them. While AI is often touted as the excuse, it’s often just a cover for another simpler reason, like cutting costs.

Whatever. The fact is, the belief that AI can — and will — displace white-collar jobs is a lie that’s been accepted by the masses, in part because of the huckster-like triangulations of scumbags like Sam Altman and Dario Amodei, and in part because credulous fuckwits in the media have repeated them verbatim.

And I am concerned that the fear I’ve described is being exploited by companies like Replit and Cursor (which is also doing the exact same influencer marketing schtick, albeit not as aggressively as Replit), who are touting their services as a way for people to escape the precariousness of this current moment.

First, the idea that we’re about to see the emergence of a thriving cottage industry of apps, all created and marketed by non-coders is fucking ludicrous. Forgive the causeness of my language, but I have to be blunt.

Vibe-coded software is simply not good. Let’s suppose that someone deploys an app and there’s a critical security vulnerability that allows a threat actor to, say, exfiltrate all their customer information. How would they know? And if they became aware of it (presumably because said threat actor exploited said vulnerability), how would they fix it?

Also, would the person who developed the app know that, under legislation like GDPR, they can be financially liable for data breaches? Because they would be! And the whole point of the financial penalty system (at least, with respect to GDPR) is to be dissuasive — to act as a deterrent to other people who would be cavalier with other people’s data.

I can very easily imagine a national data protection authority — like the UK’s ICO — giving someone a massive, massive fine in order to dissuade other people from deploying their own AI-generated, unvetted slop code.

And then there’s the cost of actually building software.

Replit’s business model allows customers to buy one of two subscriptions, each providing a certain number of credits. When you run out, you can either buy more credits, or simply rack up additional charges and pay at the end of the billing cycle.

It’s entirely possible that someone will try to build their dream app, forget to cap their costs, and end up with a bill of hundreds or thousands of dollars. And no, that’s not an exaggeration.

I do not see how this is any different than, say, someone buying a starter kit from an MLM for a few grand. Both MLMs and vibe-coding startups that are marketing to consumers are charging their victims an upfront price, without making any guarantees that the investment will pay off.

Actually, that’s a lie. There is one important difference.

At least when you spunk cash on a starter kit for Herbalife, you know upfront how much it’s going to cost. By contrast, it is impossible how many tokens a LLM will burn when performing a particular coding task.

You simply cannot predict how much a certain action will cost — or whether the LLM will execute the task correctly, or whether you’ll need to re-prompt the model, and how much that second (or third, or fourth, or fifth) prompt will cost. You do not know whether one of those prompts will get stuck in a loop, burning tokens — and thus money — with nothing to show for it.

I’ll add that none of the TikTok adverts I’ve seen have mentioned the cost of compute. I’d wager that’s because if they were truthful about the costs of vibe coding, and the limitations of the technology, it would be a much harder sell.

Let’s imagine that someone — a non-coder introduced to Replit through TikTok — actually did build something. Realistically, how are they going to monetize that app? How are they going to scale it? How are they going to bring in customers?

I know I sound cynical, but I’ve spent the past decade-and-a-half reading Hacker News, with each day bringing a new “Show HN” post where someone announces their new app or website or service or whatever, and most of those have since faded into the ether, with the only evidence they ever existed being that introductory announcement.

Hell, as a journalist, I’ve been pitched tens of thousands of stories of the years from companies that no longer exist.

Building a tech company is hard! Even when you are a coder! And have capital! And a team of VCs backing you, each bringing their own technical and business expertise. For the vast majority of people, the odds of being a successful founder are as good as them being a professional soccer player.

It’s a dream — and it’s predatory to sell that dream to people through the medium of fucking TikTok adverts.

Everyone Involved In This Scam Should Be Ashamed Of Themselves

Incidentally, while I was finishing this newsletter, I decided to check the most recent posts on the Replit subreddit, only to find that someone was complaining about how a friend of their sister was being paid to promote Replit by falsely claiming that they “make 10k at month from home,” and that they landed their first tech job because of their Replit-made app.

I do not know how truthful that poster is, but I certainly found it to be an interesting coincidence.

And, honestly, it doesn’t matter.

I believe that vibe coding — irrespective of whether it’s useful for enterprises, which I doubt — is being marketed towards consumers in a deeply unethical way. One that’s worryingly reminiscent of multi-level marketing schemes like Herbalife and Amway, or the crypto grifts of the 2010s.

I believe that neither Replit, nor those posting sponsored content on behalf of Replit, are being candid about the costs of vibe-coding a business, or the likelihood of actually building a successful tech product with no technical expertise.

And I’m worried that this campaign will be successful in convincing many to part with their money, in the same way that other similar scams have thrived during hard times.

I believe that Replit’s decision to target younger people at a time when they’re struggling to find work, or are convinced that the future workplace has no use for them, is deeply predatory.

Any creator that promotes Replit without being transparent about the likelihood of building a million-dollar app, or about the costs of building software with AI, is either willingly complicit in a cynical, harmful scam, or otherwise promoting a technology that they themselves do not understand.

I ultimately believe that the weight of the blame lies on the shoulders of those within Replit who greenlit and funded this marketing campaign, and who knew exactly what they were doing.