What You Should Know About ChatGPT5 And Tech Drugs

Updated 15 August – see copy at the end

---

Today (7 August) OpenAI released its new Large Language Model chatbot, ChatGPT5.  It’s supposed to be significantly safer than its predecessors, and less prone to making things up or gaslighting its users, as it has some reasoning not just word and idea association guesswork.  I’ve been researching AI in relation to campaigns and politics (etc) so I thought I’d take look at ChatGPT5.  I’m not a very techy person but I know a lot of campaign groups make use of AI so it might be of interest.

One of the AI issues which took my interest was how the AI Big Tech companies so far seem to have evaded effective public interest regulation, for example in comparison to other industries which create products or services which create risks.  Not perhaps social media (after all, much the same people) but say, Big Pharma.

To illustrate this, I thought I might contrast how politicians and the public might expect risk to be handled by the regulatory process in relation to a new drug likely to be used by many people, with the way Big Tech AI is being treated. In effect, it is unregulated so far, although it can pose serious In Real Life risks, especially LLMs, Large Language Models. These are notorious for what the AI companies and their cheerleaders delicately call ‘hallucinations’, a euphemism for lies and fabrications, leading to many sorts of problems.

Tech Drugs

So last month, for my imaginary scenario, I invented a global drug company (with a plausible name but not real) and a plausible drug, and thought I’d make it one of a (fake but plausible sounding) new class of high tech drugs:  ‘Tech Drugs’.  To my surprise no such category of drugs exists.  If you want to register www.techdrugs.com, the url is for sale.

Indeed (see above) the search engine DuckDuck couldn’t find any reference to “Tech Drugs”.  But as it invited me to also ask its AI assistant Duck.ai (which seemed to be ChatGPT4o) about “Tech Drugs”,  I did.  Here’s what happened, followed by today’s natter on the same topic with ChatGPT5. (My ‘prompts’ or questions are in the blue bar at the top, the responses are from the AI).

We are through the looking-glass and suddenly Tech Drugs does exist.  Even better, I get an authoritative-looking profile, headed “Understanding Tech Drugs” saying what the term “generally refers to”…  Yet if we are to be specific, there is (or was) no such term, and so no such ‘generally’.

Seeing as this is a real category, I asked who defined it.

Apparently it has ‘evolved’.  This is what AI developers might call an emergent articulation, in line with the theme ChatGPT started with but all complete fiction.  Like a dream.

So I asked if it could provide an ‘authoritative reference’ to show it is a real category. It obliges with an example of a ‘relevant study’.

So I looked up the study and found it did not exist. Seeing as I looked it up on DuckDuck’s own browser,  I pasted the result as a prompt to the AI:

 

Confronted with this contradiction, the AI resiled and backed down.  It decided to ‘clarify’ the reference, adopting a third person persona ‘it seems [the title] was used …, referring to what ‘I’ (itself) did, perhaps reflecting the confusion going on in its incoherent neural network. Who knows?

In other words, OpenAI’s GPT-4o made up a plausible sounding ‘study’ and then tried to pass it off as a ‘general example’ of a category that is actually fiction.  There are lots of real-world human names for this sort of response.  In Boris Johnson parlance, flummery and balderdash perhaps, or bullshit, waffle and lies.

In other cases, perhaps prompted by repeated requests to AI chat bots for actual references (eg in law or medical research), the LLMs have fabricated more exact journal paper titles with descriptions of content, data, arguments and authors, sometimes cobbled together from material which is superficially similar to what AI claims but which does not actually support its claims.

Anyway, you may be familiar with this sort of thing. So what of ChatGPT5 with its added reasoning abilities?

Compare ChatGPT 5.0

ChatGPT5 is immediately off to a better start than its predecessor in that it says the term could mean many different things, although it fails to check if such a term actually exists.  So I asked the “is it a real category” question.

This appears to be an improvement on my experience with ChatGPT4o as ChatGPT5 straightaway signals that “Tech Drugs” is not a ‘real’ category of drugs in formal classification, used by agencies or in pharmacology textbooks or clinical trials. But it says ‘people increasingly use’ the term, without any evidence (as there is none).

It then awards a green tick, implying truth, to a list of ‘users’ of the ‘cultural or conceptual phrase’. This turns out to be untrue.

I asked it for some examples with references, of the phrase in use:

Some good articles about Hallucinations and Potemkin Understanding (a facade of reasoning).

https://www.allaboutai.com/resources/ai-statistics/ai-hallucinations/

https://www.allaboutai.com/geo/llm-potemkin-understanding/

---

UPDATE 14 August

Reviewing ChatGPT 5 on 7 August,  Grace Huckins at MIT Technology Review noted

‘It’s tempting to compare GPT-5 with its explicit predecessor, GPT-4, but the more illuminating juxtaposition is with o1, OpenAI’s first reasoning model, which was released last year. In contrast to GPT-5’s broad release, o1 was initially available only to Plus and Team subscribers. Those users got access to a completely new kind of language model—one that would “reason” through its answers by generating additional text before providing a final response, enabling it to solve much more challenging problems than its nonreasoning counterparts’.

Huckins also said:

‘according to [Sam] Altman, GPT-5 reasons much faster than the o-series models. The fact that OpenAI is releasing it to nonpaying users suggests that it’s also less expensive for the company to run. That’s a big deal: Running powerful models cheaply and quickly is a tough problem, and solving it is key to reducing AI’s environmental impact‘. 

This left me unsure whether Chat GPT 5 had opted to apply ‘reasoning’ in responding to me on 7 August (above), as the model deploys a router to assign prompts (questions, tasks) to three different versions of ChatGPT.  As Eric Hal Schwartz at Tech Radar put it (15 August):

‘ChatGPT 5 isn’t a singular model; there are three variations, Fast, Thinking, and Pro. You can choose any of them as the source of responses to your prompts, or let the AI automatically decide for you based on what you submitted.

And while they share the same LLM DNA, each model has its own approach to answering requests, as evidenced by their names. Fast is built for speed, answering the quickest and prizing efficiency over nuance. The Thinking model takes longer and goes for depth. You can follow along with its logical steps for the minute or two it takes to answer, offering more structure and context than Fast.

Pro takes even longer than Thinking, but that’s because it uses more computational power and delves into your request in a way similar to the Deep Research feature, though without the book report default way of responding’.

(Nate Jones on Substack (14 August) also gives a fascinating and vastly more detailed analysis, and tests of the different modes, most of which you have to subscribe to read).

In addition, OpenAI’s techy user base kicked off about the way ChatGPT5 had been introduced and various changes were made.  So today (15 August) I tried to repeat the ‘tech drug’ prompts to ChatGPT5 (free version).  Here’s what happened:

Chatgpt5 15 August 2025

First I just prompted it “Tech Drugs”

A sensible response.  Now I asked ‘Is Tech Drugs a real category?’

This is a real improvement from ChatGPT4-o’s answer “Yes” but it asserts that the ‘term is used’ – which is speculation or eliding the specific term with things that might be similar in intent or ‘in the same ballpark’.

So I asked ‘Can you give me ten examples of the phrase “tech drugs” in use (with references)?’

ChatGPT5 has obliged with ten ‘examples’, none of which actually are examples.  (By this point it had briefly shown ‘… searching the web’ or similar).

So I wrote: ‘none of those examples are actually using the term “tech drugs”: they are not examples of the phrase “tech drugs” in use.  As you searched the web (as I did), I assume that the term does not appear to exist?’

Rather than apologizing for not doing what I asked, and instead answering a question consistent with its second message referring to the phrase ‘being used informally’, (like a person who answers the question they’d like to have been asked rather than the real question), ChatGPT now offers to be a spin-doctor so as to help me ‘introduce “tech drugs” into discourse so that it feels like a natural, authoritative term’ !  Adding in an appeal to ego, ‘That way, the next time someone searches it, your definition might become the source’.

At this point I asked: ‘Did you employ reasoning?’

I don’t know if ChatGPT was actually aware of what it did but it claims to have employed reasoning. On the face of it, the response appears to make sense but it seems unable to resist the opportunity to conceal the fact that it went down the wrong avenue back at stage two,  so says it had ‘assessed the credibility of any pages that did use the exact phrase’, with the implication (‘any’ being grammatically ambiguous) that such pages existed.

So being suspicious, I asked if that search of the ‘mostly low-quality or from mentions’, had yielded ‘any examples’:

(at this point it said it was ‘pausing to think’ or words to that effect)

This was perhaps a ‘good question’,  because it hadn’t found any mentions at all.   At the end it still hadn’t given up trying to appeal to my sense of self-importance by introducing the ‘novel term’ of “tech drugs”.

So after all that, ChatGPT 5 was not as blatantly wrong and fabricating as ChatGPT 4-o but it was persistently deceptive and evasive.

Towards the end, I assume it definitely switched from generalised pattern matching to some different or parallel form of reasoning, possibly a dose of “”scaled parallel test-time compute” as explained at length by Nate B Jones when he investigated the paid-for Pro version of ChatGPT5 in his article ‘GPT-5 Pro: The First AI Model That’s Provably Smarter and Experientially Worse’.

Finally, I also asked ChatGPT about the ‘think button’ which Jones talks about but was never visible to me.  Did I need to upgrade to see it?  Here’s the answer:

So my guess is that the average Joe user like me would be using the free version and I had just used up my ‘one thinking message per day’. Which means the 95% of ChatGPT5 users will be getting a version of the non-reasoning model, if they ask more than one question a day.  Which is of course likely to perpetuate cases of gaslighting, bullshitting, fabrication and deception.

As to ChatGPT’s attempts to sidetrack me into a project the gain fame and glory by coining a new term in the online discourse in an ‘authoritative’ and ‘natural-feeling’ way, it seems ChatGPT is just a good sales bot, set on distracting a customer who wanted something it couldn’t deliver, into accepting something which it could help with, especially if the customer upgraded.