You hit the nail on the head. They produce output that mimics the appearance of a thoughtful response, but isn’t that at all. LLMs do not actually think and do not have any concept of truth.
This is probably why things like ClickUp naming their AI tool “Brain” annoys me so much. It’s designed partially as a way for organizations to get aggregated access to the major LLMs. So yeah, my former coworkers are getting LLM output from “Clickup Brain.” What a marketing scam.
I’ve been wondering how people’s attitudes toward LLMs would shift if society collectively changed the language we used about them to be more accurate. Maybe there wouldn’t be so many people claiming “AI is great for research” and whatnot. Even then, though, I doubt people would fully get past the human tendency to trust confident-sounding language.




That looks far better than the mainstream AI tools, but I don’t think respecting opt-outs is quite enough. It would be so much better if it were built from solely opt-in training data. As far as I can tell, it’s not attempting to tackle the hallucinations or environmental impact issues. Still, it would be a major change for the better if ChatGPT users switched to something like that.