Yeah its really bad. Its incredibly easy to be gaslit using gpts. And they do generate seemingly usable output. And I don’t mean like, using them to analyze data. I mean just some code in an ipynb. And even there its incredibly rotten. Knowing and understanding what you are doing is the entirety of the point, and the whole AI industry misunderstands this. The code, the analyses, even the results, they’re all secondary to understanding why something works the way that it does. Your individual comprehension is the part that matters.
Good times create weak people but we are also experiencing bad times so hopefully more people will become better than the ones who don’t.
May I recommend the book The Fourth Turning? It might give you a glimmer of hope (sprinkled with a dash of salt)
Unscientific bullshit. It’s so dumb that famous Nazi Steve Bannon made a movie about it.
Speaking of a dash of salt on sweet…
“The sweet is not as sweet without the sour”
Umami or whatever. Good stuff.
I’m gonna check it out in a sec because I freaking love to read… like, a lot.
But I read your comment as “The Fourth Turing” and got really excited. I’m all about that. Lol. Litmus them robots. But anyway, I’m all in. I’ll check it out :) thanks for the rec!
In a thread about AI harming the sciences y’all are falling for pseudoscience bs. Proof it’s not the fault of LLMs.
Knowing and understanding what you are doing is the entirety of the point, and the whole AI industry misunderstands this.
The industry as a whole (in this case LLM’s, not AI in general) isn’t misunderstanding anything, their goals are not your goals.
The end users are potentially misunderstanding the goal of the capitalist infrastructure companies and supporting businesses, which is …to turn a profit.
They aren’t interested in making your life easier, giving you accurate information or providing you with a useful tool…unless doing so makes them money.
So it’s really a goal alignment question :
Do you think that your goals/ethics as a professional (or personally) are in line enough with the corporations current method of value extraction to make using their offering a viable option?
And constantly evaluate this decision over time, because it might change.
The code, the analyses, even the results, they’re all secondary to understanding why something works the way that it does. Your individual comprehension is the part that matters.
As a professional in an industry, yes, insofar as that approach fulfills the criteria of success in your industry.
I also agree with this personally.
Contextual Note (in my opinion) :
With some exceptions, people as a whole are also survival driven.
So even with what i said above there aren’t many people who would put “individual comprehension” over survival.
Ultimately i think it comes down to a difference in approach to something like a hierarchy of needs.
People have basic survival and then a bunch of stuff on top.
Corporations generally have survival(profit)…and that’s it.
Obviously I’m speaking in generalities, there are businesses that have stated (and provably followed) goals above and beyond profit but i can’t think of many.
edit: and i can’t think of any involved in LLM shenanigans right now.
I’m at least as deep in the industry as you likely are and I didn’t make comment on what “their goals” are. I made that comment after watching hundreds of thousands of interactions with these systems at a range of skill levels, and am considering it from a development and leadership perspective. And youre right. Their goals aren’t my goals. But they’ve been very clear with me about what their goals are. You could simplify them down to “turn a profit” but I think that’s a misunderstanding, or that it hides deeper goals.
I don’t think profit is the goal, at least, I don’t think it’s the goal yet, or maybe at all. I think that the goal is the collapse of the human skill set which has resulted in a wide, large, and deep pool of talented capable people. The goal isn’t to be profitable per se, but the monopolize people’s ability to work independently of their products.
For the past 15 years my job has not only been to develop ml technologies, but also to and mentor people into becoming programmers focused almost exclusively in ml. And within that, I work pretty domain specific. I work in the “for good” side and always have (except that stint in the US military when I was 17) and if you look through my posts you’re gonna find some of the more public stuff. I work with a lot of satellite and environmental data. I work in academia, public sector and also private sector, just depending on the contract. It’s mostly environmental mapping applications (water, fire and forests) globally.
What I’m seeing as an effect of these tools is an erosion of critical thinking and problem solving skills at a fundamental level. Jr scientists and developers just aren’t learning the basics anymore and with that, their ability to detect bullshit or dogshit, or cat shit wrapped in dogs it is utterly diminished. And so a lot of dog shit ends up in the final product. Bit along with that, it’s easy to let the hallucinating machine convince you it knows more than it does. It’s practically nothing but dark patterns to do exactly that. It uses couched cautionary language to create the appearance of considering a problem from multiple sides. It asks clarifying questions, which both draws a used in but also creates the illusion of understanding.
And it sounds good. But the problem (and this is my editorial opinion as some one who was in the beta for these products) truly is that the underlying models haven’t actually improved in their core capabilities in 4 years. The glitz and the glam and the tooling and the redundancy, their ability to use tools, be integrated into other things… sure. All that has improved. But they’ve never been able to overcome some core issues which imo are fundamental to the architecture and can’t be overcome in this current framework.
So to me the problem is two fold. First, they’re putting all current developers in a acid bath which is eroding their ability to solve problems independently, or preventing them from developing those skills in the first place. And there is no way to do that learning without just putting in the time. Second, they aren’t what they say they are. These tools, if you need some basic code, are phenomenal for just something small. But you need to maintain the idea or conception of what you are doing. They have no practical ability to architecture real solutions or any kind of deep critical thinking. But they’ll do their best to convince you otherwise.
And finally, into the profit motive. I think we should take Musks purchase of twitter as a cautionary tale. The internet regaled in how stupid Musk was for doing so. How he was overpaying for something that wasn’t profitable. And on and on. I’m sure you remember. But that unprofitable decision allowed them to steal an election. Even years later it’s probably still questionable if that decision was ever directly profitable for Musk, bit that was never the point for them. Theirs was a calculus of power, and they did get their considerations right in this regard. Likewise, I think if you narrowly focus on profits for these companies, you’ll miss the forest for the trees. Profit is a pathway to power, but it’s now power itself, and money asymptotes with its ability to exercise power. Power is ultimately power, and I believe that is the game these companies, acting in coordination, are pursuing.
I’m at least as deep in the industry as you likely are and I didn’t make comment on what “their goals” are. I made that comment after watching hundreds of thousands of interactions with these systems at a range of skill levels, and am considering it from a development and leadership perspective. And youre right. Their goals aren’t my goals. But they’ve been very clear with me about what their goals are. You could simplify them down to “turn a profit” but I think that’s a misunderstanding, or that it hides deeper goals.
I don’t think profit is the goal, at least, I don’t think it’s the goal yet, or maybe at all. I think that the goal is the collapse of the human skill set which has resulted in a wide, large, and deep pool of talented capable people. The goal isn’t to be profitable per se, but the monopolize people’s ability to work independently of their products.
I perhaps didn’t go into enough detail about what i think their motivations are, when i say profit i should have said profit/power/control, they aren’t all the same but they could be considered different faces of the same rough concept.
i did imply that they were only in it for the money and that i think is where a miscommunication is coming from.
For the past 15 years my job has not only been to develop ml technologies, but also to and mentor people into becoming programmers focused almost exclusively in ml. And within that, I work pretty domain specific. I work in the “for good” side and always have (except that stint in the US military when I was 17) and if you look through my posts you’re gonna find some of the more public stuff. I work with a lot of satellite and environmental data. I work in academia, public sector and also private sector, just depending on the contract. It’s mostly environmental mapping applications (water, fire and forests) globally.
What I’m seeing as an effect of these tools is an erosion of critical thinking and problem solving skills at a fundamental level. Jr scientists and developers just aren’t learning the basics anymore and with that, their ability to detect bullshit or dogshit, or cat shit wrapped in dogs it is utterly diminished. And so a lot of dog shit ends up in the final product. Bit along with that, it’s easy to let the hallucinating machine convince you it knows more than it does. It’s practically nothing but dark patterns to do exactly that. It uses couched cautionary language to create the appearance of considering a problem from multiple sides. It asks clarifying questions, which both draws a used in but also creates the illusion of understanding.
And it sounds good. But the problem (and this is my editorial opinion as some one who was in the beta for these products) truly is that the underlying models haven’t actually improved in their core capabilities in 4 years. The glitz and the glam and the tooling and the redundancy, their ability to use tools, be integrated into other things… sure. All that has improved. But they’ve never been able to overcome some core issues which imo are fundamental to the architecture and can’t be overcome in this current framework.
I fully agree with all of that and i wasn’t arguing against it.
I was simply arguing that countering all the problems you mentioned isn’t the goal of the companies/corporations selling the pickaxe infrastructure.
ML (and AI) in general hasn’t been the kind of problem we are seeing now until the introduction of the current incarnation of LLM’s, it’s why i specifically mentioned LLM’s as the target of my thoughts.
There have been AI winters and bubbles before but the scale and cultural penetration of this wave seems different (though i suppose we’ll see over time if i’m right about that)
It’s not only a difference in scale/scope but it seems to be one of those problems that’s cumulative and feeds off of itself once it’s hits a critical threshold of adoption/usage.
It happens to be wildly profitable(fiscally and in terms of power/control) for the companies involved, which doesn’t help.
By profitable i don’t mean that the companies are making money on the P&L sheets, i mean the few individuals who are accumulating power/wealth/control from the shenanigans that are ongoing.
So to me the problem is two fold. First, they’re putting all current developers in a acid bath which is eroding their ability to solve problems independently, or preventing them from developing those skills in the first place. And there is no way to do that learning without just putting in the time. Second, they aren’t what they say they are. These tools, if you need some basic code, are phenomenal for just something small. But you need to maintain the idea or conception of what you are doing. They have no practical ability to architecture real solutions or any kind of deep critical thinking. But they’ll do their best to convince you otherwise.
Also agreed.
I would further argue that the possibility of the erosion of skills being something of an active goal is non-zero. \ mid to long term power/control/profit can only be helped by fostering a dependence on throwing more and more tokens at a problem because the developer no longer has the ability to solve it themselves.
Though that’s some full tinfoil hat speculation there on my part.
And finally, into the profit motive. I think we should take Musks purchase of twitter as a cautionary tale. The internet regaled in how stupid Musk was for doing so. How he was overpaying for something that wasn’t profitable. And on and on. I’m sure you remember. But that unprofitable decision allowed them to steal an election. Even years later it’s probably still questionable if that decision was ever directly profitable for Musk, bit that was never the point for them. Theirs was a calculus of power, and they did get their considerations right in this regard. Likewise, I think if you narrowly focus on profits for these companies, you’ll miss the forest for the trees. Profit is a pathway to power, but it’s now power itself, and money asymptotes with its ability to exercise power. Power is ultimately power, and I believe that is the game these companies, acting in coordination, are pursuing.
Also agreed, although profit has always been a gateway to power , the exchange rate is at an all time high.
As i said before, i was intentionally being simplistic when i said profit, i just didn’t necessarily want to go full capitalist oligarchs/ruling class/erosion of the current version of society.
To the twitter purchase, it doesn’t have to be a line item on a sheet for it to be useful investment, he was able to leverage that inflated valuation to “borrow” money against it that he didn’t need to pay the prerequisite amount of tax against , which is a common tactic for people already rich AF.
It’s a similar thing to what’s happening with the LLM investment circle/bubble that will fully fuck over the american stock market (and tangentially everything else to some degree).
Nvidia, openAI, anthropic, microsoft etc , all trading imaginary purchases on speculative future resources to inflate the valuation of each other in a big circle is similar in concept, if more extravagant in execution.
I would further argue that the possibility of the erosion of skills being something of an active goal is non-zero. \ mid to long term power/control/profit can only be helped by fostering a dependence on throwing more and more tokens at a problem because the developer no longer has the ability to solve it themselves.
Though that’s some full tinfoil hat speculation there on my part.
I think it’s the conversation we should be having. I also can’t say for certain that this is their case, but a certain company which rhymes with “boogle” is putting a huge amount of pressure on the industry to get people to use their alpha earth embeddings to create derivative ml products. They put on a wowfest at ForesttSAT a few weeks back, but it’s clear that either a) the earth engine project is deeply unprofitable and is going to get axed, or b) they think they can run the same strat for satellite remote sensing they’re doing with llms in programing.
Unfortunately I couldn’t attend so it was just a coauthor who attended who reported back, but then low and behold, all kinds of non profits and “for good” organizations are finding funding (since, you know, Musk collapsed the state funded research apparatus via DOGE… coincidence?) to build solutions off of these embeddings. It’s a solution looking for a problem but it basically means all solutions using it will be built on that company’s architecture and hardware.
And like… I don’t have a problem with the idea of an embedding layer. And earth engine absolutely has been a major force for good…
But there is clearly an effort to center corporations as owning information we all collectively paid for with our taxes, which is what those embeddings represent.
Its incredibly easy to be gaslit using gpts
no scientist is going to be gaslit by ChatGPT
I just spent two weeks undoing something a scientist did while being gaslit by a a gpt. I promise they do.
They are human beings too, and many don’t understand this technology better than the public. It’s easy to fall for the marketing and listen to these chat bots uncritically if you don’t know better
Good guard rails are books and studies published before generative science became a thing.
Heck, even a 15 year old hard drive that sat on a shelf for 15 years with a few gigs of data and never got connected is already worth a ton.
It’s like the plague or the dark ages, but electronic. And since so many people rely on this technology, it will be devastating in many ways after the inevitable implosion.
There are guardrails, at all journals. And we now need to upload raw data. People have been fucking around in science publications since Photoshop was invented.
guardrails
guard towers
If all of these scientists are as stupid as a bunch of school children using AI, why should we have ever trusted them in the first place?
If all of these allegedly brilliant scientists are so absurdly, dumb and untrustworthy, if they are so foolish as to be fooled by AI, then why did we ever trust them in the first place?
How stupid were the scientists? How stupid were we to think that they had any answers?
Perhaps we need better scientists that know better than to ever even ask AI anything
Monitor their Internet usage. If they ever use AI, just fire them. It doesn’t seem like a difficult problem to solve.
Most people use LLMs poorly and irresponsible. Most problems they are used for, are a poor choice as well. That doesn’t mean they cannot be useful.
Oh, I forgot that the entirety of science didn’t exist before AI, and can’t exist without AI. Since science is entirely dependent on using AI exclusively for 100% of all thought, we’re obviously all fucked.
¯\_(ツ)_/¯
You’re making the argument that the deadly deadly poison is only deadly if you drink too much of it, or drink it incorrectly. I elect not to drink the deadly, deadly poison — any of it at all.
Not really what I’m saying. More along the lines of: poison is bad to drink, but great for killing weed.
As for LLMs and sciences (and most fields): “drinking poison” -> “treating LLM output as factual”. “killing weed” -> “some otherwise mundane language transformation that you verify the correctness of”.
poison is bad to drink, but great for killing weed.
But that’s not what you said, nor is it the argument you’re making
The pursuit of science by humanity has made it this far for thousands of years. And has done so quite successfully without AI poisoning that pursuit. You have yet to make the argument that the poison of AI is in any way helpful.
Because you can’t. Because it isn’t.
Ignore the evidence at your own peril. I will not.
It kinda seems to me you’re deciding both sides of the argument. It’s like a straw man monologue show.
Arriving at a conclusion without evidence… Exactly the danger that AI presents.
You OK?
“The poison is in the dose”.
Also, what an astonishingly uncritical perspective. I think there are plenty of legitimate concerns to raise about LLMs in science, but there’s a reason that researchers are adopting these tools. I suspect it’s because there’s a ton of rote work in the field (literature review, analysis, drafting a document…) and they’re under great pressure to publish on an accelerating cadence.
but there’s a reason that researchers are adopting these tools.
Yeah, it’s how you get grant money.
Oh really, and tell us…how much of that grant money goes to the actual scientist who wrote the grant?
None? But that grant money is the only way they can actually do anything.
Sprinkling AI buzzwords into grant proposals is how the sausage is made.
“I had to drink the poison because I was under pressure”
Somehow, this doesn’t ring true. Are you defending scientific discovery, or are you defending corporate pressure? Or are you just defending poison? either way, this sounds like bullshit.
it’s because there’s a ton of rote work in the field
So now it’s OK just because scientists are suddenly lazy? You gotta be kidding.
“We should stop using chainsaws and cut down these trees with our teeth”
Yet, for thousands of years, cutting down these trees with our teeth worked surprisingly well.…
Proven by that tiny little computer in your hand that you’re typing your response upon
That’s a lot of ad hominem for someone so vocally against it.
If your entire argument is “look at this mistake that was made, why did we trust scientists in the first place?” (with some personal attacks thrown in) you’re going to struggle to find a genuine conversation.
Scientists are people too, they aren’t a homogeneous pool of unassailable morals/ethics and correctness.
People fuck up, constantly, poor decisions and mistakes abound.
The whole idea of the scientific method is iteration towards success, if everyone always made the correct decisions there would be no need for iteration.
It’s not an excuse for shitty work, science as a whole has a bunch of problems that urgently need addressing, but it is an explanation that allows for more nuance than “scientists stupid, hur hur”.
Given your other answers so far I’m not expecting you to actually respond to this in good faith , i’m putting it out there mainly for me.
I can save you some time and say that if this is the calibre of response you normally provide, you should probably just block me, you are almost certainly not going to like interacting with me (or reading anything i write).
That’s a lot of words for a hypocritical “I know you are, but what am I?” Containing its own personal attacks. And coming back at me using an alternate account?
Pathetic. Blocked.
You won’t see this because i’m blocked but it went about as i expected.
- No actual refutations or rebuttals
- Claiming personal attacks with no references (because there aren’t any to reference, i assume)
- Magical thinking that two people disagreeing with you must mean it’s the same person in disguise…for some reason
- Blocking.
Predictably boring, nothing of value was lost.
Sorry about your grade 10 chemistry mark.
Also, check out the 2024 Nobel Prize for Chemistry. Warning: some big words.
It’s surprisingly easy to make a cogent argument against AI (or any argument at all) without insulting others.
Try it sometime
