https://futurism.com/the-byte/government-ai-worse-summarizing
The upshot: these AI summaries were so bad that the assessors agreed that using them could require more work down the line, because of the amount of fact-checking they require. If that’s the case, then the purported upsides of using the technology — cost-cutting and time-saving — are seriously called into question.
You must log in or register to comment.
Sure, but it’s cheaper, and so if we fire all of our employees and replace them with AI, for this next quarter our profits will go WAY up, and then I can get my bonus and retire. So it’s totally fine!
There’s a certain level of risk aversion with these decisions though. One of the justification of salaries for managers who generally don’t do shit is they take “responsibility”. Honestly even if AI was performing at or above human level, a lot of briefs would have to be done by someone you could fire anyway.
And as much as next quarter performance is all they care about, there are still some survival instincts left. My last company put a ban on using genAI for all client facing activities because a sales guy almost presented a deck with client is going to instantly walk out levels of wrong information in it.
Yeah, that’s something I was thinking about. With human employees, you can always blame workers when anything goes wrong, fire some people and call it a day. AI can’t take responsibility the same way.
They’ll fire everyone and love the short term profit boost but within a year realize it’s fucking up their production processes. But they’ll be so hooked on all that money saving that they’ll pull some sneaky ways of rehiring everyone buy for less money and benefits.
Any time a client mentions “I asked ChatGPT” or any of the other hopped-up chatbots, what follows is always, without fail, completely ass-backwards and wrong as hell. We literally note in client files the ones who keep asking some shitty chatbot instead of us because they’re frequent fuckups and knowing that they’re a chatbot pervert helps us narrow down what stupid shit they’ve done again.
Yeah I’ve purged “AI” from my vocabulary, at least for now.
These are chatbots. That’s it. “AI” is a marketing term.
I say “LLM” or “treat printer” because fuck the marketing word and fuck the bazinga cultists that keep expecting a fully sapient but also unconditionally adoring mommy bangmaid just like in the cyberpunkerino treats any day now.
I recall my AI class discussed a bunch of different things that people call AI that don’t come anywhere near “replacement human”. For instance, the AI in red alert 2 has some basic rules about buildings and gathering a certain number of units and send them the players way.
Obviously, RA2s “AI” isn’t being used for labour discipline and llms are massively overhyped but I think getting hung up on the word is… idk, kinda a waste of time (as I feel like a lot of this thread is)
People are right to be annoyed about “zero emissions” vehicles that hide their carbon waste requirements away from tailpipes, and for “smart” bombs that aren’t particularly smart when they murder even more (allegedly) unintended targets, so I think it’s fair for people to be annoyed that “AI” isn’t really that and that any generalized artificial intelligence developed in the future will have some difficulty being distinguished by the public from the marketing bullshit they’ve already taken in for prior products.
I think people are allowed to be annoyed, but if thats all you want to talk about i think its a waste of energy? It’s just language, we can call it flubbon if you like and move the conversation along.
Unless we want to get bogged down talking about whether band aids “medical adhesive strips”, which is a perfectly fine conversation to have if that’s what both participants want to talk about.
It’s not just language when that marketing bullshit moves venture capital around and that consequent accumulation of power makes specific sectors of the ruling class that much more influential over the rest, to all of our detriment.
NVIDIA currently dominates the stock market because of related marketing hype that started during the cryptocurrency hype wave.
Because people call it an AI instead of a bunch of related trained predictive algorithms? If the other things were happening (labour discipline, art theft, using a gallon of water to run a bad google search) but people were using whatever term you wanted, what would actually change?
Like, I’m not saying it’s wrong to be annoyed by these companies ad copy, and there’s absolutely people out there who think “AI” is more human than their employees, it’s just a huge amount of time and energy wasted over a relatively minor part of the whole relationship. Even this 3 reply exchange here is probably too much.
I said it before and I’ll say it one more time: YES, it does matter in this case, because the “AI” label and the consequent bullshit artistry tied to it grants the tech companies involved in it more venture capital from credulous investors persuaded by the bullshit labeling to put more money into it, which means the bad stuff you brought up happens even more as a consequence.
Like, I’m not saying it’s wrong to be annoyed by these companies ad copy
I’m not saying you can’t be annoyed at my being annoyed, but even here on this leftist shitposting site some people buy into the “AI” label meaning a lot more than it actually does and that has a ripple effect of consequences even here as well.
AI is a fine term because it’s artificial. It’s a facsimile. If they were serious it would just be I
It’s still a marketing term. AI makes people think of SciFi and robot mommies.
Exactly, there is no “real AI”. It’s an oxymoron.
AI apocalypse by super intelligence 
AI apocalypse by super incompetence 
They bury the lede in the article thought. They used llamma 2 - 70B which is not a great model
Continuing to look under LLM rocks of varying size and shininess in search of the solve-every-problem robot god of the future
Observing that newer models perform better than older models on a variety of benchmarks means you want to have an intimate relationship with your computer.
Stating that newer models that perform better than old models somehow implies that the newer models are completely living up to marketing hype, up to and including calling it “artificial intelligence” to begin with.
And yes, it’s a known and established issue where some people that stan for these treat printers do see them as replacements for people, not tools. There’s already an entire startup industry of “AI companions” selling that belief, so what I said isn’t as absurd as you claim it is. Besides, I said “robot god of the future” there, not “AI” waifus, but there’s certainly a connection that some true believers make between the two concepts.
I didn’t mention marketing, I’m talking about benchmarks. Benchmarks designed to test the machine’s abilities to perform reasoning like humans. And they’re being improved on constantly. Sorry if that rubs ya the wrong way.
I didn’t mention marketing
That’s too bad, because “AI” as it stands, and what is branded as “AI,” is not what it claims to be on the label. There are certainly scientific efforts underway to make rudimentary versions of that, but large language models and related technology simply isn’t it, and to believe otherwise is marketing, whether you accept it or not.
Benchmarks designed to test the machine’s abilities to perform reasoning like humans. And they’re being improved on constantly
Again, you’re believing in the marketing.
https://bigthink.com/the-future/artificial-general-intelligence-true-ai/
https://time.com/collection/time100-voices/6980134/ai-llm-not-sentient/
Sorry if that rubs ya the wrong way.
You’re not sorry, this isn’t /r/Futurology or /r/Singularity, and the
closer to your post only makes it worse.You seem to have a kind of “head in the sand” approach to this (I get it, we have to protect our egos). Maybe educate yourself on what some of the research in this field looks like.
Here’s a list of a lot of the common benchmarks that are used by researchers all over the world, and have nothing to do with Sam Altman trying to hype OpenAI’s stock price or whatever the latest late stage capitalist shenanigans are in the business world.
- MMLU (Massive Multitask Language Understanding)
- TruthfulQA
- HellaSwag
- ARC (AI2 Reasoning Challenge)
- Winogrande
- BIG-Bench Hard
- GSM8K (Grade School Math 8K)
- HumanEval
- MBPP (Mostly Basic Programming Problems)
- CodeXGLUE
- Chatbot Arena
- MT-Bench
I know some people are, but I’m not saying these things are sentient (nice Time link tho lmao). This is a massive leap in logic that you are making. I’m saying, these models are way better at taking standardized tests and shit than they were even months ago and that has implications for labor.
Honestly you sound scared about this stuff.
You seem to have a kind of “head in the sand” approach to this
Even more
and there’s so much more text to read. Here we go.(I get it, we have to protect our egos)
Maybe educate yourself on what some of the research in this field looks like.
Maybe stop ignoring entire fields of research that, to this date, are still figuring out what biological brains are doing and how they are doing them instead of just nodding along to what you already want to believe from people that have blinders for anything outside of their field (computers, in this case). It’s a case of someone with a hammer seeing everything as a nail, and you buying into that.
Honestly you sound scared about this stuff.
More like tired. If you weren’t so religiously defensive about the apparent advent of whatever you’re hoping for, you’d know that I have on many occasions stated that artificial intelligence is possible and may even be achieved within current lifetimes, but reiterating and refining the currently hyped “AI” product simply isn’t it.
It’s like if people were trying to develop rocketry to achieve space travel, but you and yours were smugly stating that this particularly sharp knife will cut the heavens open, just you wait.
What’s the model that does work with this use case?
(I don’t think there is one)
“Pfft! That only matters if you care about factual accuracy. So let me make it real simple: Facts don’t care about your feelings, and my financesthe future doesn’t care about your facts!”No AI apocalypse yet
The upshot: these AI summaries were so bad that the assessors agreed that using them could require more work down the line
Oh man, this’d be really bad if we structured our society in such a way that instead of taking a holistic approach of looking at things it was all random KPIs in an excel file that measure one very narrow field of view of things like how fast I am at my job
Pretty sure most people who’ve used Ai in their work know the results kinda sucks, and only use it because writing a prompt for an LLM is way faster than writing anything yourself.
I sometimes use it to bypass corporate copyright on industrial standards. Kinda eh about it and I have to double check everything. What a world we’ve built >.>
Why did you have to attack me like that?
Maybe because it’s not genuine AI
I love how all the corporate bootlickers for over three years now have just assumed some real breakthrough in emergent general intelligence took place and now humanity can build rudimentary consciousness
What world are these dipshits living in, it’s just marketing for data aggregators not a replacement flesh and blood humans
I love how all the corporate bootlickers for over three years now have just assumed some real breakthrough in emergent general intelligence took place and now humanity can build rudimentary consciousness
“AI” as it exists right now is a triumph… in marketing.
The primary driver of techbro profitability is hopes and dreams. They want a holo waifu to be their mommy bangmaid and to have godlike powers but also unconditionally love and serve them, and they want to manifest that by bullshitting about it on the internet and selling the lie.
Who would’ve guessed that inflated predictive algorithms can’t perfrom well because they’re just unable to understand anything
But if enough rain forest is burned and enough waste carbon is dumped into the air, those predictive algorithms are that much closer to understanding everything!
I still think in development environments, limited LLM systems can be used in tandem with other systems like linters and OG snippets to help maintain style and simplify boilerplate.
I use Co-Pilot at work because I do development on the side and need something to help me bash out simple scripts really fast that use our apis. The codebase we have is big enough now (50,000 ish lines and hundreds of files) so it tends to pick up primarily on the context of the codebase. It does still fallback to the general context pretty often though and that’s a fucking pain.
Having the benefits of an LLM trained on your own code and examples without the drawbacks of it occasionally just injecting random bullshit from its training data would be great.
I’ve kinda seen this in manufacturing for the last few years. Not explicitly “AI” but newer equipment designed around being smarter and not requiring skilled operators. Think like WordPress but for industrial machines; it might do basic stuff pretty well but fails at complex operations, and it’s an atrocity if you ever look behind the scenes to do some troubleshooting.
Hell yeah, smart machine? That’s gonna cost a premium. Oh, and because these machines are so sophisticated, you’ll need a higher tier support contract, that’s another premium… I mean it’s not like you have skilled technicians on staff anymore, they all retired and all your new guys just know how to press “play,” since we made the machines so easy to use… you’re not fixing anything yourself anymore.
Back to your support contract, now we have the Bronze tier which gets you one of our field techs out there within 48 hours, but if your business can’t handle that kind of downtime we could upgrade you to Silver or Gold…
I knew this was going to happen because my mom always told me I was a very smart lad. No, I’m not nervous about this at all, why?
Also, this study inexplicably used llama-2 ?? which does indeed suck and is nowhere near state of the art. Look at this scorecard from a couple months ago: https://www.trustbit.tech/en/llm-leaderboard-juli-2024
Note the massive jump in quality for open source models. We went from around ~50% for Llama 2 to now +80% for Llama 3 on a lot of benchmarks. Llama 2 was released in July 2023, and Llama 3.1 just came out on July 2024.
You don’t have to be a redditor, bazinga brain, treat enjoyer, etc. to realize these silicon valley freaks are onto something with this technology and the field is evolving quickly.
To expand on that for people who think it’s all just smoke and mirrors. I think, just like the assembly line, work places will be reorganized to facilitate the usefulness/capabilities of LLM’s and, perhaps more importantly, designed to obviate their weaknesses.
It’s just that people are still figuring out what that new organization will look like. There hasn’t been a Henry Ford type for LLM’s yet (and hopefully won’t be a Nazi this time). Obviously there’s no guarantee there will be such a person/organization but I don’t think it super unlikely either.
Well said. This is all so new, we’re still figuring out the implications of how to grapple with it.
I do think people here have a tendency to just hate all of it out of hand, which I get to some extent. The last thing we want is Elon to have terminators or something, haha.
We went from “it can’t even draw hands!!!” last year to “they’ll just use it for porn!!!” now, ignoring the fact that it can render pretty amazing looking videos in such a short time span.
I do think people here have a tendency to just hate all of it out of hand, which I get to some extent.
Yeah the hype cycle is certainly annoying. As is the accompanying fire/re-hire at lower pay cycle that follows any automation.
ignoring the fact that it can render pretty amazing looking videos in such a short time span.
I actually think the generative aspect of neural networks is the least interesting/useful/innovative/etc. Though it will admittedly be more interesting when an LLM can say, use blender to make a video rather than just wholesale generating it. Or at least generate the files/3d models necessary to have it be edited by a person just like they would anything else. I suspect there will have to be a pretty significant architecture change for them to be able to make convincing/coherent movie-length videos.
Chaotic system control, like they’re doing with nuclear fusion plasma is the most interesting, to me anyway.
Chaotic system control
Sounds like a fun rabbit hole…
I still motion for a Butlerian Jihad
Good thing they destroyed the working class for a fucking grift though.
Maybe employers will start hiring again and paying living wages…
