The chatbot excels at science, beating PhD scholars on a hard science test. But it might ‘hallucinate’ more than its predecessors.

I know people here are very skeptical of AI in general, and there is definitely a lot of hype, but I think the progress in the last decade has been incredible.

Here are some quotes

“In my field of quantum physics, it gives significantly more detailed and coherent responses” than did the company’s last model, GPT-4o, says Mario Krenn, leader of the Artificial Scientist Lab at the Max Planck Institute for the Science of Light in Erlangen, Germany.

Strikingly, o1 has become the first large language model to beat PhD-level scholars on the hardest series of questions — the ‘diamond’ set — in a test called the Graduate-Level Google-Proof Q&A Benchmark (GPQA)1. OpenAI says that its scholars scored just under 70% on GPQA Diamond, and o1 scored 78% overall, with a particularly high score of 93% in physics

OpenAI also tested o1 on a qualifying exam for the International Mathematics Olympiad. Its previous best model, GPT-4o, correctly solved only 13% of the problems, whereas o1 scored 83%.

Kyle Kabasares, a data scientist at the Bay Area Environmental Research Institute in Moffett Field, California, used o1 to replicate some coding from his PhD project that calculated the mass of black holes. “I was just in awe,” he says, noting that it took o1 about an hour to accomplish what took him many months.

Catherine Brownstein, a geneticist at Boston Children’s Hospital in Massachusetts, says the hospital is currently testing several AI systems, including o1-preview, for applications such as connecting the dots between patient characteristics and genes for rare diseases. She says o1 “is more accurate and gives options I didn’t think were possible from a chatbot”.

  • FrogPrincess@lemmy.mlEnglish
    11·
    4 hours ago

    I know people here are very skeptical of AI in general, and…

    People here are gonna have to come up with something to say about AI apart from just screeching “I hate it”

    When man attains the knowledge of this common essence, he uses it as a guide and proceeds to study various concrete things which have not yet been studied, or studied thoroughly, and to discover the particular essence of each; only thus is he able to supplement, enrich and develop his knowledge of their common essence and prevent such knowledge from withering or petrifying. These are the two processes of cognition: one, from the particular to the general, and the other, from the general to the particular. Thus cognition always moves in cycles and (so long as scientific method is strictly adhered to) each cycle advances human knowledge a step higher and so makes it more and more profound. Where our dogmatists err on this question is that, on the one hand, they do not understand that we have to study the particularity of contradiction and know the particular essence of individual things before we can adequately know the universality of contradiction and the common essence of things, and that, on the other hand, they do not understand that after knowing the common essence of things, we must go further and study the concrete things that have not yet been thoroughly studied or have only just emerged. Our dogmatists are lazy-bones. They refuse to undertake any painstaking study of concrete things, they regard general truths as emerging out of the void, they turn them into purely abstract unfathomable formulas, and thereby completely deny and reverse the normal sequence by which man comes to know truth. Nor do they understand the interconnection of the two processes in cognition— from the particular to the general and then from the general to the particular. They understand nothing of the Marxist theory of knowledge.

    • UlyssesT [he/him]@hexbear.netEnglish
      12·
      3 hours ago

      apart from just screeching

      Their emotional screeching.

      Your enlightened totally-non-emotional proselytizing.

      https://futurism.com/openai-employees-say-firms-chief-scientist-has-been-making-strange-spiritual-claims

      You voluntarily came in here, leading with your passive-aggressive “screeching” framing. Don’t whine about being greeted as a clown while you’re riding a Silicon Valley unicycle and juggling cherry picked very smart and very important quotes while internalizing all those tech startup hype pitches and corporate ad copy at the same time. clown

    • EelBolshevikism [none/use name]@hexbear.netEnglish
      11·
      4 hours ago

      we have consistently given other reasons. our criticisms of AI are salient and based both within the fundamental structure of the technology and the ways in which it’s employed.

      edit: the quote you are giving is far more critical of the highly idealistic form of ‘rationality’ present in Silicone Valley than anyone on this site.

      edit 2: you have likely seen a lot of people who have criticized it on the grounds of it’s bad outputs, like with visual arts. While this is often true, it’s a really weak criticism altogether and the far more pressing issues related to ‘AI’ are far more structural and less dependent on the specific capabilities of a given model.