No they didn’t they recorded 268 responses from a statistical text generator
And they didn’t synthesize any of them to verify the algorithm works
Right? 233 of them don’t actually exist because the AI made them up, 27 of them are already something else the AI made up a name for, and 8 of them might be a thing except it’s cost prohibitive and no one’s ever attempted it before.
WINNING
I wish authors behind pieces like this would stop confounding AI and machine learning. People read stuff like this and end up believing that a language model managed to discover new metal alloys. The use of generative AI for an illustration certainly does not help.
That illustration is, however, not the fault of the journalist - it’s credited a prompt by Skoltech PR. So the PR department of the research institute itself thought some generative AI image was the best possible way to illustrate this. Which makes me a bit sceptical.
The cited lead author currently has one citation in Google Scholar, which seems to be the product of some workshop on LLMs. Another publication is on financial time series (famously not a field deprived of complexity), and now the author is suddenly an expert of metal alloys. I’m all for supporting young researchers, but I’m also a fan of trying to understand one field before jumping to the next.
The Skolkovo Institute of Science and Technology, a private institute in Moscow, started out in 2011 with scientific help from MIT and financial backing from the Russian state. In 2022 MIT cut their partnership for obvious reasons. I’m not sure this helps to inspire confidence, but then again I’m sure there are qualified people working there.
Then we have the concerns voiced already in this thread - can we even count on any of these findings to hold at all, if they didn’t bother actually putting anything to the test beyond their possibly faulty model.
Of course, this new paper is published in Nature, which is pretty renowned and has only occasionally fucked up by publishing grifters. And I don’t know the first thing about material science to assess the value of this research. Would love to hear if anyone closer to the field could give their take on it.
Is discover the right word to use there?
“AI said it discovered 268 new high-performance metal alloys. Idiots believed it without validation. We reported it as fact without validation.”
@slazer2au @pranjalmalewar Is Researchers the right word to use there?
They narrowed down the possibilities to ones that are more likely to work than random, which is a great use of AI. Lots of testing is still needed.
They have also done this for years with algorithms. AI models are a step up though, like a chain saw compared to a hand saw for cutting down large numbers of trees.
But this is also being promoted as successful when in reality all 268 alloys might fail testing or have other properties that the model didn’t include. It is progress in the process of finding new alloys, but the results still need to be verified.
Yes, but how many fingers did they have?
