Generative AI is just tech bro rebranding of statistics. It’s basically just actuarial tables rebranded and applied to Moby Dick and Wikipedia.
You remember linear (or polynomial) regressionse from high school where you had to find a function that closely mapped the data provided? That’s “training.” Now that you have a trained model look at the location of your input data, choose a spot to the “right” on your regression and pick a random word that’s close to the regression.
Good luck getting any “intelligence,” let alone a “general” one out of that.
This one is such a big “open” secret in tech. There is no intelligence; it’s all artificial. It really is just a big text prediction algorithm.
To elaborate on your thing, the current models do a little bit better. They are able to algorithmically look a few words ahead and a few words behind to glean some context but that’s it. So if you misspell a ward or create a word like “Sally shkloinged Tom in the head with a frying pan,” it can programmatically deduce what ‘shkloinged’ meant in regards to creating coherent context around it.
People confuse the fact that it can do language under the confines of context given to it as “intelligence” but it really has none. We are years away from internal tool calls that can actually work at all, or even some of the time. AI can’t even fucking do something like keeping track of time passed. Ask Chat GPT to keep track of the time while you run and then immediately say you finished. It’ll just make up a number because algorithmically it goes through the sentence and can programmatically deduce a number should be there in the time format so it inserts one.
Yes, that’s what “neural networks” in computing are, a method of systematically making a machine that does novel pattern matching tricks without needing to understand how to make that pattern matching machine yourself. Which is cool, and has useful applications, and it starts to do some scary things when scaled up, but it’s hitting a clear limit of what just making its numbers bigger will do and all the grifters keep doubling down on just throwing more and more and more processing power at this dead end instead of admitting they were wrong and going back to rethink their fundamental approach (because the grifters don’t care and don’t know shit, and they’re the ones controlling everything).
Some good use cases of ML are crowdsourced computing projects like protein folding (folding@home) and exoplanet hunting (TESS) by manually reviewing starlight spectral plots. Pure pattern recognition, no genuine intelligence there.
Also OCR, speech-to-text transcription, and generally any sort of “find and point out/parse patterns in this huge amount of noisy data” task. It’s never foolproof, but it has massively improved all of those. The problem is just the way LLMs are being misused to try to make a worse search engine and then pretend that this shitty, extremely unreliable chatbot can replace workers. Also the insane costs of trying to roll out the infrastructure for them to just continue being awful on ever greater scales and the knock-on effects that’s having on everything even remotely related.
Back in college I took a couple machine learning classes. After the second I understood where the market would eventually end up. It’s a pattern matching machine, if you were to provide infinite data and infinite compute, you could have the machine do enough regression to match the presented surface of whatever the data represented.
I sat back and was like “oh this is sorta cool but sorta dumb”. You can’t create a novel thought process from this, you are limited by what data you can collect and label, and labeling data is an extremely time consuming process because it relies on humans. But also, you don’t really need labeled data if you don’t care about correctness. You can get away with feeding the regression machine a load of data and label more generally based on how close certain points are in a vector space. It’s how “sentiment analysis” works. You can take IMDB’s database of reviews that each have some words and a star rating, and use the star rating to categorize in “good” and “bad”, then average out the distance between certain words and the frequency they appear within the “good”, “mid”, and “bad” spectrum.
Suddenly, relationships show up, “lawyer” is close to “criminal” in the 3d space.
What modern LLMs do, is just layer this same system with a few short-circuits called context windows. It basically maintains a space of relevance within the broader context of what the model was trained in. For the IMDB example, lets say you’re asking a machine about action movies with x, y, z characteristics, the context maintains those to short-circuit the larger model to retain ‘focus’ and give you near-by relationships.
With enough data you can recreate language based on distance markers and frequency. But back to my original point, it’s the surface level of what it was trained on, a plaster mask. The mask doesn’t have the complexity of the muscles and skin it was formed on, it’s shallow.
That all being said, the ability to make a shallow mask is useful for cross-referencing large amounts of data. The disaster strikes when it’s treated as an all knowing god and used to do military strikes.
The disaster strikes when it’s treated as an all knowing god and used to do military strikes.
Or fire people.
We gave computers a stack of transparencies and told it to pick a couple at random to make something new and we pretended it was smart. And now people really be thinking humans aren’t needed for production anymore.
Generative AI is just tech bro rebranding of statistics. It’s basically just actuarial tables rebranded and applied to Moby Dick and Wikipedia.
You remember linear (or polynomial) regressionse from high school where you had to find a function that closely mapped the data provided? That’s “training.” Now that you have a trained model look at the location of your input data, choose a spot to the “right” on your regression and pick a random word that’s close to the regression.
Good luck getting any “intelligence,” let alone a “general” one out of that.
This one is such a big “open” secret in tech. There is no intelligence; it’s all artificial. It really is just a big text prediction algorithm.
To elaborate on your thing, the current models do a little bit better. They are able to algorithmically look a few words ahead and a few words behind to glean some context but that’s it. So if you misspell a ward or create a word like “Sally shkloinged Tom in the head with a frying pan,” it can programmatically deduce what ‘shkloinged’ meant in regards to creating coherent context around it.
People confuse the fact that it can do language under the confines of context given to it as “intelligence” but it really has none. We are years away from internal tool calls that can actually work at all, or even some of the time. AI can’t even fucking do something like keeping track of time passed. Ask Chat GPT to keep track of the time while you run and then immediately say you finished. It’ll just make up a number because algorithmically it goes through the sentence and can programmatically deduce a number should be there in the time format so it inserts one.
There are those that understand it’s a prediction algorithm, those who know what that means, those who use it, and those who think it’s always right.
They all work on the same team and the last guy is the manager.
Yes, that’s what “neural networks” in computing are, a method of systematically making a machine that does novel pattern matching tricks without needing to understand how to make that pattern matching machine yourself. Which is cool, and has useful applications, and it starts to do some scary things when scaled up, but it’s hitting a clear limit of what just making its numbers bigger will do and all the grifters keep doubling down on just throwing more and more and more processing power at this dead end instead of admitting they were wrong and going back to rethink their fundamental approach (because the grifters don’t care and don’t know shit, and they’re the ones controlling everything).
Some good use cases of ML are crowdsourced computing projects like protein folding (folding@home) and exoplanet hunting (TESS) by manually reviewing starlight spectral plots. Pure pattern recognition, no genuine intelligence there.
Also OCR, speech-to-text transcription, and generally any sort of “find and point out/parse patterns in this huge amount of noisy data” task. It’s never foolproof, but it has massively improved all of those. The problem is just the way LLMs are being misused to try to make a worse search engine and then pretend that this shitty, extremely unreliable chatbot can replace workers. Also the insane costs of trying to roll out the infrastructure for them to just continue being awful on ever greater scales and the knock-on effects that’s having on everything even remotely related.
Back in college I took a couple machine learning classes. After the second I understood where the market would eventually end up. It’s a pattern matching machine, if you were to provide infinite data and infinite compute, you could have the machine do enough regression to match the presented surface of whatever the data represented.
I sat back and was like “oh this is sorta cool but sorta dumb”. You can’t create a novel thought process from this, you are limited by what data you can collect and label, and labeling data is an extremely time consuming process because it relies on humans. But also, you don’t really need labeled data if you don’t care about correctness. You can get away with feeding the regression machine a load of data and label more generally based on how close certain points are in a vector space. It’s how “sentiment analysis” works. You can take IMDB’s database of reviews that each have some words and a star rating, and use the star rating to categorize in “good” and “bad”, then average out the distance between certain words and the frequency they appear within the “good”, “mid”, and “bad” spectrum.
Suddenly, relationships show up, “lawyer” is close to “criminal” in the 3d space.
What modern LLMs do, is just layer this same system with a few short-circuits called context windows. It basically maintains a space of relevance within the broader context of what the model was trained in. For the IMDB example, lets say you’re asking a machine about action movies with x, y, z characteristics, the context maintains those to short-circuit the larger model to retain ‘focus’ and give you near-by relationships.
With enough data you can recreate language based on distance markers and frequency. But back to my original point, it’s the surface level of what it was trained on, a plaster mask. The mask doesn’t have the complexity of the muscles and skin it was formed on, it’s shallow.
That all being said, the ability to make a shallow mask is useful for cross-referencing large amounts of data. The disaster strikes when it’s treated as an all knowing god and used to do military strikes.
Or fire people.
We gave computers a stack of transparencies and told it to pick a couple at random to make something new and we pretended it was smart. And now people really be thinking humans aren’t needed for production anymore.