The defense industry lost the ability to make weapons when crisis hit. The same pattern is eroding software engineering skills. The timelines are identical.
  • zalgotext@sh.itjust.works
    12·
    3 days ago

    We have computing power in our pockets a million times more powerful than we used to send man to the moon, why do you think we’ll never have enough power?

    Not the person you replied to, but I have thoughts on this point in particular:

    1. Consumer devices have started to slow down their performance improvements because we’re bumping up against the limits of physics
    2. People/corporations with way more money than the average consumer will always be able to run something orders of magnitude more powerful. Any advances that improve things for the average consumer will improve things for rich people/corporations even more.
    3. Training an LLM isn’t really even about compute speed, it’s about access to good training material. The average consumer can’t afford to buy (or pirate) every book in existence like a rich person/corporation can. An average person doesn’t have the ability or time to curate their own training data, but rich people/corporations do.
    • humanspiral@lemmy.ca
      1·
      2 days ago

      Training an LLM isn’t really even about compute speed, it’s about access to good training material.

      Consumer can’t train from scratch but can fine tune/modify open weights model with their data. There is significant open source training material available, without needing/wanting Harry Potter knowledge.