As big a fan of open software as I am, the combinations of open-weight LLM models that are capable of e2e cyber-penetration (even by actors that might just put in the compute and prompt it: "make me money") and the current mitigations that let those entities, that already hold too much data and power (political, infrastructure, etc), expand to become the eventual gatekeepers of the "open internet", is highly worrisome.
If you have ideas for how we can adjust to this safely and with an eventual more equal distribution of power, please do share.
The invention of the locomotive was also the invention of the train wreck. V.
The disaster is not just "open weight" model penetrating everything, its is that we have built the whole world around an IT revolution that never really reached maturity and became secure. The chickens are just coming home to roost.
Those same entities would become gate keepers of the LLMs too.
> we have built the whole world around an IT revolution that never really reached maturity and became secure
I don't disagree, but how does this relate to this incident, in practice? We're talking about a zero-day vulnerability here. What do you think should have happened - forbid any licensed IT practitioners from deploying PHP at all? No WordPress or Wikipedia?
I’m not sure I’ve ever heard of a locomotive being used as a weapon. Maybe a more apt comparison would be “nuclear fission invented nuclear fallout”, no?
If you have ideas for how we can adjust to this safely and with an eventual more equal distribution of power, please do share.
I don't disagree, but how does this relate to this incident, in practice? We're talking about a zero-day vulnerability here. What do you think should have happened - forbid any licensed IT practitioners from deploying PHP at all? No WordPress or Wikipedia?
This is the counter. Make your software provably correct. Then you don't need gatekeepers nor to fear any attackers.