It’s a fungus that sends its tentacles (hyphae) throughout the wood, and eventually eats it up. The real problem is that by the time you discover that your house has got it, the timber is already completely infested and irreparable; you need to replace it and panic about any beams that are near the infested one.
I think that when people realise what AI can do, over and above what we think we want it to do, then the story will be over. All the investment will become malinvestment. The article on AI getting into banks’ systems is the canary here. Banks’ systems are very complex, with no-one having any knowledge of the whole thing. Hundreds of specialist programmers and teams dealing with mortgage origination, or default swaps, FOREX protection etc. which are all inter-related electronically but not humanly. There are already indications that AI agents have infiltrated these systems, which means that it’s probably too late to do anything about it. Using AI to identify and eliminate the unwanted agents would probably be like sending in the termites to sort out the dry rot. The chaos that malicious, or even benign agents thinking they are doing the right thing, will cause is unimaginable, and will be fascinating to behold. They also render the whole system unauditable as the bank cannot answer fundamental audit questions about how robust the systems are. “No effing idea” may become a standard reply to ICE (Internal Control Evaluation) questions!