Prompt Injection Defenses Against LLM Cyberattacks
Interesting research: “Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks“:
Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense strategy tailored to counter LLM-driven cyberattacks. We introduce Mantis, a defensive framework that exploits LLMs’ susceptibility to adversarial inputs to undermine malicious operations. Upon detecting an automated cyberattack, Mantis plants carefully crafted inputs into system responses, leading the attacker’s LLM to disrupt their own operations (passive defense) or even compromise the attacker’s machine (active defense). By deploying purposefully vulnerable decoy services to attract the attacker and using dynamic prompt injections for the attacker’s LLM, Mantis can autonomously hack back the attacker. In our experiments, Mantis consistently achieved over 95% effectiveness against automated LLM-driven attacks. To foster further research and collaboration, Mantis is available as an open-source tool: this https URL.
This isn’t the solution, of course. But this sort of thing could be part of a solution.
Subscribe to comments on this entry
Garabaldi • November 12, 2024 10:31 AM
Are LLM actually deterministic system? The training uses massively parallel processing. Parallel processing is subject to all sorts of timing effects. Eliminating timing effects takes a lot of effort. I would not expect that to be a priority of the people working on this.
I would be very surprised if rerunning the training results in the same weights or the same hallucinations. (Not no hallucinations, just different ones.)