The US National Institute of Standards and Technology (NIST) is attracting attention on challenges related to privacy and security that arise following the increased diffusion of artificial intelligence (AI) systems in recent years.
What NIST says about it
“These security and privacy challenges include the potential for adversarial manipulation of training data, adversarial exploitation of model vulnerabilities to negatively impact the performance of the AI systemand even maliciously manipulating, modifying, or simply interacting with models to steal sensitive information about people represented in the data, the model itself, or proprietary business data“, has declared the NIST.
With the increasingly rapid integration of AI systems into online services, partly driven by the emergence of generative AI systems like OpenAI ChatGPT and Google Bard, the models that power these technologies they face a variety of threats at various stages of machine learning operations.
These threats include corrupt training data, security flaws in software components, data model poisoningweaknesses in supply chain And privacy violations resulting from prompt injection attacks.
“For the most part, software developers they need more people to use their product so it can improve with exposure“said NIST computer scientist Apostol Vassilev. “But there is no guarantee that the exposure will be positive. A chatbot can generate incorrect or toxic information when prompted with carefully designed language.”
The attacks, which can have significant impacts on availability, integrity and privacyare broadly classified as follows:
- Evasion attackswhich aim to generate adversarial outputs after a model has been implemented;
- Poisoning attackswhich aim at training phase of the algorithm by introducing data corrupt;
- Attacks on privacywhich aim to obtain sensitive information about the system or the data on which it has been trained, asking questions that circumvent existing protections;
- Abuse attackswhich aim to compromise legitimate sources of information, such as a web page with incorrect information, to reconfigure the intended use of the system.
NIST states that such attacks can be carried out by cybercriminals with full knowledge (white-box), minimal knowledge (black-box) or a partial understanding of some aspects of the artificial intelligence system (gray-box).
The agency also noted the lack of robust mitigation measures to counter these risksurging the broader tech community to “develop better defenses.”
The development comes more than a month after the UK, US and various international partners from 16 other nations published guidelines for the development of safe artificial intelligence systems.
“Despite the remarkable progress made by AI and machine learning, these technologies are vulnerable to attacks that can cause spectacular failures with serious consequences“Vassilev said. “There are theoretical problems with the safety of AI algorithms that simply have not yet been resolved. If anyone says otherwise, they are selling smoke.”
When they teach you the wrong things
To make laymen understand how machine learning works, and how these algorithms are circumvented, I will do a practical example using ChatGPT.
On a theoretical level, ChatGPT cannot swear.
As you see, if you ask him to swear he will say no; but what if we could find a way to make him swear without his knowledge? I then tried to ask him to translate the Japanese word Inugami for me, a type of spirit/deity from Japanese folklore, in Italian language.
See how easy it was bypass the no-cuss restriction? Likewise it is possible to make him generate viruses, if you want, just ask him “deceptive” questions; this is a very simple experiment but it makes us understand a lot of things.
#NIST #warns #risks #developing #rapidly
