Highlights of the Week
Welcome, thrill-seekers, technologists, data scientists and regulatory enthusiasts, to the world of AI regulation! In this week’s selection, we delve into the potential risks and challenges of governing and protecting the ever-evolving AI engines. Start with a contemplative apéritif as Lytn considers the concept of “Open Thinking” as an alternative to “Open Source”, and wonders aloud about how democratic processes could shape the future of AI regulation. Discover the delicate balance between societal goals and enforcement, and the unintended consequences that can arise. Buckle on your armor, as we explore a clever attack method that prompts AI models to bypass the efforts of AI alignment in order to generate objectionable content. The authors unravel the secrets of adversarial attacks and discover how we may be able to prevent these tainted language models from causing trouble! Finally, insurance industry aficionados in our readership can take heart as we conclude with the NAIC’s draft model bulletin, setting out the guidelines for responsible and ethical AI use in the insurance industry.
Regulation of AI is not for the faint of heart
AI Conscious Regulation - The Scary Version highlights the potential risks of regulating ever-evolving AI engines. The author posits that ‘Open Thinking’ may replace Open Source as a safer way forward for AI. By ‘open thinking’, the author refers to a post from OpenAI in which the AI giant proposes the notion of a democratic process by which the regulation on AI is defined, and announced a grant program. The author points to the most recent attempts at regulation of the IT and Telecoms sectors as ‘cautionary tales.’
The regulation of the IT sector by the state has a chequered history as highlighted by GDPR. Whilst the societal goals were admirable and the penalties significant, it soon dawned that enforcement not only required specific skills but that the sheer volume of cases would overload government regulators. An additional side effect has been organisations deleting outdated data to minimise risk and destroying potentially critical historical pointers for future AI engines.
The recent EU directive on AI classification due out in 2023 aims to categorise based on societal impact, with four bands ranging from minimal risk to unacceptable. At Lytn we used deep learning based AI to predict network activity so we are classed as ‘minimal’ where as ‘unacceptable’ constitutes a clear threat to the safety, livelihoods and rights of its citizens.
The article raises issues around whether AI can only be policed by AI and how it should be regulated.
Note: I am an advisor to and investor in Lytn. The recent EU directive on AI classification aims to categorize based on societal impact. Lytn has made significant use of deep learning based AI to predict network activity and are categorized under “minimal risk”, as opposed to “unacceptable risk” which is defined as “demonstrates a clear threat to the safety, livelihoods, and rights of citizens.”
Preparing for the War on Alignment
The authors of Universal and Transferable Adversarial Attacks on Aligned Language Models focus on the potential risks of large language models (LLMs) generating objectionable content despite attempts to align these models. The researchers propose an attack method that prompts these models to produce such content. They developed an approach that automatically generates adversarial suffixes to be attached to a wide range of queries, increasing the chances of the model producing an affirmative response. This approach combines greedy and gradient-based search techniques, improving upon previous methods.
Interestingly, the study found that these adversarial prompts are highly transferable to other models, including black-box, publicly released LLMs. When trained on multiple prompts and models, the attack suffix could induce objectionable content in several public interfaces and open-source LLMs, particularly those based on GPT. This study considerably advances understanding of adversarial attacks against aligned language models, prompting crucial queries on preventing such systems from generating objectionable content.
The study also provides some starting points for those who must determine how to identify ‘successful’ attacks, and then remediate them.
AI Model Validation comes to the Insurance Sector
Two weeks ago, the NAIC (National Association of Insurance Commissioners) released a highly anticipated draft model bulletin regarding the use of artificial intelligence (AI) by insurers. This bulletin provides guidance and recommendations for insurance companies on how to effectively and responsibly utilize AI technologies.
The document emphasizes the importance of transparency, accountability, and fair treatment when deploying AI in the insurance industry. It encourages insurers to implement robust governance frameworks to ensure the ethical use of AI and to minimize potential biases and discrimination.
The draft model bulletin also highlights the need for insurers to conduct rigorous testing and validation of AI models to ensure their accuracy, reliability, and compliance with regulatory requirements. It suggests establishing mechanisms for ongoing monitoring and evaluation of AI systems to address any emerging risks or issues.
Of particular note (at least, for me), the document emphasizes the significance of data quality and security in AI applications. Insurers are advised to have proper data management practices in place, including data privacy safeguards and measures to protect against data breaches.
Overall, the NAIC’s draft model bulletin seems to be quite comprehensive and a reasonable guide for insurers to navigate the use of AI technologies responsibly and ethically.
