21923 - SCTE Broadband Aug2023 COMPLETE v1 (1)

89 Vol. 45 No. 3 - September 2023 Issue technical Tiulkanov’s chart shown in Figure 4 focuses on only one of the easily identified sources of harm for a ChatGPT-powered application, the trustworthiness of the output. When all of the potential harms are considered, perhaps Figure 5 is a more realistic approach. Making things safer Even though there are many paths to ‘no’ and only one to ‘yes’ on the more detailed decision tree in Figure 5, there will still be a lot of applications that get qualified as reasonable. But this will not make them safe. In order to have confidence in a ChatGPT- powered application, it is also suggested that the following steps are implemented. 1. There should be no deception about what it is that users are interacting with. You cannot give informed consent if you are not informed. Saleema Amershi et-al [3] have published excellent guidelines for interaction for AI systems. Importantly, these provide structure for considering interaction throughout the lifecycle of a user interaction. The guidelines cover how to make it clear to the user what they are interacting with and how to instruct them about what is expected of them. Amershi’s guidance extends throughout the interaction, managing failure and overtime as the system becomes ‘business as usual’ 2. Users should have the option to not interact with the system. A real option - for example an alternative contact channel 3. There should be an impact assessment attached to every application. Put it on the website as you would a robots. txt file, or as you would add a licence to your source code. The Canadian AIA process offers a model for this sort of thing, but some fundamental questions are a good start. Who will it hurt if it works as intended? Who will be hurt if the chatbot goes wrong? Can anyone tell if the chatbot is going wrong, and can they stop it and repair the situation if it is? 4. If your system could have an adverse effect on others then there should be monitoring and logging of what the system is doing and how it is behaving. These should be maintained in such a way as to allow forensic investigation of the behaviour of the system, if required 5. If you are not personally and directly responsible for the system, a clearly documented governance process should be developed and maintained. Part of this should describe how users can call for help, and how they can complain about the system. It should also describe what the processes around addressing user distress and complaints should be. In this paper, I have laid out what the potential problems are with ChatGPT- based applications and some tactics for avoiding and mitigating them in practice. I hope that our community deepens and develops these approaches in the near future. With the correct controls and processes, new Large Language Models such as ChapGPT will provide great value in many use-cases, albeit with the essential controls and checks in place, to ensure users and end-users are protected from any misunderstanding. For more information, see www.gft.com 1. Luby J, Kertz S. Increasing Suicide Rates in Early Adolescent Girls in the United States and the Equalization of Sex Disparity in Suicide: The Need to Investigate the Role of Social Media. JAMA Netw Open. 2019;2(5). doi:10.1001/ jamanetworkopen.2019.3916 2. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. ‘Bert: Pre- training of deep bidirectional transformers for language understanding.’ arXiv preprint arXiv:1810.04805 (2018). 3. Glaese, Amelia, Nat McAleese, Maja Tr ę bacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh et al. ‘Improving alignment of dialogue agents via targeted human judgements.’ arXiv preprint arXiv:2209.14375 (2022). 4. Amershi, Saleema. ‘Guidelines for Human-AI Interaction.’ CHI conference on human factors in computing systems. CHI, 2019. 1–13. 5. Pearce, Hammond, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. ‘Asleep at the keyboard? assessing the security of github copilot’s code contributions.’ In 2022 IEEE Symposium on Security and Privacy (SP), pp. 754- 768. IEEE, 2022.