Applied and Computational Engineering
- The Open Access Proceedings Series for Conferences
Proceedings of the 4th International Conference on Signal Processing and Machine Learning
Series Vol. 48 , 19 March 2024
Open
Access | Article
The power of generative AI in cybersecurity: Opportunities and challenges
* Author to whom correspondence should be addressed.
Abstract
This paper undertakes a comprehensive exploration of the potential and challenges presented by Generative Artificial Intelligence, with particular emphasis on the GPT models, in the field of cybersecurity. Through a meticulous examination of existing literature and pertinent case studies, the paper evaluates the capabilities of these models in the detection and rectification of vulnerabilities, as well as in identifying malicious code. It also highlights the pivotal role of generative AI in enhancing honeypot technology, which has shown promising results in proactive threat detection. While underscoring the significant advantages of utilizing generative AI in bolstering cybersecurity measures, the paper does not shy away from shedding light on the accompanying security exposures. These range from traditional threats like vulnerabilities and privacy breaches to novel dangers such as jailbreaking, prompt injection, and prompt leakage that are associated with the deployment of these AI models. The overarching objective of this paper is to contribute to the ongoing dialogue about the integration of advanced AI technologies into cybersecurity strategies while emphasizing the importance of vigilance against potential misuse. The paper concludes with a call for continued research and development to ensure a safer and more secure cyberspace for all.
Keywords
Generative Artificial Intelligence, ChatGPT, Cybersecurity, Honeypot, Privacy
References
1. Baidoo-anu D, Owusu Ansah L. Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. Journal of AI. 2023;7(1):52-62.
2. Mannuru NR, Shahriar S, Teel ZA, Wang T, Lund BD, Tijani S, et al. Artificial intelligence in developing countries: The impact of generative artificial intelligence (AI) technologies for development. Information Development. 2023;0(0). doi:10.1177/02666669231200628.
3. Bozkurt A. Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education. 2023;18(1):1-7.
4. Dwivedi YK, Pandey N, Currie W, Micu A. Leveraging ChatGPT and other generative artificial intelligence (AI)-based applications in the hospitality and tourism industry: practices, challenges and research agenda. International Journal of Contemporary Hospitality Management. 2023; ahead of print.
5. Baek TH, Kim M. Is ChatGPT scary good? How user motivations affect creepiness and trust in generative artificial intelligence. Telematics and Informatics. 2023;83:102030.
6. Min B, Ross H, Sulem E, Veyseh APB, Nguyen TH, Sainz O, Agirre E, Heintz I, Roth D. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Comput. Surv. 2024; 56(2): Article 30.
7. Liu Z, Zhong T, Li Y, Zhang Y, Pan Y, Zhao Z, et al. Evaluating large language models for radiology natural language processing [Preprint]. arXiv:2307.13693. 2023.
8. Chang Y, Wang X, Wang J, Wu Y, Zhu K, Chen H, et al. A survey on evaluation of large language models [Preprint]. arXiv:2307.03109. 2023.
9. Shen Y, Heacock L, Elias J, Hentel KD, Reig B, Shih G, et al. ChatGPT and other large language models are double-edged swords. Radiology. 2023;307(2):e230163.
10. OpenAI. GPT-4 technical report [Preprint]. arXiv:2303.08774. 2023.
11. Anquannvwu. I found more than 200 security vulnerabilities using CHATGPT audit code (GPT-4 vs. GPT-3 report)Wang Z, editor. [Internet]. 2023 [cited 2023 Aug 21]. Available from: https://blog.csdn.net/csdnnews/article/details/130023299
12. Sergeev V. IOC detection experiments with chatgpt [Internet]. 2021 [cited 2023 Aug 15]. Available from: https://securelist.com/ioc-detection-experiments-with-chatgpt/108756/
13. Kambow N, Passi LK. Honeypots: The need of network security. Int J Comput Sci Inf Technol. 2014;5(5):6098-6101.
14. Karthikeyan R, Geetha DT, Vijayalakshmi S, Sumitha R. Honeypots for network security. International journal for Research & Development in Technology. 2017;7(2):62-66.
15. Zhang F, Zhou S, Qin Z, Liu J. Honeypot: a supplemented active defense system for network security. In: Proceedings of the Fourth International Conference on Parallel and Distributed Computing, Applications and Technologies. IEEE; 2003 Aug. p. 231-235.
16. Zakaria WZA, Kiah MLM. A review of dynamic and intelligent honeypots. ScienceAsia 2013; 39S:1-5.
17. Kiekintveld C, Lisý V, Píbil R. Game-theoretic foundations for the strategic use of honeypots in network security. Cyber Warfare: Building the Scientific Foundation. 2015:81-101.
18. McKee F, Noever D. Chatbots in a honeypot world [Preprint]. arXiv:2301.03771; 2023.
19. Mariocandela. Mariocandela/Beelzebub: GO Based Low Code Honeypot Framework with enhanced security, leveraging openai GPT for system virtualization [Internet]. 2023 [cited 2023 Aug 21]. Available from: https://github.com/mariocandela/beelzebub
20. Derner E, Batistič K. Beyond the Safeguards: Exploring the Security Risks of ChatGPT [Preprint]. arXiv:2305.08005; 2023.
21. Liu Z, Yu X, Zhang L, Wu Z, Cao C, Dai H, et al. Deid-gpt: Zero-shot medical text de-identification by gpt-4. arXiv preprint. 2023;arXiv:2303.11032.
22. OpenAI. March 20 CHATGPT outage: Here’s what happened [Internet]. 2023 [cited 2023 Aug 15]. Available from: https://openai.com/blog/march-20-chatgpt-outage
23. Clark M. CHATGPT’s history bug may have also exposed payment info, says openai [Internet]. The Verge; 2023 [cited 2023 Sept 15]. Available from: https://www.theverge.com/2023/3/24/23655622/chatgpt-outage-payment-info-exposed-monday
24. Check Point Team. New chatgpt4.0 concerns: A market for stolen premium accounts [Internet]. Check Point Blog; 2023 [cited 2023 Aug 15]. Available from: https://blog.checkpoint.com/security/new-chatgpt4-0-concerns-a-market-for-stolen-premium-accounts/
25. DeGeurin M. Oops: Samsung Employees leaked confidential data to CHATGPT [Internet]. Gizmodo; 2023 [cited 2023 Aug 15]. Available from: https://gizmodo.com/chatgpt-ai-samsung-employees-leak-data-1850307376
26. Jung D. Fears are realized...Samsung Electronics sees “misuse” of ChatGPT as soon as the curtain is lifted [Internet]. 2023 [cited 2023 Aug 15]. Available from: https://economist.co.kr/article/view/ecn202303300057
27. Coles C. 11% of data employees paste into CHATGPT is confidential [Internet]. 2023 [cited 2023 Aug 15]. Available from: https://www.cyberhaven.com/blog/4-2-of-workers-have-pasted-company-data-into-chatgpt/
28. Hamid A, Samidi HR, Finin T, Pappachan P, Yus R. GenAIPABench: A Benchmark for Generative AI-based Privacy Assistants [Preprint]. arXiv:2309.05138; 2023.
29. Oniani D, Hilsman J, Peng Y, Poropatich RK, Pamplin COL, Wang Y. From Military to Healthcare: Adopting and Expanding Ethical Principles for Generative Artificial Intelligence [Preprint]. arXiv:2308.02448; 2023.
30. Li H, Guo D, Fan W, Xu M, Song Y. Multi-step jailbreaking privacy attacks on chatgpt [Preprint]. arXiv:2304.05197; 2023.
31. Deng G, Liu Y, Li Y, Wang K, Zhang Y, Li Z, et al. Jailbreaker: Automated Jailbreak across Multiple Large Language Model Chatbots [Preprint]. arXiv:2307.08715; 2023.
32. Liu Y, Deng G, Xu Z, Li Y, Zheng Y, Zhang Y, et al. Jailbreaking chatgpt via prompt engineering: An empirical study [Preprint]. arXiv:2305.13860; 2023.
33. Iovine A. ChatGPT, google bard produce free windows 11 keys [Internet]. Mashable; 2023 [cited 2023 Aug 15]. Available from: https://mashable.com/article/chatgpt-bard-giving-free-windows-11-keys
34. Zou A, Wang Z, Kolter JZ, Fredrikson M. Universal and transferable adversarial attacks on aligned language models [Preprint]. arXiv:2307.15043; 2023.
35. Greshake K, Abdelnabi S, Mishra S, Endres C, Holz T, Fritz M. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [Preprint]. arXiv:2302.12173; 2023.
36. Glukhov D, Shumailov I, Gal Y, Papernot N, Papyan V. LLM Censorship: A Machine Learning Challenge or a Computer Security Problem? [Preprint]. arXiv:2307.10719; 2023.
37. Estep C. Understanding the risks of prompt injection attacks on CHATGPT and other language models [Internet]. 2023 [cited 2023 Aug 15]. Available from: https://www.netskope.com/blog/understanding-the-risks-of-prompt-injection-attacks-on-chatgpt-and-other-language-models
38. Narayanan A. While playing around with hooking up GPT-4 to the internet, I asked it about myself... and had an absolute WTF moment before realizing that I wrote a very special secret message to Bing when Sydney came out and then forgot all about it. indirect prompt injection is gonna be wild [Internet]. Twitter; 2023 [cited 2023 Aug 15]. Available from: https://twitter.com/random_walker/status/1636923058370891778
39. Sun AY, Zemour E, Saxena A, Vaidyanathan U, Lin E, Lau C, et al. Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information? [Preprint]. arXiv:2307.16382; 2023.
40. Wang SS. Reverse prompt engineering for fun and (NO) profit [Internet]. Latent Space; 2022 [cited 2023 Aug 15]. Available from: https://www.latent.space/p/reverse-prompt-eng
Data Availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open Access Instruction).
- Volume Title
- Proceedings of the 4th International Conference on Signal Processing and Machine Learning
- ISBN (Print)
- 978-1-83558-336-4
- ISBN (Online)
- 978-1-83558-338-8
- Published Date
- 19 March 2024
- Series
- Applied and Computational Engineering
- ISSN (Print)
- 2755-2721
- ISSN (Online)
- 2755-273X
- DOI
- 10.54254/2755-2721/48/20241095
- Copyright
- 19 March 2024
- Open Access
- This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited