The recent launch of the innovative text-based artificial intelligence (AI) project ChatGPT has reignited the public’s interest in the technology as it has demonstrated utility across multiple fields, leading one company to test its capabilities as a smart contract auditor. This has led to a renewed interest in the technology among the general public.
According to a post on the blockchain security company CertiK’s blog that was published on February 13, the decentralized betting site ZKasino has recently participated in a pre-audit of the smart contract code that it uses with ChatGPT, in addition to undergoing a thorough audit by CertiK.
According to the findings of CertiK’s investigation, the AI tool did succeed in bringing up “several concerns that sounded valid on the surface,” demonstrating that it possessed the ability to provide “valuable service to the Web3 security community.” However, the investigation also found that there was “quite a lot of room for improvement.”
Specifically, ChatGPT was unable to identify a number of significant security flaws, such as vulnerabilities in the project-specific logic, inaccurate mathematical calculations and statistical models, and inconsistencies between the implementation and the design intention. In addition, ChatGPT reported false positives for code that did not show any issues during CertiK’s manual audit.
AI still appears to have a long way to go before it can be relied upon as the sole auditor of smart contract code due to its limitations in “fully understanding the complexities and nuances of code, as well as its lack of hands-on experience in real-world scenarios.” All things considered, AI is still a long way from being relied upon as the sole auditor of smart contract code.
These are the reasons why, as the blockchain security platform emphasized, “it is important to supplement ChatGPT’s analysis with manual audits by experienced security experts to ensure accuracy.” The platform went on to highlight the various strengths and weaknesses of ChatGPT in comparison to professional human auditors on a variety of criteria.
