Re: Request for Information (RFI) Related to NIST's Assignments Under Sections 4.1, 4.5, and 11 of the Executive Order Concerning Artificial Intelligence (Sections 4.1, 4.5, and 11) (88 Fed. Reg. 88368)
Dear Director Locascio:
The U.S. Chamber of Commerce (“Chamber”) appreciates the opportunity to provide comment to the National Institute of Standards and Technology (NIST) on its “Request for Information (RFI) Related to NIST’s Assignments Under Section 4.1, 4.5 and 11 of the Executive Order Concerning Artificial Intelligence). Furthermore, we appreciate NIST’s previous work, such as the Artificial Intelligence Risk Management Framework (RMF), which the business community highlights as an essential step for developing responsible Artificial Intelligence (“AI”).
The Chamber believes that trust is essential for the development and deployment of AI. Indeed, it is often unrecognized that AI tools such as machine learning, chatbots or personal assistants have already been in use for years. Accordingly, we reiterate the Chamber’s interest in partnering with NIST to address those matters that are critical for the safe and responsible development of AI.
We remain concerned with the short comment period provided. Haste can lead to sloppy and ill-informed policy developments that will blunt the ability of society and the American economy to reap the benefits of AI. Accordingly, the U.S. Chamber, with other associations,1 sent a letter on January 9th emphasizing that “given the complex policy and technical issues associated with the request for information, we respectfully request a 60-day extension to allow stakeholders to provide the necessary and thoughtful comments needed to inform policies aimed at maximizing the utilization of AI while ensuring the trustworthiness of AI.” Unfortunately, that request for an extension was denied, and therefore the business community will only be able to provide limited feedback and comments. This unfortunately could, in our view, lead to potentially inferior rules that may hamper the ability of the United States to reap the benefits of AI and to remain the global leader in this groundbreaking technology.2
General Feedback:
First, standards and best practices remain important at the global level. Executive Order 14410 tasks the Department of Commerce with international standards regarding Artificial Intelligence. International harmonization is essential for the business community and innovation at scale to allow the technology to be utilized for the common good. NIST and other US government agencies must engage in helping incorporate industry-supported and agreed-upon standards with appropriate international standards bodies.
NIST must also build flexibility in any developed framework. The Chamber has strong concerns that a one-size-fits-all approach limits innovation and necessary adaption to best practices, which helps limit the risk associated with any model. Furthermore, any NIST guidance, including a companion document for generative AI, should complement the existing framework and core components for AI governance established in the NIST AI RMF. Any guidance must also be developed risk- and context-dependent, as these tools have different risks based on their utilization.
Foundational principles and existing legal regimes already govern AI, Cyber Security, Intellectual Property, and Data Privacy, which must be accounted for and reinforced within any guidance. For this reason, NIST should use this framework to address specific risks associated with sector-specific utilization of AI, which can ensure that any recommendations are rooted in best practices and law. We would further encourage NIST to develop guidance highlighting applicable alignment with existing laws and regulations in specific sectors that already enforce risk management and evaluation.
Finally, NIST should recognize that the size and scope of businesses that utilize the technology are vastly different. This means that specific access to resources and tools will be profoundly different based on the entity's specific sector and other factors. For this reason, NIST should be mindful of these limitations and differences as the agency develops any updated or new framework, as companies or organizations will not have the same specific means, resources, or specializations that may be called for within a developed one-size-fits-all framework.
I. Red Teaming
Regarding the requests for comment on “red-teaming,” the Chamber would like to highlight the importance of risk-based industry-specific guidance. The Chamber supports NIST’s work to develop a companion document for the AI RMF, providing guidelines for necessary governance and changes based on the current AI RMF structure. Additionally, regarding external vendors, future voluntary RMF guidance could expand on how red teaming applies to vendors and vendor accountability, including to what extent deployers can rely on red teaming conducted by vendors.
II. Reducing the Risk of Synthetic Content
The risks associated with synthetic content are complex issues, and there are limitations that developers face when addressing these matters on their own. This is why the entire stakeholder community must address this matter within the ecosystem in a manner that accounts for the risk associated with the technology and considers the specific industry and use of technology. For this reason, further discussions are necessary on this issue to discuss the important roles in helping reduce and mitigate risk associated with synthetic content.
III. Hallucination
Large Language Models (LLMs) will hallucinate, not because they have specific malicious intent but because these tools are statistical models that correlate across vast amounts of information and make predictions based on that information. For example, the reliability of data on the internet, used by LLMs to train, helps to create the conditions for hallucination. This is why training data needs to be curated to allow for better responses and lower the probability of hallucinations. Furthermore, reinforcement learning from human feedback is another way to reduce the risk of hallucinations. Through this process, the model can be tuned, and outputs categorized in the most valuable ways for its utility. Another way to reduce risk and address hallucinations is by changing data inputs through multi-modal retrieval and multi-model inference by adding voice, video, and audio to code. This will allow for more accurate and human-like results through increasing data sources.
Conclusion
While we have serious process concerns triggered by the Executive Order, the Chamber appreciates NIST’s role in developing the NIST AI RMF and work concerning the responsible use of AI. The Chamber stands ready and willing to work with NIST to ensure that AI is developed and used in a responsible manner.
Sincerely,
Michael Richards
Director, Policy
Chamber Technology Engagement Center
1 https://www.uschamber.com/technology/multi-association-comment-extension-request-on-nists-assignmentsunder-sections-4-1-4-5-and-11-of-the-ai-executive-order 2 See for instance, (Cite Gensler AI Letter) letter to the SEC on roundtables and stakeholder input.
