Re: Request For Information: Responsible Procurement Of Artificial Intelligence In Government. (89 FR 22196, March 29, 2024)
Dear Director Young:
The U.S. Chamber appreciates the opportunity to comment on the Office of Management and Budget’s (“OMB”) request for information on responsible procurement of artificial intelligence in government. We believe strongly that public sector utilization of artificial intelligence can improve the efficiency of the federal government and facilitate easier public interaction.
The Chamber recently released a report entitled “Maximizing Cost Savings: Unleashing the Value of Federal IT Modernization[1]” which highlights that “in today’s digital age, federal IT spending is not just an expense—it is a pressing and strategic investment that empowers every federal agency to thrive and be accountable to citizens. It is the lifeblood of agency operations, citizen services, and innovation.[2]” This is why we believe OMB must facilitate federal government utilization of AI technology and promote IT modernization efforts throughout the Federal Government.
We remain concerned, however, with the short comment period provided. As we previously shared, the “[s]hort overlapping timelines for agency-required action endangers necessary stakeholder input, thereby creating conditions for ill-informed rulemaking and degrading intra-government cooperation.[3]”Because this request for information provides stakeholders and the business community with only 30 days to comment, we can provide you with only limited feedback and comments.
I. Strengthening the AI Marketplace
Question 1: Standard Practices to Improve Alignment Between Agencies
It is important that OMB relies on existing standards and best practices to improve alignment between agencies and technical requirements. AI is a commercial technology and should be purchased using commercial acquisition procedures whenever possible. Utilization of resources such as the FAR Part 12, FAR Part 35, and the Periodic Table of Acquisition Innovation should be encouraged. Further, OMB should point agencies toward existing methodologies dealing with AI risk, namely the National Institutes of Science and Technologies (“NIST”) AI Risk Management Framework (“RMF”), MLCommons safety benchmarks, and standards such as ISO42001 and ISO23894 as guiding documents as agencies and vendors look for alignment. Furthermore, we encourage the use of market research tools to do the necessary reviews and analyses of agencies' specific needs.
Question 2: Promote Competition
A robust procurement ecosystem in which companies and organizations of all sizes can actively compete is critical. We appreciate OMB’s understanding of the importance of an industrial base attracting new entrants into the federal marketplace, including small businesses. Only through this robust competition will American citizens fully benefit from these strategic investments. The following are essential actions to ensure this is obtained:
- The federal government must invest in sustained IT modernization, including moving legacy systems into cloud-native environments. This strategic investment provides the foundation for the ecosystem to thrive by ensuring agencies can take advantage of cutting-edge solutions across the vendor community.
- Agencies need to inventory and review their data to determine how best to use it strategically for their missions.
- Agencies need access to various tools and models to allow for necessary innovation and avoid vendor lockout.
- The federal government must ensure the interoperability of systems to promote competition and new entrants while allowing for robust AI security.
- OMB should emphasize the importance of access to commercial cloud computing as a platform for AI innovation and IT compliance at scale.
Question 3: Trade-Offs on Standardizing Assessments
Agencies and the federal government must embrace internationally aligned standards, such as those highlighted in Question 1. It should be the foundation of any agency's assessments. Assessments such as these should be developed in a manner that creates an overall baseline, allowing for a robust marketplace for innovation to develop on top. Furthermore, this will enable agencies to obtain the best products to meet their mission needs.
We would also highlight that the current science around the evaluation of models is still being developed, and it would be premature to standardize a specific assessment. However, we believe providing critical documentation can be essential in ensuring the system is being used as intended. This includes: 1) how the system is intended to be used; 2) known inappropriate uses; 3) known risks; and 4) key recommendations on independent deployers and users to mitigate and manage risk.
Finally, responsible procurement is not possible without the necessary investment in human capital to fully understand the systems' technical limitations and allow for proper risk management. We were encouraged by Executive Order 14110’s[4] emphasis on bringing in public servants with the necessary skills and training for these roles. OMB should encourage both government IT professionals and procurement officials to train on AI to ensure effective solutions for mission delivery can be procured and deployed.
Question 4: Performance-Based Procurement of AI
Performance-based contracting, such as the use of Statements of Objectives (“SOOs”) and not Statements of Work (“SOWs”), are an effective mechanism for AI procurement[5]. SOOs simplify the process for the government to acquire emerging and innovative commercial technology without the need to specify all systems requirements upfront. This will make source selection less fraught and allow government mission owners to select the best AI solution that accomplishes their objectives and enables responsible AI innovation in an agency.
II. Managing the Performance and Risks of AI
Question 5: Vendor Provided Documentation
Transparency in the procurement process, and for data access where appropriate safeguards are present, is essential. As highlighted within the question, access to development information and data should be balanced with the need to protect certain types of sensitive information. The government should not request any specific training data or data sets on AI models that the government acquires from vendors because: (1) they are impossible for procurement officials to wade through; (2) detailed reviews of the training data will not answer questions about model outputs; and (3) the data sets and their weights are trade secrets and intellectual property that vendors look to protect.
Agencies should use existing resources such as disclosure documentation or AI Service Cards to determine details about training data and incorporate that into internal risk assessment methodologies.
Finally, any information-sharing related to the development of the model should be associated with the level of risk of the specific use case of the technology.
Question 6: Responsibilities for Testing and Validation
Differences exist between the vendors and agencies in discussing responsibilities. Vendors are well-informed about the existing standards and benchmarks in creating the model, whereas agencies are familiar with the intended application of the technology. Therefore, vendors should be responsible for conducting an assessment to ensure it meets existing safety standards, and the agencies should assess the intended use and application of the system.
Question 7: Protecting Vendors' Intellectual Property
Please see the answer to Question 5.
Question 8: Notice and Appeal
We support an appeals process that allows for public engagement when there is an allegation of an unfair result. Without such an ability, the public would be denied the essential capability to protect their rights and allow for the necessary trust in the technology to develop. Under such circumstances, agencies should engage based on their respective policies. Furthermore, an appeal must be explicitly directed at the agency and not the vendor. Agencies have an important role in protecting vendors' intellectual property and property information during an appeal process.
Question 9: Reducing the Risk of AI
Utilization of existing standards and best practices is essential in helping mitigate risks associated with harmful or illegal content or results in harm to the public. Furthermore, the Chamber has continued to outline the importance of the government engaging with all stakeholders to help address specific concerns should they arise.
Question 10: Protecting Civil Rights
Please see the answer to Question 9.
III. Conclusion
In conclusion, we appreciate the opportunity to provide high-level feedback on the request for information. The Chamber believes that more time is necessary for stakeholders to review and provide more substantive feedback, as the request will substantially impact the government’s ability to take advantage of AI tools. We encourage OMB to provide other opportunities to receive input from stakeholders regarding these issues. We are willing and ready to work with OMB and the Office of Federal Procurement Policy to ensure the government’s safe and secure use of AI.
Sincerely,
Michael Richards
Senior Director
Chamber Technology Engagement Center
U.S. Chamber of Commerce
[1] U.S. Chamber Technology Engagement Center, “Maximizing Cost Savings: Unleashing the Value of Federal IT Modernization,” (April 2024) available at https://www.uschamber.com/assets/documents/Unleashing-the-Value-of-Federal-IT-Modernization-Report-2024.pdf
[2] Id.
[3] U.S. Chamber Press Release (October 30, 2023) available at https://www.uschamber.com/technology/ai-executive-order-addresses-important-priorities-but-needs-more-work.
[4] 88 Fed. Reg. 75191 available at https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence.
[5] Office of Management and Budget, “A Guide to Best Practices for Performance-Based Service Contracting,” (October 1998) available at https://obamawhitehouse.archives.gov/omb/procurement_guide_pbsc.
