Regulation of AI Applications Comments

Mr. Russell T. Vought
Acting Director
Office of Management and Budget
1200 New Jersey Avenue SE
725 17th St. NW
Washington, DC 20503

Draft Memorandum to the Heads of Executive Departments and Agencies, “Guidance for Regulation of Artificial Intelligence Applications” (Docket No.: OMB- 2020-00261)

Dear Director Vought:

The U.S. Chamber of Commerce (“Chamber”)1 appreciates the opportunity to provide comments to the Office of Management and Budget (“OMB”) in response to its proceeding on the draft “Guidance for Regulation of Artificial Intelligence Applications.”2 The Chamber appreciates the Administration’s leadership in artificial intelligence (“AI”), particularly as embodied in the Executive Order 13859, “Maintaining American Leadership in Artificial Intelligence,” and is pleased with the Administration’s continued pro-innovation approach in this proceeding.3

Introduction

This year represents a pivotal moment in American global competitiveness in emerging technologies—setting in motion a process by which federal agencies will consider their regulatory approaches towards AI applications. As America’s global competitors address the policy challenges associated with AI, U.S. leadership is essential at the international level, as well as needed to reduce regulatory uncertainty for businesses in the U.S.

The Chamber strongly endorses the approach outlined in OMB’s draft memorandum, which shares a similar perspective to the Chamber’s “Principles on Artificial Intelligence.”4 Specifically, the Chamber supports a comprehensive approach that addresses federal, state, and local regulatory and non-regulatory activity, the ten principles for the stewardship of AI
applications, emphasis on voluntary consensus standards, and prioritization of U.S. leadership internationally. In addition, the Chamber believes the most important task is to get AI regulation right, which will lay the groundwork for the full innovative potential that AI will bring to consumers and the public at large. AI is rapidly evolving, so it is crucial that any regulatory approach taken by federal agencies first considers whether any AI-specific action is needed at all, or whether any non-regulatory measures would suffice, before moving forward with any regulatory measures through the rulemaking process.

Moving forward, the Chamber recommends that OMB consider the following recommendations to strengthen the draft memorandum, ensure a consistent approach across all federal entities, and provide for timely implementation of agency plans in light of the global AI policy landscape.

Scope of the Guidance

The draft memorandum focuses on the federal government’s role in pursuing regulatory and non-regulatory approaches to AI applications developed outside of the Federal government. However, this memorandum applies only to federal executive departments and agencies and not to federal independent regulatory agencies. Many independent agencies including the Federal Communications Commission, Consumer Financial Protection Bureau, and the Federal Trade Commission, will likely engage in substantial regulatory and non-regulatory activities related to AI in their respective areas of oversight. Consequently, the Chamber strongly encourages the Administration to promote the final memorandum to independent agencies, particularly agencies possessing regulatory authority.

In addition, the Chamber believes that independent agencies should proactively adopt the principles and approach taken in the final memorandum to guide their regulatory activities. This is critical to providing a clear and consistent approach to addressing the regulatory challenges of AI across the federal government. Ultimately, independent agency adoption of the guidance will provide further certainty for the business community and unlock innovation in AI across applications regardless of the regulator.

Encourage Innovation and Growth in AI

An effective government approach to AI removes barriers to innovation; provides predictable and sustainable operating environments for business; protects public safety and security; and builds public trust in the technology. One of the fundamental challenges facing both the legislative and executive branches is to ensure that public policies keep pace with innovation. The development and implementation of national standards and best practices, notably voluntary standards, and where appropriate, targeted national regulatory standards to serve specific public policy goals are priorities for the Chamber and the business community.

The Chamber strongly agrees with the approach taken in this section but recommends several improvements to further strengthen this section and make necessary clarifications:

1. The final memorandum should explicitly encourage partnership and collaboration between the business community and federal agencies on all regulatory and nonregulatory AI activities.
2. The final memorandum should provide greater direction to agencies to ensure that the guidance does not encourage overregulation. Also, agencies should be mindful of existing laws and regulations as well as articulate a clear public policy need before considering any additional regulatory actions.
3. There should be additional clarification as to the definition of “essential” to provide for a consistent approach when agencies develop their agency plans under the final memorandum. That definition should also be consistent with the principles adopted in the memorandum.
4. The draft memorandum correctly instructs agencies to consider the effect of federal regulations on state and local governments and provides that “agencies may use their authority to address inconsistent, burdensome, and duplicative State laws that prevent the
   emergence of a national market.” As state and local governments consider policies and enact laws that attempt to regulate the development and use of specific AI technologies,
   there is a greater risk of a patchwork approach across different jurisdictions. This builds an environment that discourages innovation, creates compliance complexities, and fuels consumer uncertainty and distrust. The only instance in which preemption may be inappropriate is in some regulated industries where there already is a robust regulatory scheme in place at the state level.

Principles for Stewardship of AI Applications

Public Trust in AI

Public trust in the development and deployment of AI is essential to facilitate innovation and fully unlock its benefits. The Chamber is pleased that the draft memorandum prioritizes public trust in AI and believes that public trust should be the cornerstone to any regulatory considerations pertaining to AI. To facilitate public trust, regulators should consider the
Organisation for Economic Co-operation and Development’s (“OECD”) approach to trustworthy AI, which includes values such as transparency, explainability, fairness, and accountability.5

In addition, regulators should consider adopting future AI risk management frameworks developed by entities such as the National Institute of Standards and Technology (“NIST”) that facilitate the responsible use of AI technologies. These principles should be developed through an open, voluntary, and consensus-based process. Any such frameworks should be risk-based and should include formal threat modeling concepts to identify potential risks in AI applications.

Finally, the Chamber emphasizes that public trust-building efforts be conducted in partnership with government, industry, and any other relevant stakeholders. A public-private partnership model will facilitate collaboration between all relevant stakeholders and allow for sharing of best practices.

Public Participation

The Chamber agrees that public participation is important to strengthening federal agency accountability and improving regulatory outcomes. In the U.S., the private sector plays a leadership role in developing AI applications and is central to AI innovation. Consequently, the Chamber encourages robust and ongoing agency collaboration with the private sector on regulatory and non-regulatory actions at all stages of the policymaking process. In addition, the private sector seeks to further strengthen its partnership with the federal government to educate on the perception, opportunities, and impacts of AI applications.

Scientific Integrity and Information Quality

The Chamber has long advocated for scientific integrity, robust technical analysis, and information quality throughout the rulemaking process to ensure that regulations are cost effective, flexible, and sound. The draft memorandum adheres to that approach and properly recognizes the critical roles of scientific integrity and information quality in the regulatory process. To strengthen this principle, the final memorandum should seek to define the requirement that data used to train AI systems must be of a sufficient quality. In establishing any definitions, the final memorandum should recognize that agencies handle a wide variety of information and encourage a flexible approach to clarifying how data should be considered of sufficient quality.

Risk Assessment and Management

The Chamber supports the use of a risk-based approach to AI regulation. This risk-based approach should account for the varying magnitude and nature of consequences when considering risk mitigation, recognize tradeoffs (including a clear consideration of benefits as well as risks in designing risk management approaches), and ensure that any regulatory approaches being considered or proposed are linked to specific public policies in the national interest. The Chamber recommends several improvements that should be included in the final memorandum to strengthen the risk-based approach outlined in the draft memorandum.

First, OMB should in the future consider providing guidelines further defining risk when agencies are considering the level of risk posed by AI applications. Such guidelines would help agencies better understand risk, prevent wide disparities in how risks are evaluated at different agencies, and ultimately provide certainty for the business community. Those guidelines should be specific and narrowly tailored and include a governance process to assess and manage risk. A risk-based approach would also entail considering factors such as safety and human life, impact on critical infrastructure, financial market stability, and the capability to cause concrete harm to individuals. It would further ensure a consistent evaluation of risk for AI-based applications and human-based activities, as well as a holistic approach to risk that considers risk substitution effects versus the baseline (non-AI) scenario. Finally, agencies should contemplate the various roles organizations play in the AI lifecycle, such as the designer and developer or the operator, when assessing and managing risk.

Second, it is important to note that eliminating or removing bias from models may not be technically possible in all circumstances. The Chamber supports an approach that considers mitigation as a tool to address and reduce bias. In order to mitigate bias-driven outcomes, the final memorandum should direct agencies to consider appropriate safeguards and postdeployment monitoring where appropriate.

Benefits and Costs

The Chamber has a long-standing position of supporting cost-benefit analysis during the regulatory process and endorses agencies’ adherence to Executive Order 12866, “Regulatory Planning and Review.”6 Cost-benefit analyses are particularly important in the AI context given rapid innovation in AI technologies and routine development of novel applications. As noted in the draft memorandum, it is critical that AI systems are compared to systems currently in place when such an analysis is performed, as well as the accounting for the costs associated with not implementing an AI system. By comparing AI applications to existing systems, including human-based systems, a baseline can be established to assess whether an AI system will perform more or less effectively versus the baseline.

Flexibility

AI is a changing field in which new, transformative technologies are regularly emerging. The Chamber supports the use of a flexible, non-prescriptive, and performance-based approach that can adapt to rapid changes and updates to AI applications. Furthermore, the Chamber agrees that an international lens must be applied in establishing a flexible regulatory approach to provide consistency across borders.

While the Chamber supports the draft memorandum’s proposed approach to carefully consider the use of non-regulatory approaches, the Chamber recognizes that there may be some circumstances in which an agency determines that regulation is necessary. In those circumstances, an agency should take a technology-neutral approach, on the basis that technology continuously evolves. Agencies should collaborate with industry and other
stakeholders to focus on the performance-based governance of technology in order to address the issues that arise from specific uses and applications of AI.

Fairness and Non-Discrimination

The Chamber supports the focus on fairness and non-discrimination in the draft memorandum. Fairness and non-discrimination principles are essential for establishing public trust in AI. In assessing the impact an AI application has on fairness and non-discrimination, the Chamber recommends that the final memorandum consider several additional factors.

First, the principles of fairness and non-discrimination are not unique to AI, so it is important that agencies consider how the principles are applied in existing human-based contexts, for example in decisions regarding housing, employment, distribution of public benefits, adjudications, and extension of credit. New approaches should not supersede established definitions and practices, including state laws, but instead should focus on identifying harms that could potentially arise and be empirically linked to discrimination.

Second, the Chamber cautions against any agency action, regulatory or otherwise, that extends a disparate impact standard to assess fairness and non-discrimination. Finally, the Chamber encourages consideration in the final memorandum and by agencies for the potential of a well-designed, transparent, and accountable AI-based application to actually mitigate and reduce bias and discrimination versus existing human-centric systems.

Disclosure and Transparency

The Chamber agrees that transparency is a key principle to cultivate public trust in AI applications. Transparency should include two important factors: understandability and interpretability. Understandability enables a non-technical individual to gain insight into how an algorithm operates, and what factors contribute to a given decision. It is critical that nontechnical individuals understand at a general level how their data is being used and how their actions can generate new predictions.

Interpretability allows a technical expert, such as an AI/machine learning expert or agency official engaged in oversight, model governance or an audit, to understand at some level why an algorithm made a given decision. Interpretability would allow federal agencies to better ascertain how their models will act in real world conditions. Interpretability tends to be the focus of what organizations such as DARPA deem “explainable AI,” which is defined as the ability of machines to: 1) explain their rationale; 2) characterize the strengths and weaknesses of their decision-making process; and 3) convey a sense of how they will operate in the future. Research and development on interpretability remain a point of emphasis for the Chamber and many of its members. While much progress is to be made, the private sector is working to achieve broadly accepted and interoperable technical solutions to address this objective. Interpretability can be difficult for the most advanced AI algorithms, where attention to inputs, training and outputs may offer better mechanisms for accountability.

Finally, while the Chamber strongly supports transparency and disclosure, the Administration and agencies should recognize that there should be appropriate measures to protect trade secrets, confidential business information, and other forms of intellectual property. In particular, agencies should not mandate the transfer of, or provide access to AI-related intellectual property including source code, algorithms, and data sets.

Safety and Security

The Chamber supports the draft memorandum’s statement that “agencies should…encourage the consideration of safety and security issues throughout the AI design, development, deployment, and operation process.”7 A best practice in deploying a safe, secure, and fair AI system is the development of and execution of internal governance models and safety frameworks. Such internal governance models and safety frameworks should be proactively designed to ensure that AI systems are built with safety and security protections at the beginning of the product’s development process.

In place of regulation, the Chamber also supports the development of voluntary, consensus-based standards, wherever possible, to ensure the safety and security of AI systems. Specifically, the Chamber applauds NIST’s efforts to establish “A Taxonomy and Terminology of Adversarial Machine Learning” (“ML”) as a baseline to inform future standards and best practices for assessing and managing the security of ML.8

Lastly, the final memorandum should clarify that organizations will likely not be able to guarantee systemic resilience in AI systems. It is impossible to fully guarantee the security of any system from a cyber-threat. Therefore, the final memorandum should replace the word “guarantee” in this section with “protecting” or “promoting” to ensure the workable and realistic implementation of furthering the safety and security of AI systems.9

Interagency Coordination

AI affects a wide variety of sectors and industries including financial services, education, transportation, defense, and healthcare. Consequently, many regulatory agencies will be asked to consider future regulatory actions affecting AI applications. One of the Chamber’s top priorities is to ensure robust interagency coordination and dialogue to prevent duplicative or conflicting approaches to AI regulatory activity to provide for a consistent and interoperable approach across the federal government. In particular, there is a significant risk that agencies may adopt different definitions and standards relating to AI if interagency coordination and Administration leadership is lacking. The Chamber recommends that the Administration make a concerted effort to build a platform within the federal government to facilitate collaboration and information sharing between federal agencies to ensure the adoption of best practices.

Non-Regulatory Approaches to AI

Non-regulatory approaches often can achieve the same policy objectives and offer the same level of protections as regulatory approaches, but without many of the burdens of regulation. Should federal action be necessary to address any problems arising from an AI application, the Chamber urges agencies to first consider non-regulatory approaches. The
Chamber supports all three of the non-regulatory approaches contemplated in the draft memorandum: (1) sector-specific policy guidance or frameworks, (2) pilot programs and experiments, and (3) voluntary consensus standards. Generally, the Chamber encourages agencies to consider the following guiding principles to produce and implement non-regulatory approaches: interagency collaboration, voluntary requirements, agency leadership, public and private participation, and information sharing with relevant stakeholders.

Sector-Specific Policy Guidance or Frameworks

The Chamber strongly supports sector-specific guidance, in particular in areas where there is already a robust regulatory structure in place that requires additional flexibility. A single approach should generally be avoided as use cases differ across sectors and industries. The Department of Transportation’s iterations of automated vehicle guidance are emblematic of the approach other agencies should take when developing policy guidance.10

Pilot Programs and Experiments

Pilot programs and experiments can be effective tools to facilitate the introduction of novel AI applications without placing excessive regulatory burdens or undue liability on the participating parties. Furthermore, agencies should ensure that there is a path for pilot programs and experiments to influence the policymaking process. If multiple pilot programs and experiments overlap, either between agencies or within agencies, there must be sufficient coordination and information sharing between those programs to prevent confusion and improve the policymaking process.

In addition, the Chamber strongly supports the Administration’s proposal to allow agencies the flexibility to provide safe harbors for specific AI applications by granting waivers and exemptions from regulations and pilot programs. By providing these regulatory sandboxes, agencies can encourage innovation and growth in a controlled and responsible environment.

Voluntary Consensus Standards

The development of voluntary consensus standard-setting is a highly effective means of addressing the challenges and opportunities presented by emerging technologies such as AI, and the Chamber supports the preference given in the draft memorandum to voluntary consensus standards. The development of voluntary consensus standards can help create and safeguard trust at the heart of AI-driven systems and business models, and permit the flexibility for innovation, allowing standards to develop with the technology.

Also, the Chamber notes that multi-stakeholder initiatives have the greatest capacity to create industry harmonization and equal access on AI resources for all stakeholders, and may identify gaps in existing standards and certifications, which ecosystem players can then act upon. One-size fits-all solutions should be avoided as applications vary across sectors and industries.

Finally, the Chamber endorses the approach taken by this Administration in NIST’s “U.S. Leadership in AI: A Plan for Federal Engagement in Developing Technical Standards and Related Tools.”11 Moving forward, the Chamber encourages the further implementation of NIST’s Plan and urges the Administration to ensure that NIST has the appropriate resources to facilitate the Plan’s timely implementation.

Reducing Barriers to the Deployment and Use of AI

The Chamber is pleased that the draft memorandum contemplates and promotes reducing regulatory barriers to the development and use of AI. As noted earlier, the introduction of AI applications will likely require federal agencies to modify their regulatory approaches to facilitate the adoption of AI technologies. Proactive agency leadership will be essential to ensure continued U.S. innovation and global leadership in AI.

Access to Federal Data and Models for AI R&D

The Chamber recognizes that providing access to federal data sets to improve upon and develop new AI applications is one of the most important steps the federal government can take to facilitate AI innovation. Open government data is essential to maintaining U.S. global leadership in AI given that many of our global competitors collect and utilize data in a manner inconsistent with American values. To encourage open data, the final memorandum should direct federal agencies to take the following steps:

1. Federal agencies should continue to support the timely implementation of the OPEN Government Data Act.12
2. Federal agencies should increase data quality and availability by ensuring that any datasets are accessible in a structured, commonly used, and in a machine-readable format.
3. Federal agencies should prioritize activities to open government data in their annual budget requests.
4. Federal agencies should consider additional measures to further open government data and regularly engage in dialogue with the private sector to exchange best practices and approaches.

International Regulatory Cooperation

International leadership is essential for the United States to remain globally competitive in AI systems and applications. We applaud the Administration for achieving high-level agreement on the AI recommendations at the OECD, which may serve a basis for global consensus on responsible AI practices. We note that a number of economies, including Brazil, Canada, and the European Union, are currently developing their own regulatory approaches to the technology. With this in mind, the Chamber strongly supports the Administration and federal agencies in engaging with their international counterparts to discuss various approaches to AI regulation and advance interoperable methods where appropriate. Proactive cooperation with foreign governments is necessary to mitigate regulatory divergences and secure market access for American AI systems and applications. The Chamber likewise encourages the Administration to advance state-of-the-art digital trade disciplines to counter existing and future barriers to exports of American AI-related goods and services. Finally, the Administration should make it a priority to support industry leadership in the development of international standards for AI-related activities.

Deregulatory Activity

The introduction of novel AI applications will require the Administration and federal agencies to contemplate deregulatory activities to further reduce barriers to the use and deployment of AI applications. The final memorandum should require federal agencies to consider any relevant deregulatory activities in a manner consistent with the final memorandum
and with Executive Order 13771, “Reducing Regulation and Controlling Regulatory Costs.”13

Agency Plans to Achieve Consistency with this Memorandum

Federal agency implementation of the final memorandum will be essential to fulfilling the pro-innovation approach taken in this guidance. The Chamber recommends that the final memorandum consider and include several areas to strengthen federal agency implementation and development of agency plans. First, the Chamber supports requiring agencies to collaborate with industry stakeholders on the development of agency plans through regular workshops, listening sessions, and an opportunity to comment on draft agency plans. Second, America’s global competitors are moving rapidly to position themselves as global leaders on AI regulation and standards. The U.S. cannot afford to be left behind in the global race to frame the regulatory approach to AI and prevent industry from having the flexibility and certainty to innovate. Consequently, the Chamber strongly encourages timely agency development and submission of all agency plans by the end of 2020. Administration leadership is necessary to ensure the timely development and submission of agency plans. Finally, both OMB and agencies should ensure that the agency plans developed in accordance with this memorandum does not, intentionally or unintentionally, encourage overregulation by agencies.

Appendix A: Technical Guidance of Rulemaking

Regulatory Impact Analysis

The draft memorandum includes guidance for federal agencies on how to conduct cost-benefit analyses of potential regulatory approaches and states, “[w]hen quantification of a particular benefit or cost is not possible, it should be described qualitatively.”14 The Chamber believes that all cost-benefit analyses of AI applications should include consideration of both quantitative and qualitative metrics. Qualitative metrics provide important context and should not be used only when quantitative measurements are unavailable. Indeed, qualitative analysis is often needed to understand quantitative measurements. For example, there are multiple ways to measure quantitatively the “fairness” of a particular AI application. The best way to determine which of those quantitative measurements provides the best measurement is through context and qualitative analysis. Agencies should consider metrics developed by standards bodies and NIST
to strengthen cost-benefit analyses.

Assessing Risk

The draft memorandum states, “[a]gencies should also consider that an AI application could be deployed in a manner that yields anticompetitive effects that favor incumbents at the expense of new market entrants, competitors, or up-stream or down-stream business partners.”15 Given the early stages of development and implementation of AI, the Chamber expresses strong concern with this language. Agencies should not assume that this technology will necessarily yield anti-competitive effects. Indeed, AI could be deployed in a manner that yields procompetitive effects by increasing efficiencies and introducing new business models and approaches.

Also, it is important to recognize that agencies have a key role to play in driving innovation from businesses. The most critical levers to help small enterprises take advantage of AI include access to data, technology, and people. The federal government can lead by example by sharing public-sector data-sets through the creation of public-data platforms that all businesses, including small businesses, can freely access. Additionally, the federal government can encourage the private sector and scientific and research institutions to share data and collaborate, which can help support the development of vibrant AI ecosystems. Finally, the proper venue for any competition considerations are the Department of Justice and the Federal Trade Commission, not all federal agencies undertaking regulatory activity.

Conclusion

The Chamber endorses the pro-innovation posture of this draft memorandum and commends the Administration for its leadership on AI. Accordingly, the Chamber is appreciative of OMB’s consideration of our recommendations to strengthen the draft memorandum and ensure the final memorandum’s timely and effective implementation. The Chamber looks forward to collaborating with the Administration and its agencies on the implementation of the final memorandum and its promotion to domestic and international stakeholders.

Sincerely

Neil L. Bradley
Executive Vice President & Chief Policy Officer
U.S. Chamber of Commerce

---

1. The U.S. Chamber of Commerce is the world’s largest business federation representing the interests of more than three million businesses of all sizes, sectors, and regions.

2. Draft Memorandum to the Heads of Executive Departments and Agencies, “Guidance for Regulation of Artificial Intelligence Applications,” 85 Fed. Reg. 1825 (Jan. 13, 2020) (“draft memorandum”).

3. Exec. Order No. 13,859, Maintaining American Leadership in Artificial Intelligence, 84 Fed. Reg. 3967 (Feb. 11, 2019), available at https://www.whitehouse.gov/presidential-actions/executive-order-maintainingamericanleadership-artificial-intelligence/.

4. See U.S. Chamber of Commerce Principles on Artificial Intelligence (March 2020) available at
https://www.uschamber.com/sites/default/files/chamber_ai_principles_-_general.pdf.

5. See OECD Council Recommendation on Artificial Intelligence (March 2020) available at https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449.

6. Exec. Order No, 12,866, “Regulatory Planning and Review,” 58 Fed. Reg. 51,735 (Sept. 30, 1993), available at https://www.reginfo.gov/public/jsp/Utilities/EO_12866.pdf.

7. See Draft Memorandum at 6.

8. Nat’l Inst, of Standards & Tech., U.S. Dep’t of Commerce, A Taxonomy and Terminology of Adversarial Machine Learning (October 30, 2019), available at https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8269-draft.pdf.

9. See Draft Memorandum at 6.

10. See U.S. Department of Transportation, Ensuring American Leadership in Automated Vehicle Technologies: Automated Vehicles 4.0 (January 8, 2020), available at https://www.transportation.gov/sites/dot.gov/files/2020-02/EnsuringAmericanLeadershipAVTech4.pdf.

11. Nat’l Inst. of Standards & Tech., U.S. Dep’t of Commerce, U.S. Leadership in AI: A Plan for Federal Engagement in Developing Technical Standards and Related Tools (August 9, 2019), available at https://www.nist.gov/topics/artificial-intelligence/plan-federal-engagement-developing-ai-technical-standardsandrelated.

12. 8 The Open, Public, Electronic, and Necessary (OPEN) Government Data Act, Pub. L. No. 115-435, 132 Stat. 5534 (2019) (Title II of the Foundations for Evidence-Based Policymaking Act of 2018).

13. Exec. Order No. 13,771, Reducing Regulation and Controlling Regulatory Costs, 82 Fed. Reg. 9339 (Jan. 30, 2017), available at https://www.govinfo.gov/content/pkg/FR-2017-02-03/pdf/2017-02451.pdf.

14. See Draft Memorandum at 11.

15. Ibid at 12.