Comments of the R Street Institute to the National Telecommunications and Information Administration (NTIA) on AI Accountability Policy

Re: NTIA Docket No. 230407-0093 – Request for Comment on AI Accountability Policy

Thank you for providing the R Street Institute (R Street) with the opportunity to comment in response to the National Telecommunications and Information Administrations (NTIA) request for comment on AI Accountability Policy (Request for Comment).[1] My name is Adam Thierer, and I am a senior fellow with R Street’s Technology and Innovation Policy team. R Street is a nonprofit, nonpartisan public policy research organization. I also recently served as a Commissioner on the U.S. Chamber of Commerce “Commission on Artificial Intelligence Competitiveness, Inclusion, and Innovation,” which released a major report on policy issues surrounding artificial intelligence (AI), machine learning and algorithmic systems.[2]

R Street has published several studies relevant to this proceeding, including a new report on, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence.”[3] Pages 27 to 33 of that report discussed strategies to “professionalize” AI ethics and explored how algorithmic audits and impact assessments might play a role in that process. This R Street report has been submitted for the record.  

Before posing questions about the specific issues itemized by the first 29 questions of the Request for Comment, it may be helpful to prioritize the final questions (Questions 30-34) that ask “[w]hat role should government policy have, if any, in the AI accountability ecosystem?”^[4] 

Last October, the White House released its “Blueprint for an AI Bill of Rights” (AI Blueprint) and an accompanying list of “Key Actions to Advance Tech Accountability and Protect the Rights of the American Public” (Key Actions).^[5] Taken together, the AI Blueprint, Key Actions, and this Request for Comment will play an important role in shaping the innovation culture around computational technologies. Innovation culture refers to the, “attitudes towards innovation, technology, exchange of knowledge, entrepreneurial activities, business, uncertainty,” and related activities that determine how a nation treats any particular technology or business sector.^[6]

We recommend that the NTIA and the Administration consider the following priorities when formulating AI policy in general or algorithmic audits or impact assessments in particular:

Establish a positive vision and ensure that policy steps do not undermine the potential benefits of algorithmic systems 

• The administration’s AI Blueprint, Key Actions, and this Request for Comment tend to stress worst-case scenarios that might flow from the increased use of AI, machine learning and robotics. Much less is said about the many ways that these systems and technologies will boost our living standards, improve our health, extend our lives, expand transportation options, avoid accidents, improve community safety, enhance educational opportunities, help us access superior financial services and much more.
• The danger exists that policy for AI and computational systems could be formulated in such a way that innovations are essentially treated as “guilty until proven innocent” and required to go through a convoluted and costly certification process before being allowed on the market.^[7] This could hold back important life-enriching and even life-saving algorithmic innovations. Concerns about AI risk should be addressed, but there is also a compelling public interest in ensuring that algorithmic innovations are developed and made widely available to society.^[8]

Identify barriers to algorithmic innovation, investment and competition

• The agency should consider how existing legal or regulatory barriers might be undermining the ability of U.S. developers to rollout important algorithmic services, both domestically and internationally. The agency should also consider how existing or new rules and regulations might constraint the ability of workers to prepare for the future due to barriers to flexible work and labor mobility.^[9] The Request for Comment mentions innovation only once in Question 7 when the agency asks, “Are there accountability mechanisms that unduly impact AI innovation and the competitiveness of U.S. developers?”^[10] That question should be front and center, not just with reference to audits or impact assessments, but also with all current and proposed policies.
• The administration should consider the complexity and costs that new AI policies might have for smaller enterprises and open-source systems in particular. The NTIA should specify the ways in which a compliance-laden and paperwork-intensive regulatory regime might undermine the vitality of those firms or systems. As an example of the potential significance of such costs, consider that the European Commission itself estimates that the mandates required by the forthcoming Artificial Intelligence Act will cost roughly $193,000-$330,000 upfront plus $71,400 in yearly maintenance costs.^[11] Compliance costs of that magnitude will burden small- and medium-sized enterprises (and open-source-based AL systems and applications) and limit these important sources of AI innovation and investment.

Appreciate the global competitiveness and national security ramifications of AI policy

• As the administration considers governance guidelines for algorithmic technologies, it is important to keep in mind that these policy developments are happening against the backdrop of intense global competition for geopolitical competitive advantage.^[12] The economic and strategic interests of the United States would not be well served by a compliance-intensive regulatory regime that could hinder important algorithmic innovations.^[13] It is essential that the United States be a leader in AI and strengthen our technology base to ensure our continued global competitive standing and geopolitical security.^[14]

Acknowledge statutory and constitutional constraints 

• With this proceeding, the NTIA has opened a potentially far-reaching inquiry without statutory authorization or other direction from Congress on these specific matters. The danger of administrative overreach exists whenever agencies seek to fit new technologies into old policy frameworks.^[15] Accordingly, prudence and adherence to constitutional values will be essential as the NTIA moves to investigate AI policy, as this has not traditionally been an area over which the NTIA has authority.
• Similarly, clear congressional direction and authorization would be required for any agency to mandate AI audits or impact assessments (or similar algorithmic mandates), and the NTIA and other agencies (like the Federal Trade Commission [FTC]) must be careful to not view this proceeding as the foundation for an open-ended regulatory regime. Because Congress has not yet finalized federal privacy legislation or authorized the FTC to act unilaterally and promulgate rules on issues like commercial surveillance, this NTIA Request for Comment cannot serve as the basis for justifying such actions.^[16] There are many important speech-related issues in play here. As discussed in the accompanying R Street study, if audits or impact assessments were imposed through regulations, it could give rise to the problem of political interference in speech platforms powered by algorithms. At some level, code represents speech, meaning algorithms are speech as well.
• Question 27 alludes to the important issue of the interplay between AI and intellectual property. Efforts by government to mandate audits or impact assessments could come into conflict with trade secrets. More generally, there are a host of broader IP issues that are beyond the scope of this proceeding, including the nature of IP ownership for AI-generated content, the importance of fair-use principles for large language models, and the application of certain existing copyright exceptions to AI technology or applications.

Identify how existing state capacity already addresses many concerns

• Many ex post remedies already exist and could address some of the concerns raised in this proceeding and the Biden administration’s AI Blueprint. Before pursuing audits, impact assessments or other new policies, the NTIA should first evaluate the extensive existing state capacity that can address many of the concerns raised in these and other documents.
• AI governance is not developing in a vacuum. The U.S. federal government has over 2.1 million civilian employees working at 15 Cabinet agencies, 50 independent federal commissions and over 430 federal departments, and many of them have already started considering how they might address AI and robotics.^[17] Many agencies have been active on this front, including the FTC, the Food and Drug Administration, the National Highway Traffic Safety Administration, the Federal Aviation Administration, the Equal Employment Opportunity Commission and the Consumer Product Safety Commission.^[18] The courts and common law system are also prepared to address novel AI problems as cases develop.^[19] 

Identify the trade-offs at work with algorithmic transparency and “explainability”

• Moving on to more specific matters, Questions 20 to 23 and other portions of the Request for Comment highlight an interest in ensuring greater transparency or “explainability” for algorithmic systems. These concepts defy easy definition, however, as it is impossible to make many algorithmic systems perfectly explainable. “A list of a billion operations is not an explanation that a human can understand,” notes a leading AI expert.^[20] “Even the humans who train deep networks generally cannot look under the hood and provide explanations for the decision their networks make.”^[21]
• As Question 3 suggests, there are trade-offs at work when defining these concepts. If policy is based on making AI perfectly transparent or explainable before anything launches, then innovation will suffer because of bureaucratic delays and costly compliance burdens. Mandated transparency and explainability could also have unintended consequences related to corporate confidentiality, intellectual property rights, system security and user privacy.^[22]
• AI governance should be risk-based and should focus on system outcomes instead of system inputs or design. In other words, policy should concern itself more with actual algorithmic performance in specific contexts, not the underlying processes of algorithmic development in what will likely be a failed attempt to make a system perfectly explainable to the public.^[23]

Encourage, but do not force, AI audits and algorithmic impact assessments

• As discussed in the accompanying R Street study audits and impact assessments can play a useful role in building trust in algorithmic systems and can help ensure that organizations live up to the promises they make the public about them. But the Administration should first consider the broader issues and trade-offs associated with the limitations of these mechanisms—especially if conceptualized of as formal regulatory processes.
• We must appreciate the enormous measurement challenges that would be involved in attempting to employ algorithmic audits and impacts assessments. There are no easy answers to most of the first 29 questions raised in this proceeding because most of the values or objectives being considered cannot easily be measured or tracked or have varying definitions. In a highly pluralistic society, “aligning” AI with these values assumes that there is widespread agreement about what each term means for purposes of algorithmic regulation.^[24] Auditing an algorithm is nothing like auditing an accounting ledger, where either the numbers do or do not add up. Even proponents of these approaches have admitted that, “basic components and commitments of this still nascent field require working through before audits can reliably address algorithmic harms.”^[25] 
• At this early stage, government should be design-agnostic when it comes to audits or assessments and should instead encourage ongoing dialogue among developers and stakeholders about what governance mechanisms work best in different contexts. Luckily, as recent R Street research has documented, many organizations are already working actively together to professionalize the process of AI ethics through sophisticated best-practice frameworks, as well as through voluntary AI auditing and algorithmic impact assessment efforts.^[26] 

We recommend that the NTIA:

1. Build on the important steps that the National Institute of Standards and Technology (NIST) has taken in its “Artificial Intelligence Risk Management Framework” (AI RMF), which is “designed to address new risks as they emerge.”^[27] NIST stressed that the AI RMF is meant to be “risk-based, resource-efficient, pro-innovation, and voluntary.”^[28] The AI RMF looks to be “outcome-focused and non-prescriptive … rather than prescribe one-size-fits-all requirements.”^[29] The NTIA should work with NIST to build on this consensus-driven approach and resulting voluntary guidance document, which was meant “to offer a resource to the organizations designing, developing, deploying, or using AI systems to help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems.”^[30] 
2. Work with NIST to build on the NTIA’s important past efforts to convene different technology developers and stakeholders and help build consensus around voluntary best practices tailored to unique contexts and concerns. The NTIA and other agencies have brought together diverse stakeholders in the past to find solutions to complicated technology problems as they developed.^[31] The same approach can be used to help address algorithmic risk with a standing effort to bring parties together regularly to consider practical response strategies as necessary.
3. The NTIA, NIST and other federal agencies should also work together to facilitate digital literacy efforts and technology awareness-building efforts, which can help lessen public fears about emerging algorithmic and robotic technologies.^[32] Developers have a powerful incentive to build widespread trust in their systems to ensure they get adopted, but they often fail to coordinate with other players to educate the public about the benefits and risks of digital systems. Government can help coordinate and promote more widespread public understanding of these systems and their proper use.

Importantly, impact assessments and audits are just two of many different mechanisms that can help govern algorithmic systems and generate more trust and accountability. Competition and innovation among existing and future market players can also act as a check on developer missteps and provide the public with different platforms and applications to suit specific needs and values. Policymakers should not presume that there is a one-size-fits all approach to algorithmic governance. Moreover, diverse new options can only emerge in an ecosystem free of artificial constraints on entrepreneurial activities. Burdensome mandates would be particular costly to small- and mid-sized firms looking to break into the market and offer alternatives that could provide differing levels of privacy and security protections, for example.

Finally, to reiterate, AI policymaking must not be fear-based or rooted in worst-case thinking. It must, by necessity, be risk based and highly context specific. Generally speaking, the touchstones of wise emerging technology policy are humility, agility, and adaptability, with a strong focus on ongoing communication and collaboration to address fast-moving developments.^[33] 

America’s crucial advantage over other countries on the digital technology front has been our uniquely agile and adaptive approach to technological governance.^[34] The U.S. policy approach has been rooted in a general freedom to innovate that is accompanied by a diversity of ex-post policy solutions to address problems that develop.^[35] This more iterative, bottom-up governance approach not only gives the public more options, but it also provides our nation with a safer and more secure technological base.^[36] It has produced the most successful technological revolution of the past half century—with enormous benefits for the economy and consumers.^[37] The Administration should look to build on that model and foster the development of trustworthy algorithmic innovations that benefit the public and keep the United States at the forefront of the next great technological revolution.

Respectfully submitted,

____________________________

Adam Thierer

Senior Fellow

R Street Institute

1212 New York Ave. NW,

Suite 900

Washington, D.C. 20005

athierer@rstreet.org

Footnotes

[1] “AI Accountability Policy Request for Comment,” National Telecommunications and Information Administration, Docket No. 230407-0093, RIN 0660-XC057, April 11, 2023. https://ntia.gov/issues/artificial-intelligence/request-for-comments.

[2]Commission on Artificial Intelligence Competitiveness, Inclusion, and Innovation: Report and Recommendations, U.S. Chamber of Commerce, March 9, 2023. https://www.uschamber.com/technology/artificial-intelligence-commission-report

[3] Adam Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” R Street Institute Policy Study No. 283 (April 2023). https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[4] “AI Accountability Policy Request for Comment,” p. 29. https://ntia.gov/issues/artificial-intelligence/request-for-comments.

[5] “Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People,” The White House, October 2022. https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf; “Fact Sheet: Biden-⁠Harris Administration Announces Key Actions to Advance Tech Accountability and Protect the Rights of the American Public,” The White House, Oct. 4, 2022. https://www.whitehouse.gov/ostp/news-updates/2022/10/04/fact-sheet-biden-harris-administration-announces-key-actions-to-advance-tech-accountability-and-protect-the-rights-of-the-american-public.

[6] Maike Didero et al., “Differences in Innovation Culture Across Europe: A Discussion Paper,” TransForm, February 2008, p. 3. https://www.yumpu.com/en/document/read/6683782/differences-in-innovation-culture-across-europe-transform.

[7] Ibid.

[8] Thierer, “Getting AI Innovation Culture Right,” p. 10. https://www.rstreet.org/research/getting-ai-innovation-culture-right.

[9] Adam Thierer, “Can We Predict the Jobs and Skills Needed for the AI Era?,” R Street Institute Policy Study No. 278 (March 2023), pp. 13-15. https://www.rstreet.org/research/can-we-predict-the-jobs-and-skills-needed-for-the-ai-era.  

[10] “AI Accountability Policy Request for Comment,” p. 22. https://ntia.gov/issues/artificial-intelligence/request-for-comments.

[11] European Commission, “Study supporting the impact assessment of the AI regulation,” European Commission, April 21, 2021, p. 12. https://digital-strategy.ec.europa.eu/en/library/study-supporting-impact-assessment-ai-regulation.

[12] Eric Schmidt, “Innovation Power: Why Technology Will Define the Future of Geopolitics,” Foreign Affairs (March/April 2023). https://www.foreignaffairs.com/united-states/eric-schmidt-innovation-power-technology-geopolitics.

[13] Adam Thierer, “What OpenAI’s Sam Altman Should Say at the Senate AI Hearing,” R Street Institute, May 15, 2023. https://www.rstreet.org/commentary/what-openais-sam-altman-should-say-at-the-senate-ai-hearing.

[14] Adam Thierer, “Statement for the Record on ‘Artificial Intelligence: Risks and Opportunities,’” U.S. Senate Homeland Security and Governmental Affairs Committee, March 8, 2023. https://www.rstreet.org/outreach/testimony-on-artificial-intelligence-risks-and-opportunities.

[15] Thierer, “Getting AI Innovation Culture Right,” p. 6.

[16] NTIA, “Comments of the National Telecommunications and Information Administration Regarding Commercial Surveillance ANPR R11004,” National Telecommunications and Information Administration, Docket FTC-2022-0053, (2022). https://ntia.gov/sites/default/files/publications/ftc_commercial_surveillance_anpr_ntia_comment_final.pdf.

[17] Adam Thierer, “A balanced AI governance vision for America,” The Hill, April 16, 2023. https://thehill.com/opinion/congress-blog/3953916-a-balanced-ai-governance-vision-for-america.

[18] Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” pp. 33-36. https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[19] John Villasenor, “Products liability law as a way to address AI harms,” Brookings, Oct. 31, 2019. https://www.brookings.edu/research/products-liability-law-as-a-way-to-address-ai-harms.

[22] Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” pp. 23-25. https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[20] Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans (Farrar, Straus and Giroux, 2019), p. 108.

[21] Ibid.

[22] Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” pp. 23-25. https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[23]Daniel Castro, “Ten Principles for Regulation That Does Not Harm AI Innovation,” Information Technology & Innovation Foundation, Feb. 8, 2023, p. 5. https://itif.org/publications/2023/02/08/ten-principles-for-regulation-that-does-not-harm-ai-innovation.

[24] Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” p. 30. https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[25] Ellen P. Goodman and Julia Tréehu, “AI Audit-Washing and Accountability,” German Marshall Fund, November 2022, p. 25. https://www.gmfus.org/sites/default/files/2022-11/Goodman%20%26%20Trehu%20-%20Algorithmic%20Auditing%20-%20paper.pdf.

[26] Thierer, “Flexible, Pro-Innovation Governance Strategies for Artificial Intelligence,” pp. 13-23. https://www.rstreet.org/research/flexible-pro-innovation-governance-strategies-for-artificial-intelligence.

[27] National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0),” U.S. Department of Commerce, January 2023, p. 4. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf.

[28] Ibid., p. 42.

[29] Ibid.

[30] “NIST Risk Management Framework Aims to Improve Trustworthiness of Artificial Intelligence,” National Institute of Standards and Technology, Jan. 26, 2023, p. 2. https://www.nist.gov/news-events/news/2023/01/nist-risk-management-framework-aims-improve-trustworthiness-artificial. 

[31] Ryan Hagemann et al., “Soft Law for Hard Problems: The Governance of Emerging Technologies in an Uncertain Future,” Colorado Technology Law Journal 17 (Feb. 5, 2018). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3118539.

[32] “Commission on Artificial Intelligence Competitiveness, Inclusion, and Innovation: Report and Recommendations,” U.S. Chamber of Commerce Technology Engagement Center, March 9, 2023, pp. 44-45.  https://www.uschamber.com/assets/documents/CTEC_AICommission2023_Report_v6.pdf.

[33] Adam Thierer, “Governing Emerging Technology in an Age of Policy Fragmentation and Disequilibrium,” American Enterprise Institute, April 2022. https://platforms.aei.org/can-the-knowledge-gap-between-regulators-and-innovators-be-narrowed.

[34] Adam Thierer, Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom, 2nd ed. (Mercatus Center at George Mason University, 2016).

[35] Ibid.

[36] Adam Thierer, “U.S. Chamber AI Commission Report Offers Constructive Path Forward,” R Street Institute, March 9, 2023. https://www.rstreet.org/commentary/u-s-chamber-ai-commission-report-offers-constructive-path-forward.

[37] Tina Highfill and Christopher Surfield, “New and Revised Statistics of the U.S. Digital Economy, 2005–2021,” Bureau of Economic Analysis, November 2022. https://www.bea.gov/system/files/2022-11/new-and-revised-statistics-of-the-us-digital-economy-2005-2021.pdf.