How to Design an AI Ethics Board

Introduction

It becomes increasingly clear that state-of-the-art artificial intelligence (AI) systems pose significant societal risks. AI systems used for drug discovery could be misused for the design of biochemical weapons. A failure of AI systems used to control nuclear power plants or other critical infrastructure could also have devastating consequences. Another concern is that, as models become larger and larger, certain dangerous capabilities might emerge at some point. Scholars and practitioners are increasingly worried about power-seeking behavior, situational awareness, and the ability to persuade people. Organizations that develop and deploy AI systems need to take measures to reduce these risks to an acceptable level. In this paper, we examine how AI companies could design an AI ethics board in a way that reduces risks from AI. By “ethics board”, we mean a collective body intended to promote an organization’s ethical behavior.

Some AI companies already have an AI ethics board. For example, Meta’s Oversight Board makes binding decisions about the content on Facebook and Instagram. Microsoft’s AI, Ethics and Effects in Engineering and Research (AETHER) Committee advises their leadership “on the challenges and opportunities presented by AI innovations”. DeepMind’s Institutional Review Committee (IRC) oversees their human rights policyand has already played a key role in the AlphaFold release. These examples show that AI ethics boards are of practical relevance.

But there have also been a number of failures. Google’s Advanced Technology External Advisory Council (ATEAC) faced significant resistance over the inclusion of disputable members. It was shut down only one week after its announcement. Axon’s AI and Policing Technologies Ethics Board was effectively discontinued in June 2022 after three years of operations. Nine out of eleven members resigned after Axon announced plans to develop taser-equipped drones to be used in schools without consulting the board first. (In late 2022, Axon announced their new ethics board: the Ethics & Equity Advisory Council [EEAC], which gives feedback on a limited number of products “through a racial equity and ethics lens”.) These cases show that designing an AI ethics board can be challenging. It also highlights the need for more research.

Although there has been some research on AI ethics boards, the topic remains understudied. The most important work for our purposes is a whitepaper by Accenture. They discuss key benefits of AI ethics boards and identify key design questions. However, their discussion lacks both breadth and depth. They discuss only a handful of design considerations and do not go into detail. They also do not focus on leading AI companies and risk reduction. Besides that, there is some literature on the purposeand practical challenges of AI ethics boards. There are also several case studies of existing boards, including Meta’s Oversight Boardand Microsoft’s AETHER Committee. And finally, there is some discussion of the role of AI ethics boards in academic research. Taken together, there seem to be at least two gaps in the literature. First, there is only limited work on the practical question of how to design an AI ethics board. Second, there is no discussion of how specific design considerations can help to reduce risks from AI. In light of these gaps, the paper seeks to answer two research questions (RQs):

• RQ1: What are the key design choices that AI companies have to make when setting up an AI ethics board?
• RQ2: How could different design choices affect the board’s ability to reduce risks from AI?

The paper has two areas of focus. First, it focuses on organizations that develop state-of-the-art AI systems. This includes medium-sized research labs (e.g. OpenAI, DeepMind, and Anthropic) and big tech companies (e.g. Microsoft and Google). We use the term “AI company” or “company” to refer to them. Although we do not mention other types of companies (e.g. hardware companies), we expect that they might also benefit from our analysis. Second, the paper focuses on the board’s ability to reduce risks (see RQ2). By “risk”, we mean the “combination of the probability of occurrence of harm and the severity of that harm”. (But note that there are other risk definitions). In terms of severity, we focus on adverse effects on large groups of people and society as a whole, especially threats to their lives and physical integrity. We are less interested in financial losses and risks to organizations themselves (e.g. litigation or reputation risks). In terms of likelihood, we also consider low-probability, high-impact risks, sometimes referred to as “black swans”. The two main sources of harm (“hazards”) we consider are accidentsand cases of misuse.

In the following, we consider five high-level design choices: What responsibilities should the board have (Section ch-2-responsibilities)? What should its legal structure be (Section ch-3-structure)? Who should sit on the board (Section ch-4-membership)? How should it make decisions and should its decisions be binding (Section ch-5-decision-making)? What resources does it need (Section ch-6-resources)? We break down each of these questions into more specific sub-questions, list options, and discuss how they could affect the board’s ability to reduce risks from AI. The paper concludes with a summary of the most important design considerations and suggestions for further research (Section ch-7-conclusion).

Responsibilities

What responsibilities should the board have? We use the term “responsibility” to refer to the board’s purpose (what it aims to achieve), its rights (what it can do), and duties (what it must do). The board’s responsibilities are typically specified in its charter or bylaws. In the following, we focus on responsibilities that could help to reduce risks from AI (see RQ2). The ethics board could advise the board of directors (Section ch-2-1-advising), oversee model releases and publications (Section ch-2-2-overseeing), support risk assessments (Section ch-2-3-supporting), review the company’s risk management practices (Section ch-2-4-reviewving), interpret AI ethics principles (Section ch-2-5-interpreting), or serve as a contact point for whistleblowers (Section ch-2-6-contact). Note that these responsibilities are neither mutually exclusive nor collectively exhaustive. The board could also have more than one responsibility.

Advising the board of directors

The board of directors plays a key role in the corporate governance of AI companies. It sets the company’s strategic priorities, is responsible for risk oversight, and has significant influence over management (e.g. it can replace senior executives). But since many board members only work part-time and rely on information provided to them by management, they need support from an independent ally in the company. Internal audit can be this ally, but the ethics board could serve as an additional layer of assurance.

Options. The ethics board could provide strategic advice on various topics. It could advocate against high-risk decisions and call for a more prudent and wiser course.

• Research priorities. Most AI companies have an overarching research agenda (e.g. DeepMind’s focus on reinforcement learningor Anthropic’s focus on empirical safety research). This agenda influences what projects the company works on. The ethics board could try to influence that agenda. It could advocate for increasing focus on safety and alignment research. More generally, it could caution against advancing capabilities faster than safety measures. The underlying principle is called “differential technological development”.
• Commercialization strategy. The ethics board could also advise on the company’s commercialization strategy. On the one hand, it is understandable that AI companies want to monetize their systems (e.g. to pay increasing costs for compute). On the other hand, commercial pressure might incentivize companies to cut corners on safety. For example, Google famously announced to “recalibrate” the level of risk it is willing to take in response to OpenAI’s release of ChatGPT. It has also been reported that disagreements over OpenAI’s commercialization strategy were the reason why key employees left the company to start Anthropic.
• Strategic partnerships. AI labs might enter into strategic partnerships with profit-oriented companies (see e.g. the extended partnership between Microsoft and OpenAI) or with the military (see e.g. “Project Maven”, Google’s collaboration with the U.S. Department of Defense). Although such partnerships are not inherently bad, they could contribute to an increase of risk (e.g. if they lead to an equipment of nuclear weapons with AI technology).
• Fundraising and M&A transactions. AI companies frequently need to bring in new investors. For example, in January 2023, it has been reported that OpenAI raised $10B from Microsoft. But if new investors care more about profits, this could gradually shift the company’s focus away from safety and ethics towards profit maximization. The same might happen if AI companies merge or get acquired. The underlying phenomena is called “mission drift”.

Discussion. How much would advising the board of directors reduce risk? This depends on many different factors. It would be easier if the ethics board has a direct communication channel to the board of directors, ideally to a dedicated risk committee. It would also be easier if the board of directors is able to do something about risks. They need risk-related expertise and governance structures to exercise their power (e.g. a chief risk officer [CRO] as a single point of accountability). But the board of directors also needs to take risks seriously and be willing to do something about them. This will often require a good relationship between the ethics board and the board of directors. Inversely, it would be harder for the ethics board to reduce risk if the board of directors mainly cares about other things (e.g. profits or prestige), especially since the ethics board is usually not able to force the board of directors to do something.

Overseeing model releases and publications

Many risks are caused by accidentsor the misuse of specific AI systems. In both cases, the deployment decision is a decisive moment. Ideally, companies should discover potential failure modes and vulnerabilities before they deploy a system, and stop the deployment process if they cannot reduce risks to an acceptable level. But not all risks are caused by the deployment of individual models. Some risks also stem from the publication of research, as research findings can be misused. The dissemination of potentially harmful information, including research findings, is called “infohazards”. Publications can also fuel harmful narratives. For example, it has been argued that the “arms race” rhetoric is highly problematic.

Options. An ethics board could try to reduce these risks by creating a release strategyand norms for the responsible publication of research. For example, the release strategy could establish “structured access” as the norm for deploying powerful AI systems. Instead of open-sourcing new models, companies might want to deploy them via an application programming interface (API), which would allow them to conduct know-your-customer (KYC) screenings and restrict access if necessary, while allowing the world to use and study the model. The release strategy could also specify instances where a “staged release” seems adequate. Stage release refers to the strategy of releasing a smaller model first, and only releasing larger models if no meaningful cases of misuse are observed. OpenAI has coined the term and championed the approach when releasing GPT-2. But note that the approach has also been criticized. The ethics board could also create an infohazard policy. The AI research organization Conjecture has published its policy. We expect most AI companies to have similar policies, but do not make them public. In addition to that, the board could oversee specific model releases and publications (not just the abstract strategies and policies). It could serve as an institutional review board (IRB) that cares about safety and ethics more generally, not just the protection of human subjects. In particular, it could review the risks of a model or publication itself, do a sanity check of existing reviews, or commission an external review (Section ch-2-3-supporting).

Discussion. How much would this reduce risk? Among other things, this depends on whether board members have the necessary expertise (Section ch-4-4characteristics), whether the board’s decisions are binding (Section ch-5-2-binding), and whether they have the necessary resources (Section ch-6-resources). The decision to release a model or publish research is one of the most important points of intervention for governance mechanisms that are intended to reduce risks. An additional attempt to steer such decisions in a good direction therefore seems desirable.

Supporting risk assessments

By “risk assessment”, we mean the identification, analysis, and evaluation of risks. Assessing the risks of state-of-the-art AI systems is extremely difficult: (1) The risk landscape is highly complex and evolves rapidly. For example, the increasing use of so-called “foundation models”might lead to new diffuse and systemic risks (e.g. threats to epistemic security). (2) Defining normative thresholds is extremely difficult: What level of risk is acceptable? How fair is fair enough? (3) In many cases, AI companies are also detached from the people who are most affected by their systems, often historically marginalized communities. (4) Risk assessments might become even more difficult in the future. For example, systems might become capable of deceiving their operators and only “pretending” to be safe in a testing environment.

Options. The ethics board could actively contribute to the different steps of a risk assessment. It could use a risk taxonomy to flag missing hazards, comment on a heatmap that illustrates the likelihood and severity of a risk, or try to circumvent a safety filter. It could also commission a third-party auditor red team. It could report its findings to the board of directors which would have the necessary power to intervene (Section ch-2-1-advising). Depending on its power, it might even be able to veto or at least delay deployment decisions (Section ch-5-2-binding).

Discussion. Some companies already take extensive measures to assess risks before deploying state-of-the-art AI systems. It is unclear how much value the support of an ethics board would add to such efforts. But especially when dealing with catastrophic risks, having an additional “layer of defense” seems generally desirable. The underlying concept is called “defense in depth”. This approach could be seen as a solution to the problem that “there is no silver bullet”. But supporting risk assessments could also have negative effects. If other teams rely on the board’s work, they might assess risks less thoroughly. This would be particularly problematic if the board is not able to do it properly (e.g. it can only perform sanity checks). But this effect could be mitigated by clearly communicating expectations and creating appropriate incentives.

Reviewing risk management practices

Instead of or in addition to supporting specific risk assessments (Section ch-2-3-supporting), the ethics board could review the company’s risk management practices more generally. In other words, it could try to improve the company’s “risk governance”. Risk management practices at AI companies seem to be less advanced compared to other industries like aviation. “They might look good on paper, but do not work in practice”. There are not yet any established best practices and companies rarely adhere to best practices from other industries (though there are promising developments around risk management standards). And practices that companies develop themselves might not be as effective. For example, there might be blind spots for certain types of risks (e.g. diffuse or systemic risks) or they might not account for cognitive biases (e.g. availability bias or scope neglect).

Options. The ethics board could assess the adequacy and effectiveness of the company’s risk management practices. It could assess whether the company complies with relevant regulations, standards, or its own policies and processes. It could also try to find flaws in a more open-ended fashion. Depending on its expertise and capacity, it could do this on its own (e.g. by reviewing risk-related policies and interviewing people in risk-related positions) or commission an external review of risk management practices (e.g. by an audit firm). Note that this role is usually performed by the company’s internal audit function, but the ethics board could provide an additional layer of assurance. They could report their findings directly to the risk committee of the board of directors and the chief risk officer (CRO) who could make risk management practices more effective.

Discussion. If companies already have an internal audit function, the additional value would be limited; the ethics board would merely be an additional defense layer. However, if companies do not already have an internal audit function, the added value could be significant. Without a deliberate attempt to identify ineffective risk management practices, some limitations will likely remain unnoticed. But the value ultimately depends on the individuals who conduct the review. This might be problematic because it will require a very specific type of expertise that most members of an ethics board do not have (Section ch-4-4characteristics). It is also very time-consuming, so a part-time board might not be able to do it properly (Section ch-4-5-time). Both issues should be taken into account when appointing members.

Interpreting AI ethics principles

Many AI companies have ethics principles, but “principles alone cannot guarantee ethical AI”. They are necessarily vague and need to be put into practice.

Options. The ethics board could interpret principles in the abstract (e.g. defining terms or clarifying the purpose of specific principles) or in concrete cases (e.g. whether a new research project violates a specific principle). In doing so, it could influence a wide range of risk-related decisions. For example, the board might decide that releasing a model that can easily be misused would violate the principle “be socially beneficial”, which is part of Google’s AI principles (Google, n.d.). When interpreting principles, the board could take a risk-based approach: the higher the risk, the more the company needs to do to mitigate it. The board could also suggest amendments to the principles.

Discussion. How much would this reduce risk? It will be more effective if the principles play a key role within the company. For example, Google’s motto “don’t be evil”—which it quietly removed in 2018—used to be part of its code of conduct and, reportedly, had a significant influence on its culture. Employees could threaten to leave the company or engage in other forms of activism if principles are violated. Interpreting ethics principles would also be more effective if the board’s interpretation is binding (Section ch-5-2-binding), and if the principles are public, mainly because civil society could hold the company accountable. It would be less effective if the principles are mainly a PR tool. This practice is called “ethics washing”.

Contact point for whistleblowers

Detecting misconduct is often difficult: it is hard to observe from the outside, while insiders might not report it because they face a conflict between personal values and loyaltyor because they fear negative consequences. For example, an engineer might find a severe safety flaw, but the research lead wants to release the model nonetheless and threatens to fire the engineer if they speak up. In such cases, whistleblower protection is vital.

Options. An ethics board could protect whistleblowers by providing a trusted contact point. The ethics board could report the case to the board of directors, especially the board risk committee, who could engage with management to do something about it. It could also advise the whistleblower on steps they could take to protect themselves (e.g. seeking legal assistance) or to do something about the misconduct (e.g. leaking the information to the press or a government agency).

Discussion. The ethics board would be more trustworthy than other organizational units (at least if it is independent from management). But since it would still be part of the company (Section ch-3-2-internal), or at least in a contractual relationship with it (Section ch-3-1-external), confidentiality would be less of a problem. This can be particularly important if the information is highly sensitive and its dissemination could be harmful in itself.

The ethics board can only serve this role if employees trust the ethics board, they know about the board’s commitment to whistleblower protection, and at least one board member needs to have relevant expertise and experience. For more information on the drivers of effective whistleblowing, we refer to the relevant literature. Anecdotally, whistleblowing within large AI companies has had some successes, though it did not always work. Overall, this role seems very promising, but the issue is highly delicate and could easily make things worse.

Structure

What should the board’s (legal) structure be? We can distinguish between internal (Section ch-3-1-external) and external structures (Section ch-3-2-internal). The board could also have substructures (Section ch-3-3-substructures).

External boards

The ethics board could be external. The company and the ethics board could be two separate legal entities. The relationship between the two entities would then be governed by a contract.

Options. The ethics board could be a nonprofit organization (e.g. a 501(c)(3)) or a for-profit company (e.g. a public-benefit corporation [PBC]). The individuals who provide services to the company could be members of the board of directors of the ethics board (Figure fig1a). Alternatively, they could be a group of individuals contracted by the ethics board (Figure fig1b) or by the company (Figure fig1c). There could also be more complex structures. For example, Meta’s Oversight Board consists of two separate entities: a purpose trust and a limited liability company (LLC). The purpose trust is funded by Meta and funds the LLC. The trustees are appointed by Meta, appoint individuals, and manage the LLC. The individuals are contracted by the LLC and provide services to Facebook and Instagram (Figure fig2).

Discussion. External ethics boards have a number of advantages: (1) They can legally bind the company through the contractual relationship (Section ch-5-1-process). This would be much more difficult for internal structures (Section ch-3-2-internal). (2) The board would be more independent, mainly because it would be less affected by internal incentives (e.g. board members could prioritize the public interest over the company’s interests). (3) It would be a more credible commitment because it would be more effective and more independent. The company might therefore be perceived as being more responsible. (4) The ethics board could potentially contract with more than one company. In doing so, it might build up more expertise and benefit from economies of scale. But external boards also have disadvantages. We expect that few companies are willing to make such a strong commitment, precisely because it would undermine its independence. It might also take longer to get the necessary information and a nuanced view of the inner workings of the company (e.g. norms and culture).

Internal boards

The ethics board could also be part of the company. Its members would be company employees. And the company would have full control over the board’s structure, its activities, and its members.

Options. An internal board could be a team, i.e. a permanent group of employees with a specific area of responsibility. But it could also be a working group or committee, i.e. a temporary group of employees with a specific area of responsibility, usually in addition to their main activity. For example, DeepMind’s IRC seems to be a committee, not a team.

Discussion. The key advantage of internal boards is that it is easier for them to get information (e.g. because they have a better network within the organization). They will typically also have a better understanding of the inner workings of the company (e.g. norms and culture). But internal structures also have disadvantages. They can be disbanded at the discretion of senior management or the board of directors. It would be much harder to play an adversarial role and openly talk about risks, especially when potential mitigations are in conflict with other objectives (e.g. profits). The board would not have much (legal) power. Decisions cannot be enforced. To have influence, it relies on good relationships with management (if collaborative) or the board of directors (if adversarial). Finally, board members would be less protected from repercussions if they advocate for unfavorable measures.

Substructures

Both internal and external boards could have substructures. Certain responsibilities could be delegated to a part of the ethics board.

Options. Two common substructures are committees and liaisons. (Note that an internal ethics board can be a committee of the company, but the ethics board can also have committees.) (1) Committees could be permanent (for recurring responsibilities) or temporary (to address one-time issues). For example, the board could have a permanent “deployment committee” that reviews model releases (Section ch-2-2-overseeing), or it could have a temporary committee for advising the board on an upcoming M&A transaction. For more information about the merits of committees in the context of the board of directors, we refer to the relevant literature. Meta’s Oversight Board has two types of committees: a “case selection committee” which sets criteria for cases that the board will select for review, and a “membership committee” which proposes new board members and recommends the removal or renewal of existing members. They can also set up other committees.

Liaisons are another type of substructure. Some members of the ethics board could join specific teams or other organizational structures (e.g. attend meetings of research projects or the board of directors). They would get more information about the inner workings of the company and can build better relationships with internal stakeholders (which can be vital if the board wants to protect whistleblowers, see Section ch-2-6-contact). Inversely, non-board members could be invited to attend board meetings. This could be important if the board lacks the necessary competence to make a certain decision (Section ch-4-4characteristics). For example, they could invite someone from the technical safety team to help them interpret the results of a third-party model audit. Microsoft’s AETHER Committee regularly invites engineers to working groups.

Discussion. On the one hand, substructures can make the board more complex and add friction. On the other hand, they allow for faster decision-making because less people are involved and group discussions tend to be more efficient. Against this background, we expect that substructures are probably only needed in larger ethics boards (Section ch-4-3-size).

Membership

Who should sit on the board? In particular, how should members join (Section ch-4-1-joining) and leave the board (Section ch-4-2-leaving)? How many members should the board have (Section ch-4-3-size)? What characteristics should they have (Section ch-4-4characteristics)? How much time should they spend on the board (Section ch-4-5-time)? And should they be compensated (Section ch-4-6-compensation)?

Joining the board

How should members join the board?

Options. We need to distinguish between the appointment of the initial and subsequent board members. Initial members could be directly appointed by the company’s board of directors. But the company could also set up a special formation committee which appoints the initial board members. The former was the case at Axon’s AI and Policing Technologies Ethics Board, the latter at Meta’s Oversight Board. Subsequent board members are usually appointed by the board itself. Meta’s Oversight Board has a special committee that selects subsequent members after a review of the candidates’ qualifications and a background check. But they could also be appointed by the company’s board of directors. Candidates could be suggested (not appointed) by other board members, the board of directors, or the general public. At Meta’s Oversight Board, new members can be suggested by other board members, the board of directors, and the general public.

Discussion. The appointment of initial board members is particularly important. If the company does not get this right, it could threaten the survival of the entire board. For example, Google appointed two controversial members to the initial board which sparked internal petitions to remove them and contributed to the board’s failure. The appointment should be done by someone with enough time and expertise. This suggests that a formation committee will often be advisable. The board would be more independent if it can appoint subsequent members itself. Otherwise, the company could influence the direction of the ethics board over time.

Leaving the board

How should members leave the board?

Options. There are at least three ways in which members could leave the board. First, their term could expire. The board’s charter or bylaws could specify a term limit. Members would leave the board when their term expires. For example, at Meta’s Oversight Board, the term ends after three years, but appointments can be renewed twice. Second, members could resign voluntarily. While members might resign for personal reasons, a resignation can also be used to express protest. For example, in the case of Google’s ATEAC, Alessandro Acquisti announced his resignation on Twitter to express protest against the setup of the board. Similarly, in the case of Axon’s AI and Policing Technologies Ethics Board, nine out of eleven members publically resigned after Axon announced plans to develop taser-equipped drones to be used in schools without consulting the board first. Third, board members could be removed involuntarily.

Discussion. Since any removal of board members is a serious step, it should only be possible under special conditions. In particular, it should require a special majority and a special reason (e.g. a violation of the board’s code of conduct or charter). To preserve the independence of the board, it should not be possible to remove board members for substantive decisions they have made.

Size of the board

How many members should the board have?

Options. In theory, the board can have any number of members. In practice, most boards have between 10-20 members (Table table1).

Discussion. On the one hand, larger boards can work on more cases and they can go into more detail. They can also be more diverse. On the other hand, it will often be difficult to find enough qualified people. Group discussions in smaller boards tend to be easier and it is easier to reach consensus (e.g. if a qualified majority is required [Section ch-5-1-process]). Smaller boards allow for closer personal relationships between board members. But conflicts of interest could have an outsized effect in smaller boards. As a rule of thumb, the number of members should scale with the board’s workload (“more cases, more members”).

Characteristics of members

What characteristics should board members have?

Options. When appointing board members, companies should at least consider candidates’ expertise, diversity, seniority, and public perception.

Discussion. (1) Different boards will require different types of expertise. But we expect most boards to benefit from technical, ethical, and legal expertise. (2) Members should be diverse along various dimensions, such as gender, race, and geographical representation. For example, Meta’s Oversight Board has geographic diversity requirements in its bylaws. They should adequately represent historically marginalized communities. Diverse perspectives are particularly important in the context of risk assessment (Section ch-2-3-supporting). For example, this will make it more likely that unprecedented risks are identified. (3) Board members may be more or less senior. By “seniority”, we mean a person’s position of status which typically corresponds to their work experience and is reflected in their title. More senior people tend to have more subject-matter expertise. The board of directors and senior management might also take them more seriously. As a consequence, it might be easier for them to build trust, get information, and influence key decisions. This is particularly important for boards that only advise and are not able to make binding decisions. However, it will often be harder for the company to find senior people. And in many cases, the actual work is done by junior people. (4) Finally, some board members might be “celebrities”. They would add “glamor” to the board, which the company could use for PR reasons. Inversely, appointing highly controversial candidates (e.g. who express sympathy to extreme political views) might put off other candidates and undermine the board’s credibility.

Time commitment

How much time should members spend on the board?

Options. Board members could work full-time (around 40 hours per week), part-time (around 15-20 hours per week), or even less (around 1-2 hours per week or as needed). None of the existing (external) boards seem to require full-time work. Members of Meta’s Oversight Board work part-time. And members of Axon’s AI and Policing Technologies Ethics Board only had two official board meetings per year, with ad-hoc contact between these meetings.

Discussion. The more time members spend working on the board, the more they can engage with individual cases. This would be crucial if cases are complex and stakes are high (e.g. if the board supports pre-deployment risk assessments, see Section ch-2-3-supporting). Full-time board members would also get a better understanding of the inner workings of the company. For some responsibilities, the board needs this understanding (e.g. if the board reviews the company’s risk management practices, see Section ch-2-4-reviewving). However, we expect it to be much harder to find qualified candidates who are willing to work full-time because they will likely have existing obligations or other opportunities. This is exacerbated by the fact that the relevant expertise is scarce. And even if a company finds qualified candidates who are willing to work full-time, hiring several full-time members can be a significant expense.

Compensation

Should board members be compensated?

Options. There are three options. First, serving on the ethics board could be unpaid. Second, board members could get reimbursed for their expenses (e.g. for traveling or for commissioning outside expertise). For example, Axon paid its board members $5,000 per year, plus a $5,000 honorarium per attended board meeting, plus travel expenses (AI and Policing Technologies Ethics Board, 2019). Third, board members could be fully compensated, either via a regular salary or honorarium. For example, it has been reported that members of Meta’s Oversight Board are being paid a six-figure salary.

Discussion. Not compensating board members or only reimbursing their expenses is only reasonable for part-time or light-touch boards. Full-time boards need to be compensated. Otherwise, it will be extremely difficult to find qualified candidates. For a more detailed discussion of how compensation can affect independence, see Section ch-6-1-funding.

Decision-making

Decision-making process

How should the board make decisions?

Options. We expect virtually all boards to make decisions by voting. This raises a number of questions:

• Majority. What majority should be necessary to adopt a decision? Boards could vote by absolute majority, i.e. a decision is adopted if it is supported by more than 50% of votes. For certain types of decisions, the board may also require a qualified majority (e.g. a unanimous vote or a 67% majority). Alternatively, boards could vote by plurality (or relative majority), i.e. a decision is adopted if it gets more votes than any other but does not receive more than half of all votes cast. The majority could be calculated based on the total number of board members (e.g. if the board has 10 members, 6 votes would constitute a simple majority), or the number of members present (e.g. if 7 members are present, 4 votes would constitute a simple majority). At Meta’s Oversight board, “outcomes will be determined by majority rule, based on the number of members present”.
• Voting rights. Who should be able to vote? There are three options. First, all board members could have voting rights. Second, only some board members could have voting rights. For example, only members of subcommittees could be able to vote on issues related to that subcommittee. This is the case at Meta’s Oversight Board. It would also be conceivable that some members only advise on special issues; they might be less involved in the board’s day-to-day work. These board members, while formally being part of the board, might not have voting rights. Third, non-board members could have (temporary) voting rights. For example, the board could ask external experts to advise on specific issues. These experts could be granted voting rights for this particular issue.
• Voting power. A related, but different question is: how much should a vote count? We expect this question to be irrelevant for most boards, as “one person, one vote” is so commonsensical. However, in some cases, boards may want to deviate from this. For example, the board could use quadratic voting, which allows individuals to express the degree of their preferences, rather than just the direction of their preferences.
• Quorum. What should the minimum number of members necessary to vote be? This is called a “quorum”. In principle, the quorum can be everything between one and all board members, though there might be legal requirements for some external structures. A natural quorum is the number of board members who could constitute a majority (e.g. more than 50% of board members if a simple majority is sufficient). It is also possible to have a different quorum for different types of decisions. Note that a lack of quorum might make the decision void or voidable.
• Voting method. How should the board vote? The most common voting methods are paper ballots, show of hands, postally, or electronically (e.g. using a voting app). According to its bylaws, voting at Meta’s Oversight Board takes place “in-person or electronically”.
• Abstention. In some cases, board members may want to abstain from a vote (e.g. because they do not feel adequately informed about the issue at hand, are uncertain, or mildly disapprove of the decision, but do not want to actively oppose it). Abstention could always be permitted or prohibited. The board could also allow abstention for some decisions, but not for others. Board members must abstain if they have a conflict of interest. At Meta’s Oversight Board, abstention is only prohibited for one type of decisions, namely for case deliberation.
• Proxy voting. Some board members may want to ask someone else to vote on their behalf. This is called “proxy voting”. Proxy voting could always be permitted or prohibited. The board could also allow proxy voting under certain circumstances (e.g. in the event of illness), only for certain decisions (e.g. less consequential decisions), or upon request. Meta’s Oversight Board does not allow proxy voting.
• Frequency of board meetings. How often should the board meet to vote? There are three options. First, the board could meet periodically (e.g. weekly, monthly, quarterly, or annually). Second, the board could meet on an ad hoc basis. Special meetings could be arranged at the board’s discretion, upon request by the company, and/or based on a catalog of special occasions (e.g. prior to the deployment of a new model). Third, the board could do both, i.e. meeting periodically and on an ad hoc basis. Meta’s Oversight Board meets annually and has special board meetings “in emergency or exceptional cases”. Google’s ATEAC planned to have four meetings per year.
• In-person or remote meetings. Should board meetings be held in person or remotely? We expect this design choice to be less important than most others, but it is a necessary one nonetheless. At Meta’s Oversight Board, meetings take place in person, though it does allow exceptions “in limited and exceptional circumstances”; its committees meet either in person or remotely.
• Preparation and convocation of board meetings. How should board meetings be prepared and convened? More precisely, who can convene a board meeting? What is the notice period? How should members be invited? What should the invitation entail? And do members need to indicate if they will attend? At Meta’s Oversight Board, “written notice of periodic and special meetings must specify the date, time, location, and purpose for convening the board. This notice will be provided at least eight weeks in advance for in-person convenings and, unless in case of imminent emergency, at least two days in advance for remote convenings. Members are required to acknowledge receipt of this notice and also indicate their attendance in a timely fashion”.
• Documentation and communication of decisions. Finally, it needs to be specified how decisions are documented and communicated. More precisely, which decisions should be documented and communicated? What exactly should be documented and communicated? And who should get access to the documentation? At Facebook’s Oversight Board, “minutes will be taken and circulated to board members within one week”. It does not publicly release meeting minutes, but has sometimes allowed reporters in their meetings. Google’s ATEAC planned to “publish a report summarizing the discussions”. Axon’s AI Ethics Board published two annual reports. In their 2019 report, they also highlight “the importance of public engagement and transparency”.

Discussion. Some of these questions might seem like formalities, but they can significantly affect the board’s work. For example, if the necessary majority or the quorum are too high, the board might not be able to adopt certain decisions. This could bias the board towards inaction. Similarly, if the board is not able to convene ad hoc meetings or only upon request by the company, they would not be able to respond adequately to emergencies.

Bindingness of decisions

Should the board’s decisions be binding?

Options. This mainly depends on the board’s structure (Section ch-3-structure). External boards can be set up in a way that their decisions are binding, i.e. enforceable by legal means. Both parties need to contractually agree that the board’s decisions are in fact binding. This agreement could also contain further details about the enforcement of the board’s decisions (e.g. contractual penalties). It is worth noting, however, that the ethics board cannot force the company to follow its decisions. The worst legal consequence for the company is a contractual liability. If the ethics board is part of the company, it is very difficult, if not impossible, to ensure that the board’s decisions are legally binding. If the board is able to make binding decisions, it needs to be specified whether and, if so, under what conditions the company can override them. For example, the contract could give the company’s board of directors the option to override a decision if they achieve the same voting majority as the ethics board. But even if the board’s decisions are not enforceable by legal means, there are non-legal means that can incentivize the company to follow the board’s decision. For example, the board could make its decisions public, which could spark a public outcry. One or more board members could (threaten to) resign, which might lead to negative PR. Employees could also (threaten to) leave the company (e.g. via an open letter), which could be a serious threat, depending how talent-constraint the company is. Finally, shareholders could engage in shareholder activism. In practice, the only ethics board that is able to make binding decisions is Meta’s Oversight Board, which has the power to override content moderation decisions.

Discussion. Boards that are able to make legally binding decisions are likely more effective, i.e. they are able to achieve their goals to a higher degree (e.g. reducing risks to an acceptable level). They would also be a more credible commitment to safety and ethics. However, we expect that many companies would oppose creating such a powerful ethics board, mainly because it would undermine the company’s power. There might also be legal constraints on how much power the company can transfer to the ethics board.

Resources

What resources does the board need? In particular, how much funding does the board need and where should the funding come from (Section ch-6-1-funding)? How should the board get information (Section ch-6-2-information)? And should it have access to outside expertise (Section ch-6-3-expertise)?

Funding

How much funding does the board need and where should the funding come from?

Options. The board might need funding to pay its members salaries or reimburse expenses (Section ch-4-6-compensation), to commission outside expertise (e.g. third-party audits or expert consulting), or to organize events (e.g. in-person board meetings). Funding could also allow board members to spend their time on non-administrative tasks. For example, the Policing Project provided staff support, facilitated meetings, conducted research, and drafted reports for Axon’s former AI and Policing Technologies Ethics Board. How much funding the board needs varies widely—from essentially no funding to tens of millions of dollars. For example, Meta’s Oversight Board has an annual budget of $20 million. Funding could come from the company (e.g. directly or via a trust) or philanthropists. Other funding sources do not seem plausible (e.g. state funding or research grants).

Discussion. The board’s independence could be undermined if funding comes directly from the company. The company could use the provision of funds as leverage to make the board take decisions that are more aligned with its interests. A more indirect funding mechanism therefore seems preferable. For example, Meta funds the purpose trust for multiple years in advance.

Information

How should the board get information?

Options. What information the board needs is highly context-specific and mainly depend on the board’s responsibilities (Section ch-2-responsibilities). The board’s structure determines what sources of information are available (Section ch-3-structure). While internal boards have access to some information by default, external boards have to rely on public information and information the company decides to share with them. Both internal and external boards might be able to gather additional information themselves (e.g. via formal document requests or informal coffee chats with employees).

Discussion. Getting information from the company is convenient for the board, but the information might be biased. The company might—intentionally or not—withhold, overemphasize, or misrepresent certain information. The company could also delay the provision of information or present them in a way that makes it difficult for the board to process (e.g. by hiding important information in long documents). To mitigate these risks, the board might prefer gathering information itself. In particular, the board might want to build good relationships with a few trusted employees. While this might be less biased, it would also be more time-consuming. It might also be impossible to get certain first-hand information (e.g. protocols of past meetings of the board of directors). It is worth noting that not all company information is equally biased. For example, while reports by management might be too positive, whistleblower reports might be too negative. The most objective information will likely come from the internal audit team and external assurance providers. In general, there is no single best information source. Boards need to combine multiple sources and cross-check important information.

Outside expertise

Should the board have access to outside expertise?

Options. There are at least three types of outside expertise the ethics board could harvest. First, it could hire a specialized firm (e.g. a law or consulting firm) to answer questions that are beyond its expertise (e.g. whether the company complies with the NIST AI Risk Management Framework). Second, it could hire an audit firm (e.g. to audit a specific model, the company’s governance, or its own practices). Third, it could build academic partnerships (e.g. to red-team a model).

Discussion. It might make sense for the ethics board to rely on outside expertise if they have limited expertise or time. They could also use it to get a more objective perspective, as information provided to them by the company can be biased (Section ch-6-2-information). However, the company might use the same sources of outside expertise. For example, if a company is open to a third-party audit, it would commission the audit directly (why would it ask the ethics board to do it on its behalf?). In such cases, the ethics board would merely “double-check” the company’s or the third party’s work. While the added value would be low, the costs could be high (especially for commissioning an external audit or expert consulting).

Conclusion

Summary. In this paper, we have identified key design choices that AI companies need to make when setting up an ethics board (RQ1). For each of them, we have listed different options and discussed how they would affect the board’s ability to reduce risks from AI (RQ2). Table table2 contains a summary of the design choices we have covered.

Key claims. Throughout this paper, we have made four key claims. First, ethics boards can take many different shapes. Most design choices are highly context-specific. It is therefore very difficult to make abstract recommendations. There is no one-size-fits-all. Second, ethics boards should be seen as an additional “layer of defense”. They do not have an original role in the corporate governance of AI companies. They do not serve a function that no other organizational structure serves. Instead, most ethics boards support, complement, or duplicate existing efforts. While this reduces efficiency, an additional safety net seems warranted in high-stakes situations. Third, merely having an ethics board is not sufficient. Most of the value depends on its members and their willingness and ability to pursue its mission. Thus, appointing the right people is crucial. Inversely, there is precedent that appointing the wrong people can threaten the survival of the entire board. Fourth, while some design choices might seem like formalities (e.g. when the board is quorate), they can have a significant impact on the effectiveness of the board (e.g. by slowing down decisions). They should not be taken lightly.

Questions for further research. The paper left many questions unanswered and more research is needed. In particular, our list of design choices is not comprehensive. For example, we did not address the issue of board oversight. If an ethics board has substantial powers, the board itself also needs adequate oversight. A “meta oversight board”—a central organization that oversees various AI ethics boards—could be a possible solution. Apart from that, our list of potential responsibilities could be extended. For example, the company could grant the ethics board the right to appoint one or more members of its board of directors. The ethics board could also oversee and coordinate responses to model evals. For example, if certain dangerous capabilities are detected, the company may want to contact government and coordinate with other labs to pause capabilities research.

We wish to conclude with a word of caution. Setting up an ethics board is not a silver bullet—“there is no silver bullet”. Instead, it should be seen as yet another mechanism in a portfolio of mechanisms.

Acknowledgements

We are grateful for valuable feedback from Christina Barta, Carrick Flynn, Cullen O’Keefe, Virginia Blanton, Andrew Strait, Tim Fist, and Milan Griffes. Anka Reuel worked on the project during the 2022 CHERI Summer Research Program. All remaining errors are our own.

Bibliography

   1@article{amodei2016concrete,
   2  journal = {arXiv preprint arXiv:1606.06565},
   3  year = {2016},
   4  author = {Dario Amodei and Chris Olah and Jacob Steinhardt and Paul Christiano and John Schulman and Dan Mané},
   5  title = {Concrete Problems in {AI} Safety},
   6}
   7
   8@article{sevilla2022compute,
   9  journal = {arXiv preprint arXiv:2202.05924},
  10  year = {2022},
  11  author = {Jaime Sevilla and Lennart Heim and Anson Ho and Tamay Besiroglu and Marius Hobbhahn and Pablo Villalobos},
  12  title = {Compute Trends Across Three Eras of Machine Learning},
  13}
  14
  15@article{carlsmith2022powerseeking,
  16  journal = {arXiv preprint arXiv:2206.13353},
  17  year = {2022},
  18  author = {Joseph Carlsmith},
  19  title = {Is Power-Seeking {AI} an Existential Risk?},
  20}
  21
  22@article{solaiman2023gradient,
  23  journal = {arXiv preprint arXiv:2302.04844},
  24  year = {2023},
  25  author = {Irene Solaiman},
  26  title = {The Gradient of Generative {AI} Release: Methods and Considerations},
  27}
  28
  29@article{ngo2023alignment,
  30  journal = {arXiv preprint arXiv:2209.00626},
  31  year = {2023},
  32  author = {Richard Ngo and Lawrence Chan and Sören Mindermann},
  33  title = {The alignment problem from a deep learning perspective},
  34}
  35
  36@article{openai2023gpt4technical,
  37  journal = {arXiv preprint arXiv:2303.08774},
  38  year = {2023},
  39  author = {{OpenAI}},
  40  title = {{GPT-4} Technical Report},
  41}
  42
  43@article{brundage2018malicious,
  44  journal = {arXiv preprint arXiv:1802.07228},
  45  year = {2018},
  46  author = {Miles Brundage and Shahar Avin and Jack Clark and Helen Toner and Peter Eckersley and Ben Garfinkel and Allan Dafoe and Paul Scharre and Thomas Zeitzoff and Bobby Filar and Hyrum Anderson and Heather Roff and Gregory C. Allen and Jacob Steinhardt and Carrick Flynn and Se\'an \'O h\'Eigeartaigh and Simon Beard and Haydn Belfield and Sebastian Farquhar and Clare Lyle and Rebecca Crootof and Owain Evans and Michael Page and Joanna Bryson and Roman Yampolskiy and Dario Amodei},
  47  title = {The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation},
  48}
  49
  50@article{anderljung2023protecting,
  51  journal = {arXiv preprint arXiv:2303.09377},
  52  year = {2023},
  53  author = {Markus Anderljung and Julian Hazell},
  54  title = {Protecting Society from {AI} Misuse: When are Restrictions on Capabilities Warranted?},
  55}
  56
  57@article{schuett2022lines,
  58  journal = {arXiv preprint arXiv:2212.08364},
  59  year = {2022},
  60  author = {Jonas Schuett},
  61  title = {Three lines of defense against risks from {AI}},
  62}
  63
  64@article{hendrycks2022unsolved,
  65  journal = {arXiv preprint arXiv:2109.13916},
  66  year = {2022},
  67  author = {Dan Hendrycks and Nicholas Carlini and John Schulman and Jacob Steinhardt},
  68  title = {Unsolved Problems in {ML} Safety},
  69}
  70
  71@article{goldstein2023generative,
  72  journal = {arXiv preprint arXiv:2301.04246},
  73  year = {2023},
  74  author = {Josh A. Goldstein and Girish Sastry and Micah Musser and Renee DiResta and Matthew Gentzel and Katerina Sedova},
  75  title = {Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations},
  76}
  77
  78@article{solaiman2019release,
  79  journal = {arXiv preprint arXiv:1908.09203},
  80  year = {2019},
  81  author = {Irene Solaiman and Miles Brundage and Jack Clark and Amanda Askell and Ariel Herbert-Voss and Jeff Wu and Alec Radford and Gretchen Krueger and Jong Wook Kim and Sarah Kreps and Miles McCain and Alex Newhouse and Jason Blazakis and Kris McGuffie and Jasmine Wang},
  82  title = {Release Strategies and the Social Impacts of Language Models},
  83}
  84
  85@article{bommasani2022opportunities,
  86  journal = {arXiv preprint arXiv:2108.07258},
  87  year = {2022},
  88  author = {Rishi Bommasani and Drew A. Hudson and Ehsan Adeli and Russ Altman and Simran Arora and Sydney von Arx and Michael S. Bernstein and Jeannette Bohg and Antoine Bosselut and Emma Brunskill and Erik Brynjolfsson and Shyamal Buch and Dallas Card and Rodrigo Castellon and Niladri Chatterji and Annie Chen and Kathleen Creel and Jared Quincy Davis and Dora Demszky and Chris Donahue and Moussa Doumbouya and Esin Durmus and Stefano Ermon and John Etchemendy and Kawin Ethayarajh and Li Fei-Fei and Chelsea Finn and Trevor Gale and Lauren Gillespie and Karan Goel and Noah Goodman and Shelby Grossman and Neel Guha and Tatsunori Hashimoto and Peter Henderson and John Hewitt and Daniel E. Ho and Jenny Hong and Kyle Hsu and Jing Huang and Thomas Icard and Saahil Jain and Dan Jurafsky and Pratyusha Kalluri and Siddharth Karamcheti and Geoff Keeling and Fereshte Khani and Omar Khattab and Pang Wei Koh and Mark Krass and Ranjay Krishna and Rohith Kuditipudi and Ananya Kumar and Faisal Ladhak and Mina Lee and Tony Lee and Jure Leskovec and Isabelle Levent and Xiang Lisa Li and Xuechen Li and Tengyu Ma and Ali Malik and Christopher D. Manning and Suvir Mirchandani and Eric Mitchell and Zanele Munyikwa and Suraj Nair and Avanika Narayan and Deepak Narayanan and Ben Newman and Allen Nie and Juan Carlos Niebles and Hamed Nilforoshan and Julian Nyarko and Giray Ogut and Laurel Orr and Isabel Papadimitriou and Joon Sung Park and Chris Piech and Eva Portelance and Christopher Potts and Aditi Raghunathan and Rob Reich and Hongyu Ren and Frieda Rong and Yusuf Roohani and Camilo Ruiz and Jack Ryan and Christopher Ré and Dorsa Sadigh and Shiori Sagawa and Keshav Santhanam and Andy Shih and Krishnan Srinivasan and Alex Tamkin and Rohan Taori and Armin W. Thomas and Florian Tramèr and Rose E. Wang and William Wang and Bohan Wu and Jiajun Wu and Yuhuai Wu and Sang Michael Xie and Michihiro Yasunaga and Jiaxuan You and Matei Zaharia and Michael Zhang and Tianyi Zhang and Xikun Zhang and Yuhui Zhang and Lucia Zheng and Kaitlyn Zhou and Percy Liang},
  89  title = {On the Opportunities and Risks of Foundation Models},
  90}
  91
  92@article{mökander2023auditing,
  93  journal = {arXiv preprint arXiv:2302.08500},
  94  year = {2023},
  95  author = {Jakob Mökander and Jonas Schuett and Hannah Rose Kirk and Luciano Floridi},
  96  title = {Auditing large language models: A three-layered approach},
  97}
  98
  99@article{ganguli2022red,
 100  journal = {arXiv preprint arXiv:2209.07858},
 101  year = {2022},
 102  author = {Deep Ganguli and Liane Lovitt and Jackson Kernion and Amanda Askell and Yuntao Bai and Saurav Kadavath and Ben Mann and Ethan Perez and Nicholas Schiefer and Kamal Ndousse and Andy Jones and Sam Bowman and Anna Chen and Tom Conerly and Nova DasSarma and Dawn Drain and Nelson Elhage and Sheer El-Showk and Stanislav Fort and Zac Hatfield-Dodds and Tom Henighan and Danny Hernandez and Tristan Hume and Josh Jacobson and Scott Johnston and Shauna Kravec and Catherine Olsson and Sam Ringer and Eli Tran-Johnson and Dario Amodei and Tom Brown and Nicholas Joseph and Sam McCandlish and Chris Olah and Jared Kaplan and Jack Clark},
 103  title = {Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned},
 104}
 105
 106@article{perez2022red,
 107  journal = {arXiv preprint arXiv:2202.03286},
 108  year = {2022},
 109  author = {Ethan Perez and Saffron Huang and Francis Song and Trevor Cai and Roman Ring and John Aslanides and Amelia Glaese and Nat McAleese and Geoffrey Irving},
 110  title = {Red Teaming Language Models with Language Models},
 111}
 112
 113@article{rando2022redteaming,
 114  journal = {arXiv preprint arXiv:2210.04610},
 115  year = {2022},
 116  author = {Javier Rando and Daniel Paleka and David Lindner and Lennart Heim and Florian Tramèr},
 117  title = {Red-Teaming the {Stable Diffusion} Safety Filter},
 118}
 119
 120@article{weidinger2021ethical,
 121  journal = {arXiv preprint arXiv:2112.04359},
 122  year = {2021},
 123  author = {Laura Weidinger and John Mellor and Maribeth Rauh and Conor Griffin and Jonathan Uesato and Po-Sen Huang and Myra Cheng and Mia Glaese and Borja Balle and Atoosa Kasirzadeh and Zac Kenton and Sasha Brown and Will Hawkins and Tom Stepleton and Courtney Biles and Abeba Birhane and Julia Haas and Laura Rimell and Lisa Anne Hendricks and William Isaac and Sean Legassick and Geoffrey Irving and Iason Gabriel},
 124  title = {Ethical and social risks of harm from Language Models},
 125}
 126
 127@article{brundage2020trustworthy,
 128  journal = {arXiv preprint arXiv:2004.07213},
 129  year = {2020},
 130  author = {Miles Brundage and Shahar Avin and Jasmine Wang and Haydn Belfield and Gretchen Krueger and Gillian Hadfield and Heidy Khlaaf and Jingying Yang and Helen Toner and Ruth Fong and Tegan Maharaj and Pang Wei Koh and Sara Hooker and Jade Leung and Andrew Trask and Emma Bluemke and Jonathan Lebensold and Cullen O'Keefe and Mark Koren and Théo Ryffel and JB Rubinovitz and Tamay Besiroglu and Federica Carugati and Jack Clark and Peter Eckersley and Sarah de Haas and Maritza Johnson and Ben Laurie and Alex Ingerman and Igor Krawczuk and Amanda Askell and Rosario Cammarota and Andrew Lohn and David Krueger and Charlotte Stix and Peter Henderson and Logan Graham and Carina Prunkl and Bianca Martin and Elizabeth Seger and Noa Zilberman and Se\'an \'O h\'Eigeartaigh and Frens Kroeger and Girish Sastry and Rebecca Kagan and Adrian Weller and Brian Tse and Elizabeth Barnes and Allan Dafoe and Paul Scharre and Ariel Herbert-Voss and Martijn Rasser and Shagun Sodhani and Carrick Flynn and Thomas Krendl Gilbert and Lisa Dyer and Saif Khan and Yoshua Bengio and Markus Anderljung},
 131  title = {Toward Trustworthy {AI} Development: Mechanisms for Supporting Verifiable Claims},
 132}
 133
 134@article{schuett2022risk,
 135  year = {2023},
 136  pages = {1--19},
 137  journal = {European Journal of Risk Regulation},
 138  author = {Jonas Schuett},
 139  title = {Risk management in the {Artificial Intelligence Act}},
 140}
 141
 142@article{urbina2022dual,
 143  year = {2022},
 144  pages = {189--191},
 145  number = {3},
 146  volume = {4},
 147  journal = {Nature Machine Intelligence},
 148  author = {Urbina, Fabio and Lentzos, Filippa and Invernizzi, C{\'e}dric and Ekins, Sean},
 149  title = {Dual use of artificial-intelligence-powered drug discovery},
 150}
 151
 152@article{degrave2022magnetic,
 153  publisher = {Nature Publishing Group},
 154  year = {2022},
 155  pages = {414--419},
 156  number = {7897},
 157  volume = {602},
 158  journal = {Nature},
 159  author = {Degrave, Jonas and Felici, Federico and Buchli, Jonas and Neunert, Michael and Tracey, Brendan and Carpanese, Francesco and Ewalds, Timo and Hafner, Roland and Abdolmaleki, Abbas and de Las Casas, Diego and others},
 160  title = {Magnetic control of tokamak plasmas through deep reinforcement learning},
 161}
 162
 163@article{klonick2020facebook,
 164  year = {2020},
 165  number = {2418},
 166  volume = {129},
 167  journal = {Yale Law Journal},
 168  author = {Klonick, Kate},
 169  title = {The {Facebook Oversight Board}: Creating an independent institution to adjudicate online free expression},
 170}
 171
 172@article{wong2022meta,
 173  year = {2022},
 174  pages = {1--24},
 175  journal = {Minds and Machines},
 176  author = {Wong, David and Floridi, Luciano},
 177  title = {{Meta’s Oversight Board}: A review and critical assessment},
 178}
 179
 180@article{morley2021ethics,
 181  year = {2021},
 182  pages = {239--256},
 183  number = {2},
 184  volume = {31},
 185  journal = {Minds and Machines},
 186  author = {Morley, Jessica and Elhalal, Anat and Garcia, Francesca and Kinsey, Libby and M{\"o}kander, Jakob and Floridi, Luciano},
 187  title = {Ethics as a service: a pragmatic operationalisation of {AI} ethics},
 188}
 189
 190@article{bernstein2021ethics,
 191  year = {2021},
 192  number = {52},
 193  volume = {118},
 194  journal = {Proceedings of the National Academy of Sciences},
 195  author = {Bernstein, Michael S and Levi, Margaret and Magnus, David and Rajala, Betsy A and Satz, Debra and Waeiss, Quinn},
 196  title = {Ethics and society review: Ethics reflection as a precondition to research funding},
 197}
 198
 199@article{aven2013meaning,
 200  year = {2013},
 201  pages = {44--51},
 202  volume = {57},
 203  journal = {Safety Science},
 204  author = {Aven, Terje},
 205  title = {On the meaning of a black swan in a risk context},
 206}
 207
 208@article{kolt2023algorithmic,
 209  year = {2023},
 210  volume = {101},
 211  journal = {Washington University Law Review},
 212  author = {Kolt, Noam},
 213  title = {Algorithmic Black Swans},
 214}
 215
 216@article{cihon2021corporate,
 217  year = {2021},
 218  number = {7},
 219  volume = {12},
 220  journal = {Information},
 221  author = {Cihon, Peter and Schuett, Jonas and Baum, Seth D},
 222  title = {Corporate governance of artificial intelligence in the public interest},
 223}
 224
 225@article{davies2018three,
 226  year = {2018},
 227  pages = {34--42},
 228  volume = {9},
 229  journal = {Global Policy},
 230  author = {Davies, Howard and Zhivitskaya, Maria},
 231  title = {Three lines of defence: A robust organising framework, or just lines in the sand?},
 232}
 233
 234@article{silver2021reward,
 235  year = {2021},
 236  volume = {299},
 237  journal = {Artificial Intelligence},
 238  author = {Silver, David and Singh, Satinder and Precup, Doina and Sutton, Richard S},
 239  title = {Reward is enough},
 240}
 241
 242@article{bostrom2001analyzing,
 243  year = {2001},
 244  number = {1},
 245  volume = {9},
 246  journal = {Journal of Evolution and Technology},
 247  author = {Bostrom, Nick},
 248  title = {Existential risks: Analyzing human extinction scenarios and related hazards},
 249}
 250
 251@article{armstrong2016racing,
 252  year = {2016},
 253  pages = {201--206},
 254  volume = {31},
 255  journal = {AI \& Society},
 256  author = {Armstrong, Stuart and Bostrom, Nick and Shulman, Carl},
 257  title = {Racing to the precipice: A model of artificial intelligence development},
 258}
 259
 260@article{naude2020race,
 261  year = {2020},
 262  pages = {367--379},
 263  volume = {35},
 264  journal = {AI \& Society},
 265  author = {Naud{\'e}, Wim and Dimitri, Nicola},
 266  title = {The race for an artificial general intelligence: implications for public policy},
 267}
 268
 269@article{grimes2019anchors,
 270  year = {2019},
 271  pages = {819--845},
 272  number = {4},
 273  volume = {44},
 274  journal = {Academy of Management Review},
 275  author = {Grimes, Matthew G and Williams, Trenton Alma and Zhao, Eric Yanfei},
 276  title = {Anchors aweigh: The sources, variety, and challenges of mission drift},
 277}
 278
 279@article{maas2019viable,
 280  year = {2019},
 281  pages = {285--311},
 282  number = {3},
 283  volume = {40},
 284  journal = {Contemporary Security Policy},
 285  author = {Maas, Matthijs M},
 286  title = {How viable is international arms control for military artificial intelligence? Three lessons from nuclear weapons},
 287}
 288
 289@article{bostrom2019vulnerable,
 290  year = {2019},
 291  pages = {455--476},
 292  number = {4},
 293  volume = {10},
 294  journal = {Global Policy},
 295  author = {Bostrom, Nick},
 296  title = {The vulnerable world hypothesis},
 297}
 298
 299@article{bostrom2011information,
 300  year = {2011},
 301  pages = {44--79},
 302  volume = {10},
 303  journal = {Review of Contemporary Philosophy},
 304  author = {Bostrom, Nick},
 305  title = {Information hazards: A typology of potential harms from knowledge},
 306}
 307
 308@article{srikumar2022advancing,
 309  year = {2022},
 310  pages = {1061--1064},
 311  number = {12},
 312  volume = {4},
 313  journal = {Nature Machine Intelligence},
 314  author = {Srikumar, Madhulika and Finlay, Rebecca and Abuhamad, Grace and Ashurst, Carolyn and Campbell, Rosie and Campbell-Ratcliffe, Emily and Hongo, Hudson and Jordan, Sara R and Lindley, Joseph and Ovadya, Aviv and others},
 315  title = {Advancing ethics review practices in {AI} research},
 316}
 317
 318@article{cotton2020defence,
 319  year = {2020},
 320  pages = {271--282},
 321  number = {3},
 322  volume = {11},
 323  journal = {Global Policy},
 324  author = {Cotton-Barratt, Owen and Daniel, Max and Sandberg, Anders},
 325  title = {Defence in depth against human extinction: Prevention, response, resilience, and why they all matter},
 326}
 327
 328@article{van2011risk,
 329  year = {2011},
 330  pages = {431--449},
 331  number = {4},
 332  volume = {14},
 333  journal = {Journal of Risk Research},
 334  author = {Van Asselt, Marjolein BA and Renn, Ortwin},
 335  title = {Risk governance},
 336}
 337
 338@article{lundqvist2015firms,
 339  year = {2015},
 340  pages = {441--466},
 341  number = {5},
 342  volume = {34},
 343  journal = {Journal of Accounting and Public Policy},
 344  author = {Lundqvist, Sara A},
 345  title = {Why firms implement risk governance: Stepping beyond traditional risk management to enterprise risk management},
 346}
 347
 348@article{mokander2022operationalising,
 349  year = {2022},
 350  pages = {1--18},
 351  journal = {AI and Ethics},
 352  author = {M{\"o}kander, Jakob and Floridi, Luciano},
 353  title = {Operationalising {AI} governance through ethics-based auditing: An industry case study},
 354}
 355
 356@article{jobin2019global,
 357  year = {2019},
 358  pages = {389--399},
 359  number = {9},
 360  volume = {1},
 361  journal = {Nature Machine Intelligence},
 362  author = {Jobin, Anna and Ienca, Marcello and Vayena, Effy},
 363  title = {The global landscape of {AI} ethics guidelines},
 364}
 365
 366@article{hagendorff2020ethics,
 367  year = {2020},
 368  pages = {99--120},
 369  number = {1},
 370  volume = {30},
 371  journal = {Minds and Machines},
 372  author = {Hagendorff, Thilo},
 373  title = {The ethics of {AI} ethics: An evaluation of guidelines},
 374}
 375
 376@article{mittelstadt2019principles,
 377  year = {2019},
 378  pages = {501--507},
 379  number = {11},
 380  volume = {1},
 381  journal = {Nature machine intelligence},
 382  author = {Mittelstadt, Brent},
 383  title = {Principles alone cannot guarantee ethical {AI}},
 384}
 385
 386@article{morley2020initial,
 387  year = {2020},
 388  pages = {2141--2168},
 389  number = {4},
 390  volume = {26},
 391  journal = {Science and Engineering Ethics},
 392  author = {Morley, Jessica and Floridi, Luciano and Kinsey, Libby and Elhalal, Anat},
 393  title = {From what to how: an initial review of publicly available {AI} ethics tools, methods and research to translate principles into practices},
 394}
 395
 396@article{zhou2022ai,
 397  year = {2022},
 398  pages = {1--11},
 399  journal = {AI \& Society},
 400  author = {Zhou, Jianlong and Chen, Fang},
 401  title = {{AI} ethics: From principles to practice},
 402}
 403
 404@article{baldwin2016driving,
 405  year = {2016},
 406  pages = {565--595},
 407  number = {4},
 408  volume = {43},
 409  journal = {Journal of Law and Society},
 410  author = {Baldwin, Robert and Black, Julia},
 411  title = {Driving Priorities in Risk-Based Regulation: What's the Problem?},
 412}
 413
 414@article{mahler2021between,
 415  year = {2021},
 416  journal = {Nordic Yearbook of Law and Informatics},
 417  author = {Mahler, Tobias},
 418  title = {Between risk management and proportionality: The risk-based approach in the {EU’s Artificial Intelligence Act} proposal},
 419}
 420
 421@article{mokander2021ethics,
 422  year = {2021},
 423  number = {44},
 424  volume = {27},
 425  journal = {Science and Engineering Ethics},
 426  author = {M{\"o}kander, Jakob and Morley, Jessica and Taddeo, Mariarosaria and Floridi, Luciano},
 427  title = {Ethics-based auditing of automated decision-making systems: Nature, scope, and limitations},
 428}
 429
 430@article{falco2021governing,
 431  year = {2021},
 432  pages = {566--571},
 433  number = {7},
 434  volume = {3},
 435  journal = {Nature Machine Intelligence},
 436  author = {Falco, Gregory and Shneiderman, Ben and Badger, Julia and Carrier, Ryan and Dahbura, Anton and Danks, David and Eling, Martin and Goodloe, Alwyn and Gupta, Jerry and Hart, Christopher and others},
 437  title = {Governing {AI} safety through independent audits},
 438}
 439
 440@article{chamberlain2022risk,
 441  year = {2022},
 442  pages = {1--13},
 443  number = {1},
 444  volume = {14},
 445  journal = {European Journal of Risk Regulation},
 446  author = {Chamberlain, Johanna},
 447  title = {The Risk-Based Approach of the {European Union’s} Proposed Artificial Intelligence Regulation: Some Comments from a Tort Law Perspective},
 448}
 449
 450@article{crofts2020negotiating,
 451  year = {2020},
 452  pages = {1--16},
 453  number = {1},
 454  volume = {2},
 455  journal = {Law, Technology and Humans},
 456  author = {Crofts, Penny and van Rijswijk, Honni},
 457  title = {Negotiating 'evil': {Google}, {Project Maven} and the corporate form},
 458}
 459
 460@article{seger2022defence,
 461  year = {2022},
 462  pages = {45},
 463  number = {2},
 464  volume = {35},
 465  journal = {Philosophy \& Technology},
 466  author = {Seger, Elizabeth},
 467  title = {In defence of principlism in {AI} ethics and governance},
 468}
 469
 470@article{jubb1999whistleblowing,
 471  year = {1999},
 472  pages = {77--94},
 473  volume = {21},
 474  journal = {Journal of Business Ethics},
 475  author = {Jubb, Peter B},
 476  title = {Whistleblowing: A restrictive definition and interpretation},
 477}
 478
 479@article{dungan2015psychology,
 480  year = {2015},
 481  pages = {129--133},
 482  volume = {6},
 483  journal = {Current Opinion in Psychology},
 484  author = {Dungan, James and Waytz, Adam and Young, Liane},
 485  title = {The psychology of whistleblowing},
 486}
 487
 488@article{bjorkelo2013workplace,
 489  year = {2013},
 490  pages = {306--323},
 491  number = {3},
 492  volume = {28},
 493  journal = {Journal of Managerial Psychology},
 494  author = {Bj{\o}rkelo, Brita},
 495  title = {Workplace bullying after whistleblowing: Future research and implications},
 496}
 497
 498@article{near1995effective,
 499  year = {1995},
 500  pages = {679--708},
 501  number = {3},
 502  volume = {20},
 503  journal = {Academy of management review},
 504  author = {Near, Janet P and Miceli, Marcia P},
 505  title = {Effective whistle-blowing},
 506}
 507
 508@article{apaza2011makes,
 509  year = {2011},
 510  pages = {113--130},
 511  number = {2},
 512  volume = {13},
 513  journal = {Public Integrity},
 514  author = {Apaza, Carmen R and Chang, Yongjin},
 515  title = {What makes whistleblowing effective: Whistleblowing in {Peru} and {South Korea}},
 516}
 517
 518@article{mohamed2020decolonial,
 519  year = {2020},
 520  pages = {659--684},
 521  volume = {33},
 522  journal = {Philosophy \& Technology},
 523  author = {Mohamed, Shakir and Png, Marie-Therese and Isaac, William},
 524  title = {Decolonial {AI}: Decolonial theory as sociotechnical foresight in artificial intelligence},
 525}
 526
 527@article{posner2014quadratic,
 528  year = {2014},
 529  pages = {251--272},
 530  number = {1},
 531  volume = {81},
 532  journal = {The University of Chicago Law Review},
 533  author = {Posner, Eric A and Weyl, E Glen},
 534  title = {Quadratic voting as efficient corporate governance},
 535}
 536
 537@article{floridi2021translating,
 538  year = {2021},
 539  pages = {81--90},
 540  journal = {Ethics, Governance, and Policies in Artificial Intelligence},
 541  author = {Floridi, Luciano},
 542  title = {Translating principles into practices of digital ethics: Five risks of being unethical},
 543}
 544
 545@book{taleb2007black,
 546  publisher = {Random House},
 547  year = {2007},
 548  author = {Taleb, Nassim Nicholas},
 549  title = {The Black Swan: The Impact of the Highly Improbable},
 550}
 551
 552@book{ord2020precipice,
 553  publisher = {Hachette Books},
 554  year = {2020},
 555  author = {Ord, Toby},
 556  title = {The precipice: Existential risk and the future of humanity},
 557}
 558
 559@inproceedings{ashurst2022disentangling,
 560  year = {2022},
 561  pages = {2057--2068},
 562  booktitle = {2022 ACM Conference on Fairness, Accountability, and Transparency},
 563  author = {Ashurst, Carolyn and Barocas, Solon and Campbell, Rosie and Raji, Deborah},
 564  title = {Disentangling the Components of Ethical Research in Machine Learning},
 565}
 566
 567@inproceedings{shevlane2020offense,
 568  year = {2020},
 569  pages = {173--179},
 570  booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
 571  author = {Shevlane, Toby and Dafoe, Allan},
 572  title = {The Offense-Defense Balance of Scientific Knowledge: Does Publishing {AI} Research Reduce Misuse?},
 573}
 574
 575@inproceedings{cave2018ai,
 576  year = {2018},
 577  pages = {36--40},
 578  booktitle = {Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society},
 579  author = {Cave, Stephen and {\'O}h{\'E}igeartaigh, Se{\'a}n S},
 580  title = {An {AI} race for strategic advantage: Rhetoric and risks},
 581}
 582
 583@inproceedings{ashurst2022ai,
 584  year = {2022},
 585  pages = {2047--2056},
 586  booktitle = {2022 ACM Conference on Fairness, Accountability, and Transparency},
 587  author = {Ashurst, Carolyn and Hine, Emmie and Sedille, Paul and Carlier, Alexis},
 588  title = {{AI} ethics statements: Analysis and lessons learnt from {NeurIPS} broader impact statements},
 589}
 590
 591@inproceedings{raji2019actionable,
 592  year = {2019},
 593  pages = {429--435},
 594  booktitle = {Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society},
 595  author = {Raji, Inioluwa Deborah and Buolamwini, Joy},
 596  title = {Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial {AI} products},
 597}
 598
 599@inproceedings{raji2022outsider,
 600  year = {2022},
 601  pages = {557--571},
 602  booktitle = {Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society},
 603  author = {Raji, Inioluwa Deborah and Xu, Peggy and Honigsberg, Colleen and Ho, Daniel},
 604  title = {Outsider oversight: Designing a third party audit ecosystem for {AI} governance},
 605}
 606
 607@inproceedings{yudkowsky2008cognitive,
 608  year = {2008},
 609  pages = {91--119},
 610  booktitle = {Global catastrophic risks},
 611  author = {Yudkowsky, Eliezer},
 612  title = {Cognitive biases potentially affecting judgment of global risks},
 613}
 614
 615@inproceedings{belfield2020activism,
 616  year = {2020},
 617  pages = {15--21},
 618  booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
 619  author = {Belfield, Haydn},
 620  title = {Activism by the {AI} community: Analysing recent achievements and future prospects},
 621}
 622
 623@inproceedings{bietti2020ethics,
 624  year = {2020},
 625  pages = {210--219},
 626  booktitle = {Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency},
 627  author = {Bietti, Elettra},
 628  title = {From ethics washing to ethics bashing: A view on tech ethics from within moral philosophy},
 629}
 630
 631@inproceedings{lalley2018quadratic,
 632  year = {2018},
 633  pages = {33--37},
 634  volume = {108},
 635  booktitle = {AEA Papers and Proceedings},
 636  author = {Lalley, Steven P and Weyl, E Glen},
 637  title = {Quadratic voting: How mechanism design can radicalize democracy},
 638}
 639
 640@inproceedings{jordan2019designing,
 641  year = {2019},
 642  pages = {1--7},
 643  booktitle = {2019 IEEE International Symposium on Technology and Society (ISTAS)},
 644  author = {Jordan, Sara R},
 645  title = {Designing artificial intelligence review boards: creating risk metrics for review of {AI}},
 646}
 647
 648@inproceedings{shevlane2022structured,
 649  year = {2022},
 650  booktitle = {The Oxford Handbook of AI Governance},
 651  author = {Shevlane, Toby},
 652  title = {Structured Access: An Emerging Paradigm for Safe {AI} Deployment},
 653}
 654
 655@article{sandler2019building,
 656  year = {2019},
 657  journal = {Accenture \& Northeastern University},
 658  author = {Sandler, Ronald and Basl, John and Tiell, Steven},
 659  title = {Building data and {AI} ethics committees},
 660}
 661
 662@article{newman2020decisionpoints,
 663  year = {2020},
 664  journal = {Center for Long-Term Cybersecurity, UC Berkeley},
 665  author = {Jessica Newman},
 666  title = {Decision Points in {AI} Governance},
 667}
 668
 669@article{arnold2021aiaccidents,
 670  year = {2021},
 671  journal = {Center for Security and Emerging Technology, Georgetown University},
 672  author = {Zachary Arnold and Helen Toner},
 673  title = {{AI} Accidents: An Emerging Threat},
 674}
 675
 676@article{adalovelace2022looking,
 677  year = {2022},
 678  journal = {Ada Lovelace Institute},
 679  author = {Mylene Petermann and Niccolo Tempini and Ismael Kherroubi Garcia and Kirstie Whitaker and Andrew Strait},
 680  title = {Looking before we leap},
 681}
 682
 683@article{seger2020tackling,
 684  year = {2020},
 685  journal = {The Alan Turing Institute},
 686  author = {Seger, Elizabeth and Avin, Shahar and Pearson, Gavin and Briers, Mark and Heigeartaigh, Se{\'a}n {\'O} and Bacon, Helena and Ajder, Henry and Alderson, Claire and Anderson, Fergus and Baddeley, Joseph and others},
 687  title = {Tackling threats to informed decision-making in democratic societies: Promoting epistemic security in a technologically-advanced world},
 688}
 689
 690@article{hunt2020flight,
 691  year = {2020},
 692  journal = {Center for Long-Term Cybersecurity, UC Berkeley},
 693  author = {Will Hunt},
 694  title = {The Flight to Safety-Critical {AI}},
 695}
 696
 697@misc{oversightboard2023website,
 698  howpublished = {\url{https://www.oversightboard.com}},
 699  year = {2023},
 700  author = {{Oversight Board}},
 701}
 702
 703@misc{microsoft2023ourapproach,
 704  howpublished = {\url{https://www.microsoft.com/en-us/ai/our-approach}},
 705  year = {2023},
 706  author = {Microsoft},
 707  title = {Our approach},
 708}
 709
 710@misc{deepmind2023humanrightspolicy,
 711  howpublished = {\url{https://www.deepmind.com/human-rights-policy}},
 712  year = {2022},
 713  author = {{DeepMind}},
 714  title = {Human Rights Policy},
 715}
 716
 717@misc{deepmind2023principles,
 718  howpublished = {\url{https://www.deepmind.com/blog/how-our-principles-helped-define-alphafolds-release}},
 719  year = {2022},
 720  author = {Koray Kavukcuoglu and Pushmeet Kohli and Lila Ibrahim and Dawn Bloxwich and Sasha Brown},
 721  title = {How our principles helped define {AlphaFold’s} release},
 722}
 723
 724@article{piper2019googlefail,
 725  howpublished = {\url{https://www.vox.com/future-perfect/2019/4/3/18292526/google-ai-ethics-board-letter-acquisti-kay-coles-james}},
 726  journal = {Vox},
 727  year = {2019},
 728  author = {Kelsey Piper},
 729  title = {Google’s brand-new {AI} ethics board is already falling apart},
 730}
 731
 732@misc{acquisti2019tweet,
 733  year = {2019},
 734  howpublished = {\url{https://twitter.com/ssnstudy/status/1112099054551515138}},
 735  author = {Alessandro Aquisti},
 736}
 737
 738@misc{walker2019external,
 739  howpublished = {\url{https://blog.google/technology/ai/external-advisory-council-help-advance-responsible-development-ai/}},
 740  year = {2019},
 741  author = {Walker, K},
 742  title = {An external advisory council to help advance the responsible development of {AI}},
 743}
 744
 745@misc{smith2022axon,
 746  year = {2022},
 747  howpublished = {\url{https://www.axon.com/news/technology/axon-committed-to-listening-and-learning}},
 748  author = {Smith, Rick},
 749  title = {Axon committed to listening and learning so that we can fulfill our mission to protect life, together},
 750}
 751
 752@article{policing2022resigning,
 753  year = {2022},
 754  howpublished = {\url{https://www.policingproject.org/statement-of-resigning-axon-ai-ethics-board-members}},
 755  journal = {Policing Project},
 756  author = {Barry Friedman and Wael Abd-Almageed and Miles Brundage and Ryan Calo and Danielle Citron and Rebekah Delsol and Chris Harris and Jennifer Lynch and Mecole McBride},
 757  title = {Statement of Resigning {Axon AI} Ethics Board Members},
 758}
 759
 760@article{tiell2019create,
 761  year = {2019},
 762  volume = {15},
 763  journal = {Harvard Business Review},
 764  author = {Tiell, Steven},
 765  title = {Create an ethics committee to keep your {AI} Initiative in check},
 766}
 767
 768@article{gupta2020ai,
 769  year = {2020},
 770  journal = {MIT Technology Review},
 771  author = {Gupta, Abhishek and Heath, Victoria},
 772  title = {{AI} ethics groups are repeating one of society’s classic mistakes},
 773}
 774
 775@article{sandbrink2022differential,
 776  year = {2022},
 777  journal = {SSRN},
 778  author = {Sandbrink, Jonas and Hobbs, Hamish and Swett, Jacob and Dafoe, Allan and Sandberg, Anders},
 779  title = {Differential technology development: A responsible innovation principle for navigating technology risks},
 780}
 781
 782@article{grant2023googlehelp,
 783  year = {2023},
 784  howpublished = {\url{https://www.nytimes.com/2023/01/20/technology/google-chatgpt-artificial-intelligence.html}},
 785  journal = {The New York Times},
 786  author = {Nico Grant},
 787  title = {Google Calls In Help From {Larry Page} and {Sergey Brin} for {A.I.} Fight},
 788}
 789
 790@misc{waters2021rebelai,
 791  year = {2021},
 792  howpublished = {\url{https://www.ft.com/content/8de92f3a-228e-4bb8-961f-96f2dce70ebb}},
 793  journal = {Financial Times},
 794  author = {Richard Waters and Miles Kruppa},
 795  title = {Rebel {AI} group raises record cash after machine learning schism},
 796}
 797
 798@misc{microsoft2023openai,
 799  year = {2023},
 800  howpublished = {\url{https://blogs.microsoft.com/blog/2023/01/23/microsoftandopenaiextendpartnership/}},
 801  author = {Microsoft},
 802  title = {Microsoft and {OpenAI} extend partnership},
 803}
 804
 805@article{conger2018pentagonai,
 806  year = {2018},
 807  howpublished = {\url{https://gizmodo.com/google-is-helping-the-pentagon-build-ai-for-drones-1823464533}},
 808  journal = {Gizmodo},
 809  author = {Kate Conger and Dell Cameron},
 810  title = {Google Is Helping the {Pentagon} Build {AI} for Drones},
 811}
 812
 813@article{hoffman2023microsoftchatgpt,
 814  year = {2023},
 815  journal = {Semafor},
 816  howpublished = {\url{https://www.semafor.com/article/01/09/2023/microsoft-eyes-10-billion-bet-on-chatgpt}},
 817  author = {Liz Hoffman and Reed Albergotti},
 818  title = {Microsoft eyes \$10 billion bet on {ChatGPT}},
 819}
 820
 821@misc{openai2023microsoft,
 822  year = {2023},
 823  url = {https://openai.com/blog/openai-and-microsoft-extend-partnership},
 824  author = {OpenAI},
 825  title = {{OpenAI} and {Microsoft} extend partnership},
 826}
 827
 828@article{leahy2022conjecture,
 829  year = {2022},
 830  howpublished = {\url{https://www.alignmentforum.org/posts/Gs29k3beHiqWFZqnn/conjecture-internal-infohazard-policy}},
 831  journal = {Alignment Forum},
 832  author = {Connor Leahy and Sid Black and Chris Scammell and Andrea Miotti},
 833  title = {Conjecture: Internal Infohazard Policy},
 834}
 835
 836@misc{openai2022bestpractices,
 837  year = {2022},
 838  howpublished = {\url{https://openai.com/blog/best-practices-for-deploying-language-models}},
 839  author = {{OpenAI}},
 840  title = {Best practices for deploying language models},
 841}
 842
 843@article{crootof2019artificial,
 844  year = {2019},
 845  journal = {Lawfare Blog},
 846  author = {Crootof, Rebecca},
 847  title = {Artificial intelligence research needs responsible publication norms},
 848}
 849
 850@misc{partnershiponai2021managingrisks,
 851  year = {2021},
 852  author = {{Partnership on AI}},
 853  title = {Managing the Risks of {AI} Research},
 854}
 855
 856@article{birhane2022power,
 857  year = {2022},
 858  pages = {1--8},
 859  journal = {Equity and Access in Algorithms, Mechanisms, and Optimization},
 860  author = {Birhane, Abeba and Isaac, William and Prabhakaran, Vinodkumar and D{\'\i}az, Mark and Elish, Madeleine Clare and Gabriel, Iason and Mohamed, Shakir},
 861  title = {Power to the People? {Opportunities} and Challenges for Participatory {AI}},
 862}
 863
 864@misc{kavukcuoglu2022alphafold,
 865  year = {2022},
 866  howpublished = {\url{https://www.deepmind.com/blog/how-our-principles-helped-define-alphafolds-release}},
 867  journal = {DeepMind},
 868  author = {Koray Kavukcuoglu and Pushmeet Kohli and Lila Ibrahim and Dawn Bloxwich and Sasha Brown},
 869  title = {How our principles helped define AlphaFold’s release},
 870}
 871
 872@misc{axon2022ethicsequity,
 873  year = {2022},
 874  howpublished = {\url{https://www.axon.com/eeac}},
 875  author = {Axon},
 876  title = {Ethics \& {Equity Advisory Council}},
 877}
 878
 879@misc{iso51_2014,
 880  year = {2014},
 881  title = {{Guide 51:2014 Safety aspects — Guidelines for their inclusion in standards}},
 882  author = {{ISO/IEC}},
 883}
 884
 885@misc{iso31000_2018,
 886  year = {2018},
 887  title = {{31000:2018 Risk management — Guidelines}},
 888  author = {ISO},
 889}
 890
 891@misc{nist2023artificial,
 892  year = {2023},
 893  author = {NIST},
 894  title = {{Artificial Intelligence Risk Management Framework (AI RMF 1.0})},
 895}
 896
 897@misc{iso23894_2023,
 898  year = {2023},
 899  title = {{23894:2023 Information technology — Artificial intelligence — Guidance on risk management}},
 900  author = {{ISO/IEC}},
 901}
 902
 903@article{thomas2019independence,
 904  year = {2019},
 905  journal = {Business Law Today},
 906  author = {Thomas, VC and Duda, JP and Maurer, TG},
 907  title = {Independence With A Purpose: Facebook’s Creative Use Of {Delaware’s} Purpose Trust Statute To Establish Independent Oversight},
 908}
 909
 910@misc{oversightboard2023trustees,
 911  year = {2023},
 912  howpublished = {\url{https://www.oversightboard.com/governance}},
 913  title = {Trustees},
 914  author = {{Oversight Board}},
 915}
 916
 917@misc{anthropic2023coreviews,
 918  year = {2023},
 919  howpublished = {\url{https://www.anthropic.com/index/core-views-on-ai-safety}},
 920  title = {Core Views on {AI} Safety: When, Why, What, and How},
 921  author = {Anthropic},
 922}
 923
 924@misc{iec31010_2019,
 925  year = {2019},
 926  title = {{31010:2019 Risk management — Risk assessment techniques}},
 927  author = {{IEC}},
 928}
 929
 930@article{openai2022lessonslearned,
 931  year = {2022},
 932  howpublished = {\url{https://openai.com/research/language-model-safety-and-misuse}},
 933  journal = {OpenAI},
 934  title = {Lessons learned on language model safety and misuse},
 935  author = {Miles Brundage and Katie Mayer and Tyna Eloundou and Sandhini Agarwal and Steven Adler and Gretchen Krueger and Jan Leike and Pamela Mishkin},
 936}
 937
 938@misc{oversight2022bylaws,
 939  year = {2023},
 940  howpublished = {\url{https://www.oversightboard.com/sr/governance/bylaws}},
 941  author = {{Oversight Board}},
 942  title = {Bylaws},
 943}
 944
 945@misc{oversight2023charter,
 946  year = {2023},
 947  howpublished = {\url{https://oversightboard.com/attachment/494475942886876/}},
 948  author = {{Oversight Board}},
 949  title = {Charter},
 950}
 951
 952@misc{oversight2022funding,
 953  year = {2022},
 954  howpublished = {\url{https://www.oversightboard.com/news/1111826643064185-securing-ongoing-funding-for-the-oversight-board/}},
 955  author = {{Oversight Board}},
 956  title = {Securing ongoing funding},
 957}
 958
 959@article{klonick2021insights,
 960  year = {2021},
 961  howpublished = {\url{https://www.newyorker.com/tech/annals-of-technology/inside-the-making-of-facebooks-supreme-court}},
 962  journal = {New Yorker},
 963  author = {Kate Klonick},
 964  title = {Insight The Making Of {Facebook's} Supreme Court},
 965}
 966
 967@misc{microsoft2020principles,
 968  year = {2020},
 969  howpublished = {\url{https://www.microsoft.com/cms/api/am/binary/RE4pKH5}},
 970  author = {Microsoft},
 971  title = {Putting principles into practice: How we approach responsible {AI} at {Microsoft}},
 972}
 973
 974@misc{policingproject2020reports,
 975  year = {2020},
 976  howpublished = {\url{https://www.policingproject.org/axon}},
 977  author = {{Policing Project}},
 978  title = {Reports of the Axon {AI} Ethics Board},
 979}
 980
 981@misc{axon2019policing,
 982  year = {2019},
 983  author = {Axon},
 984  title = {First Report of the {Axon AI \& Policing Technology Ethics Board}},
 985}
 986
 987@misc{oversightboard2023members,
 988  year = {2023},
 989  howpublished = {\url{https://www.oversightboard.com/meet-the-board/}},
 990  author = {{Oversight Board}},
 991  title = {Our Commitment},
 992}
 993
 994@article{transphobia2019googlers,
 995  year = {2019},
 996  howpublished = {\url{https://medium.com/@against.transphobia/googlers-against-transphobia-and-hate-b1b0a5dbf76}},
 997  journal = {Medium},
 998  author = {{Googlers Against Transphobia}},
 999  title = {Googlers against transphobia and hate},
1000}

Attribution

arXiv:2304.07249v1 [cs.CY]
License: cc-by-4.0