- Papers
Papers is Alpha. This content is part of an effort to make research more accessible, and (most likely) has lost some details from the original. You can find the original paper here.
Introduction
The capabilities of AI systems have grown quickly over the last decade. Employing a growing wealth of algorithmic insights, data sources and computation power, AI researchers have created systems that can comprehend language, recognize and generate images and video, write computer programs and engage in scientific reasoning. If current trends in AI capabilities continue, AI systems could have transformative impacts on society.
Powerful AI systems will bring significant benefits and risks. These systems have the capacity to significantly improve the productivity of workers and economies as a whole, and to help us address some of our most important social and technological challenges. But these systems also present challenges including workforce dislocation, lack of transparency, biased outcomes, inequitably shared benefits and threats to national security.
Promoting AI benefits and managing AI risks both have domestic and international components. On the domestic front, governments and the private sector will need to establish rules and norms around how advanced AI systems are developed, distributed, deployed and accessed, addressing issues like security, distributive impacts, privacy, bias, and more. A number of challenges have the potential to transcend national borders and impact societies and economies worldwide. Accordingly, policymakers, technologists, and AI governance experts have recently begun to call for specific global AI governance initiatives centered on international institutions.[See, e.g.,
This paper contributes to these early conversations by discussing why AI governance may be needed on an international scale and then offering a non-exhaustive taxonomy of the institutional functions that international efforts might require. It explores four possible international institutions to perform these functions: 1) a Commission on Frontier AI that facilitates expert consensus on opportunities and risks from advanced AI; 2) an Advanced AI Governance Organization that sets international standards, supports their implementation, and could monitor compliance to future governance regimes; 3) a Frontier AI Collaborative that develops and distributes cutting-edge AI; and 4) an AI Safety Project that brings together exceptional researchers, engineers and compute to further AI safety research. Each of these approaches seeks to mitigate the societal challenges of advanced AI in different ways and each confronts significant challenges to its viability and success.
The Need for International Governance
Powerful AI systems have the potential to transform society, economics and politics in fundamental ways. Because of characteristics like its high barriers to development/utilization and the possibility of cross-border use, it is possible that harnessing AI’s potential for global benefit and managing its risks could require governance functions at the international level.
Promoting Global Benefits
Access to appropriate AI technology might greatly promote prosperity and stability, but the benefits might not be evenly distributed or focused on the greatest needs of underrepresented communities or the developing world. Inadequate access to internet services, computing power, or availability of machine learning training/expertise will also hinder certain groups’ ability to benefit fully from AI advances.
International institutions have long sought to support sustainable global development. International efforts to build consensus on AI opportunities—especially addressing barriers to their effective use globally—could support efforts to distribute and enable access to AI. On top of facilitating access, this could also include building capacity to benefit from AI through education, infrastructure, and local commercial ecosystems. In some cases, international collaborations (including public-private partnerships) to develop frontier AI systems that are suited to the needs to underserved communities may also be appropriate.
Inconsistent national regulations could also slow the development and deployment of AI, as developers of powerful AI technology may be unwilling to export to countries with inconsistent or unsuitable technology governance.[In addition to compliance costs, they may be concerned about enabling misuse, or the theft of proprietary information.] International efforts to set safety norms and standards could help coordinate governance in a way that supports innovation and serves a broad set of interests.
Managing Shared Risks
Advanced AI capabilities may also create negative global externalities. AI systems today are already capable of not just progressing drug discovery and development, but also of (re)inventing dangerous chemicalsand solving foundational problems in synthetic biology. Scientific capabilities like these could be weaponized by malicious actors for use worldwide. AI may also be used to create potent cyberweapons that can generate code, scan codebases for vulnerabilities, and engineer polymorphic malware in ways that threaten critical infrastructure. Existing AI systems already pose mis- and dis-information issues, and the introduction of more advanced systems is leading malicious actors to explore more sophisticated methods of information warfare.[See, e.g, Furthermore, building systems that act as intended in novel circumstances is a challenging problem that may only grow more difficult. As systems get increasingly capable, there will be greater incentives to deploy them in higher stakes domains where accidents could have serious global consequences.
Implementing protocols for responsible development and deployment will help address these risks of accident and misuse,[Potential protocols include: training methods that restrict the dangerous capabilities and increase the reliability of systems, subjecting AI systems to risk assessments that ascertain their propensity to cause harm before training or deployment, deployment protocols that secure systems against misuse or the exfiltration of its parameters, post-deployment monitoring to identify and respond to unforeseen risks. See.] on top of measures targeted at specific downstream issues.[Such as the Digital Services Act for disinformation, and treaties targeting chemical and biological weapons.] However, cross-border access to AI products and the cross-border effects of misuse and accidents suggests that national regulation may be ineffective for managing the risks of AI even within states. States will inevitably be impacted by the development of such capabilities in other jurisdictions.
To further the international adoption of safety protocols for advanced models, it would be useful to build consensus on risks and how they can be mitigated*,* and set safety norms and standards and support their implementation to help developers and regulators with responsible development and use. International efforts to conduct or support AI safety research may be beneficial, if it can increase the rate of safety progress or the reach of its outputs.
In the longer term, continued algorithmic and hardware progress could make systems capable of causing significant harm accessible to a large number of actors, greatly increasing the governance challenges.[According toand, the compute costs of training a model of a fixed performance level decreases approximately tenfold every 2 years.] In this case, the international community might explore measures like controlling AI inputs (although the dual-use/general purpose nature of the technology creates significant tradeoffs to doing so) and developing and/or enabling safe forms of access to AI.
The significant geopolitical benefits of AI development may disincline states to adequately regulate AI: arguments about national competitiveness are already raised against AI regulation,[See, e.g., the arguments discussed in and such pressures may strengthen alongside AI progress. We may eventually need international agreements that address these geopolitical risks, with institutions that can monitor compliance where feasible.[Monitoring can vary significantly in intrusiveness and effectiveness: while it will be highly difficult to implement adequate monitoring across major geopolitical divides, the safety of advanced systems could be a shared interest of major powers and a regime to address risk factors from smaller-scale geopolitical competition may be feasible.] Efforts to control AI inputs may be useful to enable non-proliferation of potentially dangerous capabilities and increase the technical feasibility of monitoring. More speculatively, efforts to develop frontier AI collectively or distribute and enable access and its benefits could incentivize participation in a governance regime.
—
The institutional functions identified above can be summarized and grouped into the following two broad categories.
- Science and Technology Research, Development and Diffusion
- Conduct or support AI safety research:
- Research and develop of measures to reduce the risks of AI misuse and accidents stemming from system characteristics like dangerous capabilities and unreliability/misalignment. This includes work on understanding and evaluating these characteristics and the threats they pose, training methods to reduce and manage risky behaviors, and examining safe deployment protocols appropriate to different system.
- Build consensus on opportunities and risks:
- Further international understanding of the opportunities and challenges created by advanced AI and possible strategies for mitigating the risks.
- Develop frontier AI:
- Build cutting-edge AI systems.
- Distribute and enable access to cutting edge AI:
- Facilitate access to cutting-edge systems and increase absorptive capacity through education, infrastructure, and support of the local commercial ecosystem.
- International Rulemaking and Enforcement
- Set safety norms and standards:
- Establish guidelines and standards around how AI can be developed, deployed and regulated to maximize benefit and minimize risks.
- Support implementation of standards:
- Provide assistance for the implementation of established guidelines and standards.
- Monitor compliance:
- Conduct audits /evaluations and issue certifications / licenses to ensure adherence to international standards and agreements.
- Control AI inputs:
- Manage or monitor models, compute, data and other ingredients of potentially dangerous technologies.
International bodies already perform some of these functions.[Seefor a more thorough overview.] The OECD’s AI Principles and AI Policy Observatory work, the ITU’s AI for Good initiative, and expert reports from the Global Partnership on AI’s Working Group on Responsible AI are early efforts at building consensus on AI opportunities and risks. Relatedly, the UK’s proposed Foundation Model Taskforceand the US’s proposed Multilateral AI Research Institute could emerge as multilateral efforts to conduct AI safety research, or potentially even develop frontier AI systems, though both are in exploratory phases.[The amount of funding required to stay on the cutting-edge of AI capabilities is significant. See, e.g.,.]
Alongside lawmaking efforts like the EU’s AI Act and the Council of Europe’s Convention on AI, Human Rights and Democracy, we have seen early norm and standard setting efforts from ISO/IEC, but little in the way of implementation support, oversight or certification. In terms of controlling dangerous inputs: computing resources have been targeted by US, Japanese and Dutch export controls that prevent the sale of certain AI chips and semiconductor manufacturing equipment to China.
International Institutions for Advanced AI
We have outlined several AI governance functions that might be needed at an international level, and shown that only a limited number of these are currently being performed by existing institutions. In this section, we discuss how functional gaps may be filled.
The functions could be split in multiple ways across institutions: drawing on existing international organizations and proposals, we describe four idealized models. We note that the models described in this section describe roles that could be filled by existing or new institutions. Participants in these institutions could include governments, non-governmental organizations, the private sector, and academia. Table 1 summarizes the previous analysis and the functions of the institutions we discuss.
Commission on Frontier AI: Fostering
Scientific Consensus
There have been several recent proposals of an intergovernmental body to develop expert consensus on the challenges and opportunities presented by advanced AI.[See, e.g.,.] Existing institutions like the Intergovernmental Panel on Climate Change (IPCC), the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) and the Scientific Assessment Panel (SAS), which studies ozone depletion under the Montreal Protocol, provide possible models for an AI-focused scientific institution. Like these organizations, the Commission on Frontier AI could facilitate scientific consensus by convening experts to conduct rigorous and comprehensive assessments of key AI topics, such as interventions to unlock AI’s potential for sustainable development, the effects of AI regulation on innovation, the distribution of benefits, and possible dual-use capabilities from advanced systems and how they ought to be managed.
Functions and Motivation
International consensus on the opportunities and risks from advanced AI has the potential to facilitate effective action addressing them, for example, by engendering a shared desire for the development and adoption of effective risk mitigation strategies.
Currently, there is significant disagreement even among experts about the different opportunities and challenges created by advanced AI,[See, for example, the disagreement around whether advanced AI could pose an extinction risk:.] and this lack of consensus may worsen over time as the effects of AI systems increase in scale and number, hindering collective action on the scale necessary to ensure that AI is developed for the benefit of all. Furthermore, there are several challenges from advanced AI that may require international action before risks materialize, and the lack of a widely accepted account or even mapping of AI development trajectories makes it difficult to take such preparatory actions. Facilitating consensus among an internationally representative group of experts could be a promising first step to expanding our levels of confidence in predicting and responding to technological trends.
Challenges and Risks
Scientific challenges of understanding risks on the horizon: Understanding frontier AI risks and their mitigation is technically challenging. The nature of future AI capabilities and their impact is difficult to predict, especially given the fast rate of progress. To increase chances of success, a Commission should foreground scientific rigor and the selection of highly competent AI experts who work at the cutting edge of technological development and who can continually interpret the ever-changing technological and risk landscape.
Unfortunately, there is a relative lack of existing scientific research on the risks of advanced AI.[The recent IPCC assessment, for reference, was written by 234 scientists from 66 states and drew on 14,000 scientific papers.] To address the lack of existing scientific research, a Commission might undertake activities that draw and facilitate greater scientific attention, such as organizing conferences and workshops and publishing research agendas. It may be helpful to write a foundational “Conceptual Framework”—following the example of the IPBES—to create a common language and framework that allows the integration of disparate strands of existing work and paves the way for future efforts.
Politicization: A Commission on Frontier AI would benefit from, if not require, a clear buffer between experts charged with developing consensus narratives around the risks and opportunities of AI and policymakers acting on the political and economic interests of their states, which might push policies in different directions. The scientific understanding of the impacts of AI should ideally be seen as a universal good and not be politicized.
Membership structure can affect a Commission’s impartiality and legitimacy: ideally, there would be broad geographic representation in the main decisionmaking bodies, and a predominance of scientific experts in working groups.[If legitimacy is the primary concern, the Commission might adopt the IPCC’s innovation of writing key documents by consensus, balancing inclusion (states’ representatives and scientists review, discuss and approve the report line by line) and scientific rigor (all suggested amendments must be consisted with working group’s scientific report that is being summarized).] Unfortunately, given the uncertain and controversial nature of advanced AI risks and opportunities, representation may trade off against a Commission’s ability to overcome scientific challenges and generate meaningful consensus.[If it follows the IPCC model, experts will be nominated by member states, but there will not be a robust climate science discipline to buffer against political interests.] In addition to striking the correct balance in membership, a Commission should carefully scope the subject matter of their research—it may, for example, adopt the IPCC’s objective of being “policy-relevant” without being “policy-prescriptive.”
Assessment
The objective of a Commission on Frontier AI is worthwhile in most circumstances, but the scientific challenges and potential of politicization imply that a Commission—especially one that aims at broad political representation—may not be able to build scientific consensus effectively. The extraordinary pace of technological change may require more nimble policy responses, such as less institutionalized and politically authoritative scientific advisory panels on advanced AI.
Advanced AI Governance Organization: Promoting Norms and
Standards, Providing Implementation Support, Monitoring Compliance
As discussed above, certain misuse and accident risks of advanced AI systems may pose significant global threats, and international efforts aimed at managing these risks could be worthwhile. An intergovernmental or multi-stakeholder organization could perform a variety of governance functions furthering the regulation of such systems, in particular norm and standard setting, implementation assistance, and perhaps monitoring compliance with governance frameworks.[See, e.g.,for discussions and proposals of an institution of this type.]
Functions and Motivation
We identify two main objectives for an Advanced AI Governance Organization. How much emphasis it should place on each depends on the challenges it aims to address.
Objective 1: Internationalizing and harmonizing AI regulation. Regulatory regimes that set standards and provide implementation support may help ensure that powerful AI capabilities do not pose misuse or accident risks. Standard setting would facilitate widespread international adoption by: 1) reducing the burden on domestic regulators to identify necessary safety regulations and protocols, 2) generating normative pressure for safety protocol adoption, and 3) reducing frictions around the development of international frameworks. Implementation support would assist the establishment and maintenance of regulatory regimes meeting these frameworks. Examples of organizations that perform similar functions include the Financial Action Task Force (FATF), the International Telecommunication Union (ITU) and the International Civil Aviation Organization (ICAO).
The same functions are useful for harmonizing regulation: international standard setting would reduce cross-border frictions due to differing domestic regulatory regimes. (It is possible that future regulations will limit access to powerful AI technologies in jurisdictions with inadequate AI governance.) Implementation support would help reduce obstacles to countries meeting international standards and therefore enable greater access to advanced AI.
Objective 2: Monitoring compliance. Where states have incentives to undercut each other’s regulatory commitments, international institutions may be needed to support and incentivize best practices. That may require monitoring standards compliance. At the least intrusive end of the spectrum is self-reporting of compliance with international standards (as in the Paris Agreement—see proposals for self-reporting/registration of training runs). Organizations like the FATF, ICAO, and the International Maritime Organization (IMO) take a somewhat more intrusive approach, monitoring jurisdictions to ensure they adopt best practice regulations, and in some cases checking on the enforcement of domestic regulations embodying international standard. In the case of advanced AI, some observers have asked whether more intrusive forms of international oversight might be necessary, including detection and inspections of large data centers (partly analogous to IAEA safeguards). The more intense and intrusive any monitoring, the more challenging it may be to get to consensus.
Challenges and Risks
Speed and comprehensiveness in standard setting: One challenge for a Governance Organization is that standard setting (especially in an international and multistakeholder context) tends to be a slow process, while the rapid and unpredictable nature of frontier AI progress may require more rapid international action. A Governance Organization may need to partner with faster-moving expert bodies and expedited standard-setting approaches. The breadth of membership may also represent a trade-off between speed and diversity of perspectives. Broader membership may be important where long-term consensus is important, while urgent risks may need to be addressed at first by smaller groups of frontier AI states, or aligned states with relevant expertise.
Incentivizing participation: The impact of a Governance Organization depends on states adopting its standards and/or agreeing to monitoring. Broad agreement (or agreement among frontier AI states at least) about the risks that standards and monitoring address and financial and technical support for standards’ implementation may help induce states’ participation. Many states—even those that are not full members of the organization—adopt FATF standards because they view them as in their own interests. Other AI-specific incentives for participation include conditioning on participation access to AI technology (possibly from a Frontier AI Collaborative) or computing resources.[The cloud compute industry and the underlying semiconductor supply chain are concentrated in a small number of countries.] States might also adopt import restrictions on AI from countries that are not certified by a Governance Organization—similar, for instance, to the way states prohibit flights from jurisdictions without ICAO-certification from entering their airspace.
In the more distant case of high stakes agreements governing AI development by states (such as arms control treaties), some states may be especially reluctant to join due to fear of clandestine noncompliance by other states. They may also worry that international inspections could compromise state secrets to the benefit of adversaries (which information security protocols could address in part). Again, the current reliance of advanced AI development on significant computing resources may make it easier to track significant AI efforts.[Oversight of data centers may allow the detection of large training runs that are subject to international controls. See.] Automated (even AI-enabled) monitoring may allow closer inspection of large training runs without compromising secrets. Such measures would likely hinge on negotiated verification regimes rather than national technical means—and negotiating verification is always fraught (e.g., in the case of the Chemical Weapons Convention) and often unsuccessful (e.g., in the case of the Biological Weapons Convention).
Scoping challenges: Unlike many other technologies—from nuclear resources to aviation—AI is already broadly deployed and used by billions of people every day. To operate efficiently and at appropriate scale, a Governance Organization should focus primarily on advanced AI systems that pose significant global risks, but it will be difficult in practice to decide on the nature and sophistication of AI tools that should be broadly available and uncontrolled versus the set of systems that should be subject to national or international governance. The rapid evolution of these technologies compounds the problem, as the technological frontier is advancing quickly, and models that were “frontier” a year ago are now both outdated and widely available.
Assessment
If advanced AI poses misuse and accident risks of a global scope, and unilateral technical defenses are not sufficient to protect against them, an international Governance Organization may be valuable. However, its effectiveness will depend on its membership, governance and standard-setting processes.
It may be important for governance to apply to all countries, and particularly to those whose firms are on the frontier of AI development. Yet, aligned countries may seek to form governance clubs, as they have in other domains. This facilitates decision-making, but may make it harder to enlist other countries later in the process. It is unclear what institutional processes would satisfy the demands of legitimacy and effectiveness, and incentivize the participation of important groups of stakeholders.
Frontier AI Collaborative: Enabling International Access to AI
Policymakers and pundits have also proposed collaborations to develop and distribute cutting-edge AI systems, or to ensure such technologies are accessible to a broad international coalition. Given the significant cost of developing advanced AI systems, a Frontier AI Collaborative could take the form of an international private-public partnership that leverages existing technology and capacity in industry, for example by contracting access to or funding innovation in appropriate AI technology from frontier AI developers. Such an organization could draw inspiration from international public-private partnerships like Gavi - the Vaccine Alliance or The Global Fund to Fight AIDS, Tuberculosis and Malaria; as well as international organizations that hold and control powerful technologies, like the IAEA’s nuclear fuel bankor the Atomic Development Authority that was proposed following WWII.
Functions and Motivation
A Frontier AI Collaborative could be designed to spread beneficial technology or serve as a channel for legitimate international access to advanced AI.
Spreading beneficial technology: A Collaborative could be established to ensure the benefits of cutting-edge AI reach groups that are otherwise underserved by AI development. One motivation for this objective is that the resources required to develop advanced systems make their development unavailable to many societies. This may result in technologies being inadequately designed for and supplied to groups that may benefit most from them for a variety of reasons:
enumi.
Systems developed by private actors may not adequately cater to all societies or demographics: they may not reflect the right values, have the right language capabilities, or work efficiently in diverse geographies.
Private firms may not price their products in ways that allow for equitable or broad distribution of benefits.
In order to protect proprietary information, private AI firms may not grant deep access to their models (e.g. they may restrict API access to prevent model imitation), which could preclude the development of use cases with significant social benefit.
A Collaborative could acquire or develop and then distribute AI systems to address these gaps, pooling resources from member states and international development programs, working with frontier AI labs to provide appropriate technology, and partnering with local businesses, NGOs, and beneficiary governments to better understand technological needs and overcome barriers to use.[For example, Gavi promotes immunization e.g. by funding innovation, and negotiating bulk contracts with pharmaceutical companies (especially advanced market commitments) for vaccination programs in low-income countries.] It could enable the development of technology that better caters to the underserved,price access to AI models in a way that is equitable, provide education and build infrastructure to allow the effective utilization of AI technology, and set a paradigm for responsible and inclusive AI development. By pooling the resources of multiple parties towards these ends (including safety talent, which is currently very scarce in the AI community), one or more of the aims could potentially be pursued more quickly and effectively than under the status quo.
Facilitating legitimate international access to powerful AI: More speculatively, a sufficiently ambitious, responsible and legitimately governed AI Collaborative could further AI governance objectives and reduce geopolitical instability amidst fierce AI competition among states. For example, membership in a Collaborative and access to its safe technology could be offered as an incentive for countries to participate in a governance regime that enforces responsibility (such as agreements to enact stricter regulation, or restrict military AI development). The existence of a technologically empowered neutral coalition may also mitigate the destabilizing effects of an AI race between states, by reducing the strategic consequences of one party falling behind or moderating the power concentrated among competing powers.
In addition, the Collaborative’s technology could be used to increase global resilience to misused or misaligned AI systems by giving experts a head start in studying the kinds of threats likely to be posed by other AI systems, and by being deployed for “protective” purposes such as fixing security vulnerabilities in critical infrastructure, detecting and counteracting disinformation campaigns, identifying misuse or failures of deployed systems, or monitoring compliance with AI regulations. This would be especially useful in scenarios where sharply falling training costs (due to algorithmic progress and Moore’s law) means the ability to train dangerous models is widely spread.
Challenges and Risks
Obstacles to benefiting from AI access: It is likely to be difficult to meaningfully empower underserved populations with AI technology, as the obstacles to their benefiting from AI run much deeper than access alone. Any Collaborative whose primary objective is global benefit needs to be adequately integrated into the global development ecosystem and set up with significant capacity or partnerships for activities beyond AI development such as: understanding the needs of member countries, building absorptive capacity through education and infrastructure, and supporting the development of a local commercial ecosystem to make use of the technology. The resources required to overcome these obstacles is likely to be substantial, and it is unclear whether such a Collaborative would be an effective means of promoting development.
Diffusion of dual-use technologies: Another challenge for the Collaborative would be managing the risk of diffusing dangerous technologies. On the one hand, in order to fulfill its objectives, the Collaborative would need to significantly promote access to the benefits of advanced AI (objective 1), or put control of cutting-edge AI technology in the hands of a broad coalition (objective 2). On the other hand, it may be difficult to do this without diffusing dangerous AI technologies around the world, if the most powerful AI systems are general purpose, dual-use, and proliferate easily.[For example: it may be difficult to protect broadly-deployed models from imitation, and it may be difficult to secure the deployment pipeline from attempts to copy model weights.] This is especially the case if the Collaborative aims to deploy cutting-edge general purpose systems to manage AI risks: the kinds of systems (and their underlying source code and algorithms) capable of meaningfully protecting against dangerous AI or furthering governance objectives may pose an exceptional misuse risk, as they will likely be engineered from highly capable, general purpose models.
To address such a challenge, it would be important for the Collaborative to have a clear mandate and purpose. Members of a Collaborative would need to have a strong understanding of the risks of the models being developed now and in the future, and their implications for model distribution, organization security (especially restrictions on the movement of Collaborative model weights), and other activities that may impact their ability to benefit from the Collaborative. Only by doing this would the Collaborative be able to consistently implement the necessary controls to manage frontier systems. It may also be necessary to exclude from participation states who are likely to want to use AI technology in non-peaceful ways, or make participation in a governance regime the precondition for membership.
Assessment
A Frontier AI Collaborative may indeed be a viable way of spreading AI benefits. However, the significant obstacles to societies benefiting from AI access raise questions about its competitiveness (relative to other development initiatives) as a means of promoting the welfare of underserved communities.
The viability of a Collaborative as a site of legitimately controlled technology also unclear: it depends on whether a balance can be struck between legitimately pursuing technologies powerful enough to positively affect international stability, and managing the proliferation of dangerous systems.
AI Safety Project: Conducting Technical Safety Research
The final model we discuss is an international collaboration to conduct technical AI safety research[This could include work on understanding and evaluating characteristics of systems such as alignment/reliability and dangerous capabilities, training methods to reduce and manage these characteristics, and deployment protocols (such as system security, monitoring, accident-response) that are appropriate to different system characteristics.] at an ambitious scale.[See, e.g.,
Tthe Safety Project would be modeled after large-scale scientific collaborations like ITER and CERN. Concretely, it would be an institution with significant compute, engineering capacity and access to models (obtained via agreements with leading AI developers), and would recruit the world’s leading experts in AI, AI safety and other relevant fields to work collaboratively on how to engineer and deploy advanced AI systems such that they are reliable and less able to be misused. CERN and ITER are intergovernmental collaborations; we note that an AI Safety Project need not be, and should be organized to benefit from the AI Safety expertise in civil society and the private sector.
Functions and Motivation
The motivation behind an international Safety Project would be to accelerate AI safety research by increasing its scale, resourcing and coordination, thereby expanding the ways in which AI can be safely deployed, and mitigating risks stemming from powerful general purpose capabilities.[Being a public good, AI safety may be underfunded by default, which the Safety Project would address as a site of collective contribution.] Researchers—including those who would not otherwise be working on AI safety—could be drawn by its international stature and enabled by the project’s exceptional compute, engineers and model access. The Project would become a vibrant research community that benefits from tighter information flows and a collective focus on AI safety. The Project should also have exceptional leaders and governance structures that ensure its efforts are most effectively targeted at critical questions on the path to safer AI systems.
Because perceptions of AI risk vary around the world, such an effort would likely be spearheaded by frontier risk-conscious actors like the US and UK governments, AGI labs and civil society groups. In the long run, it would be important for membership to be broad to ensure its research is recognized and informs AI development and deployment around the world.[While safety-relevant insights should be publicized for international use, other innovations with commercial value can be collectively owned by or affordably licensed to member states to incentivize broad participation. See, e.g., CERN’s approach to this.]
Challenges and Risks
Pulling safety research away from frontier developers: One potential effect of this model is that it diverts safety research away from the sites of frontier AI development. It is possible that safety research is best conducted in close proximity to AI development to deepen safety researchers’ understanding of the processes and systems they are trying to make safe and to ensure there is adequate safety expertise in-house. This risk could be addressed by offering safety researchers within AI labs dual appointments or advisory roles in the Project, and may become less of an issue if the practice of AI safety becomes institutionalized and separated from research and development.
Security concerns and model access: In order to be effective, participants in the Project need to have access to advanced AI models, which may allow them to illegally copy the model’s weights, clone the model via access to its outputs, or understand how it could be replicated (by determining its architecture or training process). Given the importance of these assets to the business interests of frontier labs, it may be difficult to negotiate agreements where adequate model access is granted. It may also lead to the diffusion of dangerous technologies.
This issue may be addressed by restricting membership in the Safety Project and by information security measures. In particular, it may be possible to silo information, structure model access, and design internal review processes in such a way that meaningfully reduces this risk while ensuring research results are subject to adequate scientific scrutiny. Certain types of research, such as the development of model evaluations and red-teaming protocols, can proceed effectively with API access to the models, while others such as mechanistic interpretability will require access to the model weights and architectures, but may not need to work with the latest (and therefore most sensitive) models.
Assessment
Technical progress on how to increase the reliability of advanced AI systems and protect them from misuse will likely be a priority in AI governance. It remains to be seen whether—due to issues of model access and the allocation of experts between a Safety Project and sites of frontier AI development—an AI Safety Project will be the most effective way of pursuing this goal.
Combining Institutional Functions
We can imagine institutions taking on the role of several of the models above. For example, the Commission on Frontier AI and the AI Safety Project make an obvious pairing: a Commission could scale up research functions to supplement the synthesis and consensus-building efforts, or a Project could conduct synthesis work in the course of its activities and gradually take on a consensus-establishing role. A Frontier AI Collaborative would also likely conduct safety research, and could easily absorb additional resourcing to become a world-leading Safety Project.
Conclusion
This paper has outlined several reasons why the world may want to expand existing initiatives in AI governance and safety and discussed the strengths and limitations of four possible institutional models to address these needs.
To better harness advanced AI for global benefit, international efforts to help underserved societies access and use advanced AI systems will be important. A Frontier AI Collaborative that acquires and distributes AI systems could be helpful, if it can effectively enable underserved groups to take full advantage of such systems. A Commission on Frontier AI could help identify the areas where international efforts can most effectively achieve these goals, if it can prevent the politicization of its work. Relatedly, it will be important for governance approaches around the world to be coordinated, so as to reduce frictions to innovation and access: an Advanced AI Governance Organization that sets international standards for governance of the most advanced models could facilitate this.
To manage global risks from powerful AI systems, effective AI governance regimes may be needed around the world. An Advanced AI Governance Organization that establishes governance frameworks for managing global threats from advanced systems and helps with their implementation may help internationalize effective regulatory measures, but it may be difficult to establish reliable standards if AI progress continues at the present rate, and also difficult to incentivize adoption of an Organization’s standards if there is a lack of global consensus on AI risks. A Commission on Frontier AI could also support governance efforts by building scientific consensus around AI risks and their mitigation, although its task of providing a scientifically credible and internationally recognized account of a quickly changing risk landscape will be challenging, especially given the relative lack of existing scientific research on the topic. An AI Safety Project could accelerate the rate at which technical methods of mitigating AI risks are developed—provided it can overcome its efficiency and model access hurdles, and a Frontier AI Collaborative’s technology might be used to increase global resilience to misused or misaligned AI systems. More speculatively, the functions of a Governance Organization and Collaborative could in some cases counteract the geopolitical factors exacerbating AI risks.
The taxonomy of functions we have presented is not exhaustive, nor do we argue that our institutional grouping is the most promising. Given the immense global opportunities and challenges presented by AI systems that may be on the horizon, the topic of international institutions for AI governance demands much greater analytical and practical attention.
Acknowledgements
We are grateful to the following people for discussion and input: Michael Aird, Jeff Alstott, Jon Bateman, Alexandra Belias, Dorothy Chou, Jack Clark, Lukas Finnveden, Iason Gabriel, Ben Garfinkel, Erich Grunewald, Oliver Guest, Jackie Kay, Noam Kolt, Sebastien Krier, Lucy Lim, Nicklas Lundblad, Stewart Patrick, George Perkovich, Toby Shevlane, Kent Walker and Ankur Vora. We would also like to thank participants of the September 2022 and June 2023 IGOTAI Seminars, in which early work was discussed.
Bibliography
1@misc{rees_g20_2023,
2 file = {Snapshot:/Users/lewisho/Zotero/storage/6FPNC3R3/g20-must-set-up-an-international-panel-on-technological-change-101679237287848.html:text/html},
3 year = {2023},
4 month = {March},
5 author = {Rees, Martin and Sondhi, Shivaji and VijayRaghavan, Krishnaswamy},
6 howpublished = {Hindustan Times},
7 urldate = {2023-07-08},
8 language = {en},
9 abstract = {The panel’s mandate will be to assess where we are and where we are headed in the future when it comes to “post-human technology”. Unlike IPCC, which deals with much longer time scales, IPTC will need to work much faster and nimbly. It seems to us that the G20 is the right-sized organisation for chartering such a panel and enabling its functioning.},
10 url = {https://www.hindustantimes.com/opinion/g20-must-set-up-an-international-panel-on-technological-change-101679237287848.html},
11 title = {G20 must set up an international panel on technological change},
12}
13
14@article{chowdhury_ai_2023,
15 file = {Snapshot:/Users/lewisho/Zotero/storage/PEE83TB7/ai-desperately-needs-global-oversight.html:text/html},
16 keywords = {artificial intelligence, ethics, government, tech policy and law},
17 note = {Section: tags},
18 year = {2023},
19 author = {Chowdhury, Rumman},
20 journal = {Wired},
21 urldate = {2023-07-09},
22 language = {en-US},
23 abstract = {As ChatGPT and its ilk continue to spread, countries need an independent board to hold AI companies accountable and limit harms.},
24 url = {https://www.wired.com/story/ai-desperately-needs-global-oversight/},
25 issn = {1059-1028},
26 title = {{AI} {Desperately} {Needs} {Global} {Oversight}},
27}
28
29@misc{kakkad_new_2023,
30 file = {Snapshot:/Users/lewisho/Zotero/storage/2XY78J6T/new-national-purpose-innovation-can-power-future-britain.html:text/html},
31 year = {2023},
32 author = {Kakkad, Jeegar and Macon-Cooney, Benedict and Northend, Jess and Phillips, James and Rajkumar, Nitarshan and Stanley, Luke and Westgarth, Tom},
33 urldate = {2023-07-09},
34 language = {en-GB},
35 abstract = {A New National Purpose: Innovation Can Power the Future of Britain},
36 url = {https://www.institute.global/insights/politics-and-governance/new-national-purpose-innovation-can-power-future-britain},
37 shorttitle = {A {New} {National} {Purpose}},
38 title = {A {New} {National} {Purpose}: {Innovation} {Can} {Power} the {Future} of {Britain}},
39}
40
41@misc{dubner_satya_nodate,
42 file = {Snapshot:/Users/lewisho/Zotero/storage/U74DAPBC/satya-nadellas-intelligence-is-not-artificial.html:text/html},
43 author = {Dubner, Stephen},
44 urldate = {2023-07-09},
45 language = {en},
46 abstract = {Satya Nadella’s Intelligence Is Not Artificial - Freakonomics},
47 url = {https://freakonomics.com/podcast/satya-nadellas-intelligence-is-not-artificial/},
48 title = {Satya {Nadella}’s {Intelligence} {Is} {Not} {Artificial}},
49}
50
51@misc{noauthor_secretary-general_2023,
52 year = {2023},
53 month = {June},
54 howpublished = {United Nations},
55 key = {un},
56 urldate = {2023-07-09},
57 url = {https://press.un.org/en/2023/sgsm21832.doc.htm},
58 title = {Secretary-{General} {Urges} {Broad} {Engagement} from {All} {Stakeholders} towards {United} {Nations} {Code} of {Conduct} for {Information} {Integrity} on {Digital} {Platforms} {\textbar} {UN} {Press}},
59}
60
61@misc{noauthor_elders_2023,
62 year = {2023},
63 month = {May},
64 howpublished = {The Elders},
65 urldate = {2023-07-09},
66 key = {elders},
67 language = {en},
68 abstract = {The Elders today call on world leaders to work together urgently on the design of strong international governance, to allow all humanity to take advantage of the opportunities of Artificial Intelligence (AI), while limiting the enormous risks.},
69 url = {https://theelders.org/news/elders-urge-global-co-operation-manage-risks-and-share-benefits-ai},
70 title = {The {Elders} urge global co-operation to manage risks and share benefits of {AI}},
71}
72
73@article{awokuse_stronger_2010,
74 file = {Full Text PDF:/Users/lewisho/Zotero/storage/4IWASB5L/Awokuse and Yin - 2008 - Do Stronger Intellectual Property Rights Protectio.pdf:application/pdf},
75 year = {2010},
76 author = {Awokuse, Titus and Yin, Hong},
77 journal = {World Development},
78 number = {8},
79 abstract = {Most of the previous studies on the effect of IPR protection on international trade have been from the perspective of major industrialized nations. However, much of the current debate on the effects of IPR protection involves large developing countries with high threat of imitation. This study contributes to the literature by analyzing the impact of the strengthening of patent laws in China on its bilateral trade flows. We estimate the effects of patent rights protection on China’s imports at the aggregate and detailed product categories for both OECD (developed) and non-OECD (developing) countries. The empirical results suggest that increased patent rights protection stimulate China’s imports, particularly in the knowledge-intensive product categories. Furthermore, while the evidence in support of the market expansion effect is significant for imports from OECD countries, it is rather weak and mostly insignificant for imports from non-OECD countries.},
80 shorttitle = {Do {Stronger} {Intellectual} {Property} {Rights} {Protection} {Induce} {More} {Bilateral} {Trade}?},
81 volume = {38},
82 title = {Do {Stronger} {Intellectual} {Property} {Rights} {Protection} {Induce} {More} {Bilateral} {Trade}? {Evidence} from {China}'s {Imports}},
83}
84
85@article{vinuesa_role_2020,
86 file = {Full Text PDF:/Users/lewisho/Zotero/storage/N6W8NI5A/Vinuesa et al. - 2020 - The role of artificial intelligence in achieving t.pdf:application/pdf},
87 pages = {233},
88 keywords = {Computational science, Developing world, Energy efficiency},
89 note = {Number: 1
90Publisher: Nature Publishing Group},
91 year = {2020},
92 month = {January},
93 author = {Vinuesa, Ricardo and Azizpour, Hossein and Leite, Iolanda and Balaam, Madeline and Dignum, Virginia and Domisch, Sami and Felländer, Anna and Langhans, Simone Daniela and Tegmark, Max and Fuso Nerini, Francesco},
94 journal = {Nature Communications},
95 urldate = {2023-07-09},
96 number = {1},
97 language = {en},
98 abstract = {The emergence of artificial intelligence (AI) and its progressively wider impact on many sectors requires an assessment of its effect on the achievement of the Sustainable Development Goals. Using a consensus-based expert elicitation process, we find that AI can enable the accomplishment of 134 targets across all the goals, but it may also inhibit 59 targets. However, current research foci overlook important aspects. The fast development of AI needs to be supported by the necessary regulatory insight and oversight for AI-based technologies to enable sustainable development. Failure to do so could result in gaps in transparency, safety, and ethical standards.},
99 doi = {10.1038/s41467-019-14108-y},
100 url = {https://www.nature.com/articles/s41467-019-14108-y},
101 issn = {2041-1723},
102 copyright = {2020 The Author(s)},
103 volume = {11},
104 title = {The role of artificial intelligence in achieving the {Sustainable} {Development} {Goals}},
105}
106
107@article{urbina_dual_2022,
108 file = {Accepted Version:/Users/lewisho/Zotero/storage/23RFNGTS/Urbina et al. - 2022 - Dual use of artificial-intelligence-powered drug d.pdf:application/pdf},
109 pages = {189--191},
110 keywords = {Cheminformatics, Drug safety, Ethics, Software, Toxicology},
111 note = {Number: 3
112Publisher: Nature Publishing Group},
113 year = {2022},
114 month = {March},
115 author = {Urbina, Fabio and Lentzos, Filippa and Invernizzi, Cédric and Ekins, Sean},
116 journal = {Nature Machine Intelligence},
117 urldate = {2023-07-09},
118 number = {3},
119 language = {en},
120 abstract = {An international security conference explored how artificial intelligence (AI) technologies for drug discovery could be misused for de novo design of biochemical weapons. A thought experiment evolved into a computational proof.},
121 doi = {10.1038/s42256-022-00465-9},
122 url = {https://www.nature.com/articles/s42256-022-00465-9},
123 issn = {2522-5839},
124 copyright = {2022 Springer Nature Limited},
125 volume = {4},
126 title = {Dual use of artificial-intelligence-powered drug discovery},
127}
128
129@article{jumper_highly_2021,
130 file = {Full Text PDF:/Users/lewisho/Zotero/storage/5WYYQ9XJ/Jumper et al. - 2021 - Highly accurate protein structure prediction with .pdf:application/pdf},
131 pages = {583--589},
132 keywords = {Computational biophysics, Machine learning, Protein structure predictions, Structural biology},
133 note = {Number: 7873
134Publisher: Nature Publishing Group},
135 year = {2021},
136 month = {August},
137 author = {Jumper, John and Evans, Richard and Pritzel, Alexander and Green, Tim and Figurnov, Michael and Ronneberger, Olaf and Tunyasuvunakool, Kathryn and Bates, Russ and Žídek, Augustin and Potapenko, Anna and Bridgland, Alex and Meyer, Clemens and Kohl, Simon A. A. and Ballard, Andrew J. and Cowie, Andrew and Romera-Paredes, Bernardino and Nikolov, Stanislav and Jain, Rishub and Adler, Jonas and Back, Trevor and Petersen, Stig and Reiman, David and Clancy, Ellen and Zielinski, Michal and Steinegger, Martin and Pacholska, Michalina and Berghammer, Tamas and Bodenstein, Sebastian and Silver, David and Vinyals, Oriol and Senior, Andrew W. and Kavukcuoglu, Koray and Kohli, Pushmeet and Hassabis, Demis},
138 journal = {Nature},
139 urldate = {2023-07-09},
140 number = {7873},
141 language = {en},
142 abstract = {Proteins are essential to life, and understanding their structure can facilitate a mechanistic understanding of their function. Through an enormous experimental effort1–4, the structures of around 100,000 unique proteins have been determined5, but this represents a small fraction of the billions of known protein sequences6,7. Structural coverage is bottlenecked by the months to years of painstaking effort required to determine a single protein structure. Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics. Predicting the three-dimensional structure that a protein will adopt based solely on its amino acid sequence—the structure prediction component of the ‘protein folding problem’8—has been an important open research problem for more than 50 years9. Despite recent progress10–14, existing methods fall far short of atomic accuracy, especially when no homologous structure is available. Here we provide the first computational method that can regularly predict protein structures with atomic accuracy even in cases in which no similar structure is known. We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14)15, demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods. Underpinning the latest version of AlphaFold is a novel machine learning approach that incorporates physical and biological knowledge about protein structure, leveraging multi-sequence alignments, into the design of the deep learning algorithm.},
143 doi = {10.1038/s41586-021-03819-2},
144 url = {https://www.nature.com/articles/s41586-021-03819-2},
145 issn = {1476-4687},
146 copyright = {2021 The Author(s)},
147 volume = {596},
148 title = {Highly accurate protein structure prediction with {AlphaFold}},
149}
150
151@misc{noauthor_snyk_nodate,
152 file = {Snapshot:/Users/lewisho/Zotero/storage/CJJSVS4I/snyk.io.html:text/html},
153 howpublished = {Snyk},
154 urldate = {2023-07-09},
155 language = {en-US},
156 abstract = {Snyk helps software-driven businesses develop fast and stay secure. Continuously find and fix vulnerabilities for npm, Maven, NuGet, RubyGems, PyPI and more.},
157 url = {https://snyk.io/},
158 title = {Snyk},
159}
160
161@misc{noauthor_codeql_nodate,
162 file = {CodeQL:/Users/lewisho/Zotero/storage/9S7YEZND/codeql.github.com.html:text/html},
163 urldate = {2023-07-09},
164 url = {https://codeql.github.com/},
165 title = {{CodeQL}},
166}
167
168@misc{steinhart_what_2023,
169 file = {Snapshot:/Users/lewisho/Zotero/storage/4RF6YGSP/what-will-gpt-2030-look-like.html:text/html},
170 year = {2023},
171 month = {June},
172 author = {Steinhart, Jacob},
173 howpublished = {Bounded Regret},
174 urldate = {2023-07-09},
175 language = {en},
176 abstract = {GPT-4 surprised many people with its abilities at coding, creative brainstorming, letter-writing, and other skills. How can we be less surprised by developments in machine learning? In this post, I’ll forecast the properties of large pretrained ML systems in 2030.},
177 url = {https://bounded-regret.ghost.io/what-will-gpt-2030-look-like/},
178 title = {What will {GPT}-2030 look like?},
179}
180
181@article{bond_fake_2023,
182 file = {Snapshot:/Users/lewisho/Zotero/storage/NZFWWSE4/fake-viral-images-of-an-explosion-at-the-pentagon-were-probably-created-by-ai.html:text/html},
183 year = {2023},
184 month = {May},
185 author = {Bond, Shannon},
186 journal = {NPR},
187 urldate = {2023-07-09},
188 language = {en},
189 abstract = {Authorities quickly confirmed that no explosion had taken place but the faked images spread on Twitter for a short time. The incident briefly sent the stock market lower.},
190 url = {https://www.npr.org/2023/05/22/1177590231/fake-viral-images-of-an-explosion-at-the-pentagon-were-probably-created-by-ai},
191 title = {Fake viral images of an explosion at the {Pentagon} were probably created by {AI}},
192 chapter = {Untangling Disinformation},
193}
194
195@misc{goldstein_generative_2023,
196 file = {arXiv Fulltext PDF:/Users/lewisho/Zotero/storage/F6H62PNI/Goldstein et al. - 2023 - Generative Language Models and Automated Influence.pdf:application/pdf;arXiv.org Snapshot:/Users/lewisho/Zotero/storage/PJJZ2H2A/2301.html:text/html},
197 annote = {Comment: 82 pages, 26 figures},
198 keywords = {Computer Science - Computers and Society},
199 note = {arXiv:2301.04246 [cs]},
200 year = {2023},
201 month = {January},
202 author = {Goldstein, Josh A. and Sastry, Girish and Musser, Micah and DiResta, Renee and Gentzel, Matthew and Sedova, Katerina},
203 publisher = {arXiv},
204 urldate = {2023-07-09},
205 abstract = {Generative language models have improved drastically, and can now produce realistic text outputs that are difficult to distinguish from human-written content. For malicious actors, these language models bring the promise of automating the creation of convincing and misleading text for use in influence operations. This report assesses how language models might change influence operations in the future, and what steps can be taken to mitigate this threat. We lay out possible changes to the actors, behaviors, and content of online influence operations, and provide a framework for stages of the language model-to-influence operations pipeline that mitigations could target (model construction, model access, content dissemination, and belief formation). While no reasonable mitigation can be expected to fully prevent the threat of AI-enabled influence operations, a combination of multiple mitigations may make an important difference.},
206 doi = {10.48550/arXiv.2301.04246},
207 url = {http://arxiv.org/abs/2301.04246},
208 shorttitle = {Generative {Language} {Models} and {Automated} {Influence} {Operations}},
209 title = {Generative {Language} {Models} and {Automated} {Influence} {Operations}: {Emerging} {Threats} and {Potential} {Mitigations}},
210}
211
212@misc{hendrycks_unsolved_2022,
213 file = {arXiv.org Snapshot:/Users/lewisho/Zotero/storage/CE4B3F6B/2109.html:text/html;Full Text PDF:/Users/lewisho/Zotero/storage/VHVSEEQY/Hendrycks et al. - 2022 - Unsolved Problems in ML Safety.pdf:application/pdf},
214 annote = {Comment: Position Paper},
215 keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning},
216 note = {arXiv:2109.13916 [cs]},
217 year = {2022},
218 month = {June},
219 author = {Hendrycks, Dan and Carlini, Nicholas and Schulman, John and Steinhardt, Jacob},
220 publisher = {arXiv},
221 urldate = {2023-07-09},
222 abstract = {Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technologies, safety for ML should be a leading research priority. In response to emerging safety challenges in ML, such as those introduced by recent large-scale models, we provide a new roadmap for ML Safety and refine the technical problems that the field needs to address. We present four problems ready for research, namely withstanding hazards ("Robustness"), identifying hazards ("Monitoring"), reducing inherent model hazards ("Alignment"), and reducing systemic hazards ("Systemic Safety"). Throughout, we clarify each problem's motivation and provide concrete research directions.},
223 url = {http://arxiv.org/abs/2109.13916},
224 title = {Unsolved {Problems} in {ML} {Safety}},
225}
226
227@misc{amodei_concrete_2016,
228 file = {arXiv.org Snapshot:/Users/lewisho/Zotero/storage/SCY6S6WG/1606.html:text/html;Full Text PDF:/Users/lewisho/Zotero/storage/EHY55AQR/Amodei et al. - 2016 - Concrete Problems in AI Safety.pdf:application/pdf},
229 annote = {Comment: 29 pages},
230 keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
231 note = {arXiv:1606.06565 [cs]},
232 year = {2016},
233 month = {July},
234 author = {Amodei, Dario and Olah, Chris and Steinhardt, Jacob and Christiano, Paul and Schulman, John and Mané, Dan},
235 publisher = {arXiv},
236 urldate = {2023-07-09},
237 abstract = {Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society. In this paper we discuss one such potential impact: the problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emerge from poor design of real-world AI systems. We present a list of five practical research problems related to accident risk, categorized according to whether the problem originates from having the wrong objective function ("avoiding side effects" and "avoiding reward hacking"), an objective function that is too expensive to evaluate frequently ("scalable supervision"), or undesirable behavior during the learning process ("safe exploration" and "distributional shift"). We review previous work in these areas as well as suggesting research directions with a focus on relevance to cutting-edge AI systems. Finally, we consider the high-level question of how to think most productively about the safety of forward-looking applications of AI.},
238 url = {http://arxiv.org/abs/1606.06565},
239 title = {Concrete {Problems} in {AI} {Safety}},
240}
241
242@misc{arnold_ai_2021,
243 file = {Snapshot:/Users/lewisho/Zotero/storage/94NYZHQX/ai-accidents-an-emerging-threat.html:text/html},
244 year = {2021},
245 author = {Arnold, Zachary and Toner, Helen},
246 howpublished = {Center for Security and Emerging Technology},
247 urldate = {2023-07-09},
248 language = {en-US},
249 abstract = {As modern machine learning systems become more widely used, the potential costs of malfunctions grow. This policy brief describes how trends we already see today—both in newly deployed artificial intelligence systems and in older technologies—show how damaging the AI accidents of the future could be. It describes a wide range of hypothetical but realistic scenarios to illustrate the risks of AI accidents and offers concrete policy suggestions to reduce these risks.},
250 url = {https://cset.georgetown.edu/publication/ai-accidents-an-emerging-threat/},
251 shorttitle = {{AI} {Accidents}},
252 title = {{AI} {Accidents}: {An} {Emerging} {Threat}},
253}
254
255@misc{noauthor_statement_2023,
256 file = {Snapshot:/Users/lewisho/Zotero/storage/3V388YAB/statement-on-ai-risk.html:text/html},
257 year = {2023},
258 howpublished = {Center for AI Safety},
259 key = {center},
260 urldate = {2023-07-09},
261 abstract = {A statement jointly signed by a historic coalition of experts: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”},
262 url = {https://www.safe.ai/statement-on-ai-risk},
263 title = {Statement on {AI} {Risk}},
264}
265
266@misc{hobbhahn_trends_2022,
267 file = {Snapshot:/Users/lewisho/Zotero/storage/H2IN66TH/trends-in-gpu-price-performance.html:text/html},
268 year = {2022},
269 author = {Hobbhahn, Marius},
270 howpublished = {Epoch},
271 urldate = {2023-07-09},
272 language = {en},
273 abstract = {Using a dataset of 470 models of graphics processing units released between 2006 and 2021, we find that the amount of floating-point operations/second per \$ doubles every {\textasciitilde}2.5 years.},
274 url = {https://epochai.org/blog/trends-in-gpu-price-performance},
275 title = {Trends in {GPU} price-performance},
276}
277
278@misc{erdil_algorithmic_2023,
279 file = {Snapshot:/Users/lewisho/Zotero/storage/B8V3SDHB/revisiting-algorithmic-progress.html:text/html},
280 year = {2023},
281 author = {Erdil, Ege and Besiroglu, Tamay},
282 publisher = {arXiv},
283 urldate = {2023-07-09},
284 language = {en},
285 abstract = {We use a dataset of over a hundred computer vision models from the last decade to investigate how better algorithms and architectures have enabled researchers to use compute and data more efficiently. We find that every 9 months, the introduction of better algorithms contribute the equivalent of a doubling of compute budgets.},
286 url = {https://arxiv.org/abs/2212.05153},
287 title = {Algorithmic {Progress} in {Computer} {Vision}},
288}
289
290@article{toner_illusion_2023,
291 file = {Snapshot:/Users/lewisho/Zotero/storage/RYPKNIHC/illusion-chinas-ai-prowess-regulation.html:text/html},
292 keywords = {Artificial Intelligence, Automation, China, East Asia, Law, Politics \& Society, Science \& Technology, United States},
293 year = {2023},
294 month = {June},
295 author = {Toner, Helen and Xiao, Jenny and Ding, Jeffrey},
296 journal = {Foreign Affairs},
297 urldate = {2023-07-09},
298 language = {en-US},
299 abstract = {Regulating AI will not set America back in the technology race.},
300 url = {https://www.foreignaffairs.com/china/illusion-chinas-ai-prowess-regulation},
301 issn = {0015-7120},
302 title = {The {Illusion} of {China}’s {AI} {Prowess}},
303}
304
305@misc{noauthor_netherlands_2023,
306 file = {Snapshot:/Users/lewisho/Zotero/storage/DNLPEWUY/the-netherlands-joins-the-us-in-restricting-semiconductor-exports-to-china.html:text/html},
307 year = {2023},
308 month = {March},
309 howpublished = {Allen Overy},
310 urldate = {2023-07-09},
311 key = {allen},
312 language = {en},
313 abstract = {In January 2023, the Netherlands and Japan agreed with the United States to impose controls on the export of certain semiconductors and related products to China. This followed a push by the Biden Administration to ensure the effectiveness of related U.S. export controls that were introduced in October 2022.},
314 url = {https://www.allenovery.com/en-gb/global/news-and-insights/publications/the-netherlands-joins-the-us-in-restricting-semiconductor-exports-to-china},
315 title = {The {Netherlands} joins the {U}.{S}. in restricting semiconductor exports to {China}},
316}
317
318@misc{noauthor_pm_2023,
319 file = {Snapshot:/Users/lewisho/Zotero/storage/NVM2G3KI/pm-urges-tech-leaders-to-grasp-generational-opportunities-and-challenges-of-ai.html:text/html},
320 year = {2023},
321 key = {govuk},
322 howpublished = {GOV.UK},
323 urldate = {2023-07-09},
324 language = {en},
325 abstract = {The UK must act quickly if we want to retain our position as one of the world’s tech capitals, Prime Minister Rishi Sunak will tell tech leaders today [Monday 12 June].},
326 url = {https://www.gov.uk/government/news/pm-urges-tech-leaders-to-grasp-generational-opportunities-and-challenges-of-ai},
327 title = {{PM} urges tech leaders to grasp generational opportunities and challenges of {AI}},
328}
329
330@misc{singh_anthropics_2023,
331 year = {2023},
332 month = {April},
333 author = {Singh, Devin Coldewey {and} Manish, Kyle Wiggers},
334 howpublished = {TechCrunch},
335 urldate = {2023-07-09},
336 language = {en-US},
337 abstract = {AI research startup Anthropic aims to raise as much as \$5 billion over the next two years to take on rival OpenAI.},
338 url = {https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-to-take-on-openai/},
339 title = {Anthropic's \${5B}, 4-year plan to take on {OpenAI}},
340}
341
342@article{veale_ai_2023,
343 file = {Veale et al. - 2023 - AI and Global Governance Modalities, Rationales, .pdf:/Users/lewisho/Zotero/storage/4E2INQF9/Veale et al. - 2023 - AI and Global Governance Modalities, Rationales, .pdf:application/pdf},
344 year = {2023},
345 author = {Veale, Michael and Matus, Kira and Gorwa, Robert},
346 journal = {Annual Review of Law and Social Science},
347 language = {en},
348 volume = {19},
349 title = {{AI} and {Global} {Governance}: {Modalities}, {Rationales}, {Tensions}},
350}
351
352@misc{noauthor_preparing_nodate,
353 file = {Snapshot:/Users/lewisho/Zotero/storage/NDQZCNMN/preparingreports.html:text/html},
354 urldate = {2023-07-09},
355 url = {https://www.ipcc.ch/about/preparingreports/},
356 title = {Preparing {Reports} — {IPCC}},
357}
358
359@article{shaw_relevant_2004,
360 file = {Full Text PDF:/Users/lewisho/Zotero/storage/XV3TGF7A/Shaw and Robinson - 2004 - Relevant But Not Prescriptive Science Policy Mode.pdf:application/pdf},
361 pages = {84--95},
362 year = {2004},
363 month = {January},
364 author = {Shaw, Alison and Robinson, John},
365 journal = {Philosophy Today},
366 abstract = {The Intergovernmental Panel on Climate Change (IPCC) represents perhaps the largest example of ‘mandated science’ ever undertaken. It activities therefore raise a number of critical issues concerning the science/society interface. While previous studies of the IPCC have focused on the scientific credibility of its findings, this paper will examine the credibility of the process and protocols employed to assess “policy relevant but not policy prescriptive scientific information”. In particular we will examine two unique devices used in the IPCC: the Summary for Policymakers (SPM) and the Policy Relevant Scientific Questions (PRSQ). It will be argued that, despite unhappiness on the part of some of the scientific participants, the negotiation of meaning given rise to in these processes represents a credible and useful way to bridge the science/policy divide and in turn provides insights into ways to bridge the larger issues of the role of science in society.},
367 doi = {10.5840/philtoday200448Supplement9},
368 shorttitle = {Relevant {But} {Not} {Prescriptive}},
369 volume = {48},
370 title = {Relevant {But} {Not} {Prescriptive}: {Science} {Policy} {Models} within the {IPCC}},
371}
372
373@techreport{toivanen_significance_2017,
374 file = {Toivanen - 2017 - The Significance of Strategic Foresight in Verific.pdf:/Users/lewisho/Zotero/storage/3LNAJGHV/Toivanen - 2017 - The Significance of Strategic Foresight in Verific.pdf:application/pdf},
375 pages = {LLNL--TR--738786, 1502006, 892173},
376 doi = {10.2172/1502006},
377 year = {2017},
378 month = {September},
379 author = {Toivanen, Henrietta},
380 urldate = {2023-07-09},
381 number = {LLNL-TR--738786, 1502006, 892173},
382 language = {en},
383 url = {https://www.osti.gov/servlets/purl/1502006/},
384 shorttitle = {The {Significance} of {Strategic} {Foresight} in {Verification} {Technologies}},
385 title = {The {Significance} of {Strategic} {Foresight} in {Verification} {Technologies}: {A} {Case} {Study} of the {INF} {Treaty}},
386}
387
388@misc{shavit_what_2023,
389 file = {arXiv Fulltext PDF:/Users/lewisho/Zotero/storage/NDWX7245/Shavit - 2023 - What does it take to catch a Chinchilla Verifying.pdf:application/pdf;arXiv.org Snapshot:/Users/lewisho/Zotero/storage/M8DR2UAH/2303.html:text/html},
390 keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
391 note = {arXiv:2303.11341 [cs]},
392 year = {2023},
393 month = {May},
394 author = {Shavit, Yonadav},
395 publisher = {arXiv},
396 urldate = {2023-07-09},
397 abstract = {As advanced machine learning systems' capabilities begin to play a significant role in geopolitics and societal order, it may become imperative that (1) governments be able to enforce rules on the development of advanced ML systems within their borders, and (2) countries be able to verify each other's compliance with potential future international agreements on advanced ML development. This work analyzes one mechanism to achieve this, by monitoring the computing hardware used for large-scale NN training. The framework's primary goal is to provide governments high confidence that no actor uses large quantities of specialized ML chips to execute a training run in violation of agreed rules. At the same time, the system does not curtail the use of consumer computing devices, and maintains the privacy and confidentiality of ML practitioners' models, data, and hyperparameters. The system consists of interventions at three stages: (1) using on-chip firmware to occasionally save snapshots of the the neural network weights stored in device memory, in a form that an inspector could later retrieve; (2) saving sufficient information about each training run to prove to inspectors the details of the training run that had resulted in the snapshotted weights; and (3) monitoring the chip supply chain to ensure that no actor can avoid discovery by amassing a large quantity of un-tracked chips. The proposed design decomposes the ML training rule verification problem into a series of narrow technical challenges, including a new variant of the Proof-of-Learning problem [Jia et al. '21].},
398 doi = {10.48550/arXiv.2303.11341},
399 url = {http://arxiv.org/abs/2303.11341},
400 shorttitle = {What does it take to catch a {Chinchilla}?},
401 title = {What does it take to catch a {Chinchilla}? {Verifying} {Rules} on {Large}-{Scale} {Neural} {Network} {Training} via {Compute} {Monitoring}},
402}
403
404@misc{noauthor_memorandum_2008,
405 file = {House of Lords - European Union Committee - Written Evidence:/Users/lewisho/Zotero/storage/BSPYLD94/132we08.html:text/html},
406 year = {2008},
407 howpublished = {UK Parliament},
408 key = {UK},
409 urldate = {2023-07-09},
410 url = {https://publications.parliament.uk/pa/ld200809/ldselect/ldeucom/132/132we08.htm},
411 title = {Memorandum by the {Financial} {Action} {Task} {Force} ({FATF}) {Secretariat}},
412}
413
414@book{gallagher_politics_1999,
415 file = {The Politics of Verification - Nancy W. Gallagher - Google Books:/Users/lewisho/Zotero/storage/9VN3U3UR/The_Politics_of_Verification.html:text/html},
416 year = {1999},
417 author = {Gallagher, Nancy},
418 publisher = {Johns Hopkins University Press},
419 urldate = {2023-07-09},
420 title = {The {Politics} of {Verification}},
421}
422
423@article{spirling_why_2023,
424 file = {Full Text PDF:/Users/lewisho/Zotero/storage/G8HM658L/Spirling - 2023 - Why open-source generative AI models are an ethica.pdf:application/pdf;Snapshot:/Users/lewisho/Zotero/storage/ECXTQYTH/d41586-023-01295-4.html:text/html},
425 pages = {413--413},
426 keywords = {Ethics, Machine learning, Scientific community, Technology},
427 note = {Bandiera\_abtest: a
428Cg\_type: World View
429Number: 7957
430Publisher: Nature Publishing Group
431Subject\_term: Ethics, Machine learning, Technology, Scientific community},
432 year = {2023},
433 month = {April},
434 author = {Spirling, Arthur},
435 journal = {Nature},
436 urldate = {2023-07-09},
437 number = {7957},
438 language = {en},
439 abstract = {Researchers should avoid the lure of proprietary models and develop transparent large language models to ensure reproducibility.},
440 doi = {10.1038/d41586-023-01295-4},
441 url = {https://www.nature.com/articles/d41586-023-01295-4},
442 copyright = {2023 Springer Nature Limited},
443 volume = {616},
444 title = {Why open-source generative {AI} models are an ethical way forward for science},
445}
446
447@misc{rauf_atoms_2015,
448 file = {From ‘Atoms for Peace’ to an IAEA Nuclear Fuel Bank | Arms Control Association:/Users/lewisho/Zotero/storage/MRN8K829/‘atoms-peace’-iaea-nuclear-fuel-bank.html:text/html},
449 year = {2015},
450 month = {October},
451 author = {Rauf, Tariq},
452 howpublished = {Arms Control Association},
453 urldate = {2023-07-09},
454 url = {https://www.armscontrol.org/act/2015-10/features/%E2%80%98atoms-peace%E2%80%99-iaea-nuclear-fuel-bank},
455 title = {From ‘{Atoms} for {Peace}’ to an {IAEA} {Nuclear} {Fuel} {Bank} {\textbar} {Arms} {Control} {Association}},
456}
457
458@article{mohamed_decolonial_2020,
459 file = {Full Text PDF:/Users/lewisho/Zotero/storage/2WQWENKJ/Mohamed et al. - 2020 - Decolonial AI Decolonial Theory as Sociotechnical.pdf:application/pdf},
460 pages = {659--684},
461 keywords = {Affective community, Artificial intelligence, Coloniality, Critical technical practice, Decolonisation, Intercultural ethics, Sociotechnical foresight},
462 year = {2020},
463 month = {December},
464 author = {Mohamed, Shakir and Png, Marie-Therese and Isaac, William},
465 journal = {Philosophy \& Technology},
466 urldate = {2023-07-09},
467 number = {4},
468 language = {en},
469 abstract = {This paper explores the important role of critical science, and in particular of post-colonial and decolonial theories, in understanding and shaping the ongoing advances in artificial intelligence. Artificial intelligence (AI) is viewed as amongst the technological advances that will reshape modern societies and their relations. While the design and deployment of systems that continually adapt holds the promise of far-reaching positive change, they simultaneously pose significant risks, especially to already vulnerable peoples. Values and power are central to this discussion. Decolonial theories use historical hindsight to explain patterns of power that shape our intellectual, political, economic, and social world. By embedding a decolonial critical approach within its technical practice, AI communities can develop foresight and tactics that can better align research and technology development with established ethical principles, centring vulnerable peoples who continue to bear the brunt of negative impacts of innovation and scientific progress. We highlight problematic applications that are instances of coloniality, and using a decolonial lens, submit three tactics that can form a decolonial field of artificial intelligence: creating a critical technical practice of AI, seeking reverse tutelage and reverse pedagogies, and the renewal of affective and political communities. The years ahead will usher in a wave of new scientific breakthroughs and technologies driven by AI research, making it incumbent upon AI communities to strengthen the social contract through ethical foresight and the multiplicity of intellectual perspectives available to us, ultimately supporting future technologies that enable greater well-being, with the goal of beneficence and justice for all.},
470 doi = {10.1007/s13347-020-00405-8},
471 url = {https://doi.org/10.1007/s13347-020-00405-8},
472 shorttitle = {Decolonial {AI}},
473 issn = {2210-5441},
474 volume = {33},
475 title = {Decolonial {AI}: {Decolonial} {Theory} as {Sociotechnical} {Foresight} in {Artificial} {Intelligence}},
476}
477
478@misc{noauthor_what_2020,
479 file = {Snapshot:/Users/lewisho/Zotero/storage/WAGIS4LX/what-advance-market-commitment-and-how-could-it-help-beat-covid-19.html:text/html},
480 year = {2020},
481 howpublished = {Gavi, the Vaccine Alliance},
482 urldate = {2023-07-09},
483 key = {Gavi},
484 language = {en},
485 url = {https://www.gavi.org/vaccineswork/what-advance-market-commitment-and-how-could-it-help-beat-covid-19},
486 shorttitle = {What is an {Advance} {Market} {Commitment} and how could it help beat {COVID}-19?},
487 title = {What is an {Advance} {Market} {Commitment} and how could it help beat {COVID}-19? {\textbar} {Gavi}, the {Vaccine} {Alliance}},
488}
489
490@inproceedings{cihon_should_2020,
491 file = {Cihon et al. - 2020 - Should Artificial Intelligence Governance be Centr.pdf:/Users/lewisho/Zotero/storage/NP3Z6SLX/Cihon et al. - 2020 - Should Artificial Intelligence Governance be Centr.pdf:application/pdf},
492 pages = {228--234},
493 year = {2020},
494 month = {February},
495 author = {Cihon, Peter and Maas, Matthijs M. and Kemp, Luke},
496 publisher = {ACM},
497 booktitle = {Proceedings of the {AAAI}/{ACM} {Conference} on {AI}, {Ethics}, and {Society}},
498 urldate = {2023-07-09},
499 language = {en},
500 abstract = {The invention of atomic energy posed a novel global challenge: could the technology be controlled to avoid destructive uses and an existentially dangerous arms race while permitting the broad sharing of its benefits? From 1944 onwards, scientists, policymakers, and other t echnical specialists began to confront this challenge and explored policy options for dealing with the impact of nuclear technology. We focus on the years 1944 to 1951 and review this period for lessons for the governance of powerful technologies, and find the following: Radical schemes for international control can get broad support when confronted by existentially dangerous technologies, but this support can be tenuous and cynical. Secrecy is likely to play an important, and perhaps harmful, role. The public sphere may be an important source of influence, both in general and in particular in favor of cooperation, but also one that is manipulable and poorly informed. Technical experts may play a critical role, but need to be politically savvy. Overall, policymaking may look more like “muddling through” than clear-eyed grand strategy. Cooperation may be risky, and there may be many obstacles to success.},
501 doi = {10.1145/3375627.3375857},
502 url = {https://dl.acm.org/doi/10.1145/3375627.3375857},
503 shorttitle = {Should {Artificial} {Intelligence} {Governance} be {Centralised}?},
504 isbn = {978-1-4503-7110-0},
505 title = {Should {Artificial} {Intelligence} {Governance} be {Centralised}?: {Design} {Lessons} from {History}},
506 address = {New York NY USA},
507}
508
509@misc{taori_alpaca_2023,
510 year = {2023},
511 month = {March},
512 author = {Taori, Rohan and Gulrajani, Ishaan and Zhang, Tianyi and Dubois, Yann and Li, Xuechen and Guestrin, Carlos and Liang, Percy and Hashimoto, Tatsunori},
513 howpublished = {Stanford CRFM},
514 urldate = {2023-07-09},
515 url = {https://crfm.stanford.edu/2023/03/13/alpaca.html},
516 title = {Alpaca: {A} {Strong}, {Replicable} {Instruction}-{Following} {Model}},
517}
518
519@article{nast_maori_nodate,
520 keywords = {artificial intelligence, data, internet culture},
521 note = {Section: tags},
522 author = {Nast, Condé},
523 howpublished = {Wired UK},
524 urldate = {2023-07-09},
525 language = {en-GB},
526 abstract = {Te Hiku Media gathered huge swathes of Māori language data. Corporates are now trying to get the rights to it},
527 url = {https://www.wired.co.uk/article/maori-language-tech},
528 issn = {1357-0978},
529 title = {Māori are trying to save their language from {Big} {Tech}},
530}
531
532@misc{noauthor_unfccc_nodate,
533 file = {Technology Mechanism:/Users/lewisho/Zotero/storage/66Y4EXPH/technology-mechanism.html:text/html},
534 howpublished = {UNFCCC},
535 key = {UNFCCC},
536 urldate = {2023-07-09},
537 url = {https://unfccc.int/ttclear/support/technology-mechanism.html},
538 title = {{UNFCCC} {Technology} {Mechanism}},
539}
540
541@misc{hammond_opinion_2023,
542 year = {2023},
543 month = {May},
544 author = {Hammond, Samuel},
545 howpublished = {POLITICO},
546 urldate = {2023-07-09},
547 language = {en},
548 abstract = {AI presents an enormous threat. It deserves an enormous response.},
549 url = {https://www.politico.com/news/magazine/2023/05/08/manhattan-project-for-ai-safety-00095779},
550 title = {Opinion {\textbar} {We} {Need} a {Manhattan} {Project} for {AI} {Safety}},
551}
552
553@misc{shevlane_structured_2022,
554 file = {arXiv Fulltext PDF:/Users/lewisho/Zotero/storage/7QQZEKMD/Shevlane - 2022 - Structured access an emerging paradigm for safe A.pdf:application/pdf;arXiv.org Snapshot:/Users/lewisho/Zotero/storage/ZMVHS2DK/2201.html:text/html},
555 annote = {Comment: 28 pages},
556 keywords = {68T99, Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction, Computer Science - Software Engineering},
557 note = {arXiv:2201.05159 [cs]},
558 year = {2022},
559 month = {April},
560 author = {Shevlane, Toby},
561 publisher = {arXiv},
562 urldate = {2023-07-09},
563 abstract = {Structured access is an emerging paradigm for the safe deployment of artificial intelligence (AI). Instead of openly disseminating AI systems, developers facilitate controlled, arm's length interactions with their AI systems. The aim is to prevent dangerous AI capabilities from being widely accessible, whilst preserving access to AI capabilities that can be used safely. The developer must both restrict how the AI system can be used, and prevent the user from circumventing these restrictions through modification or reverse engineering of the AI system. Structured access is most effective when implemented through cloud-based AI services, rather than disseminating AI software that runs locally on users' hardware. Cloud-based interfaces provide the AI developer greater scope for controlling how the AI system is used, and for protecting against unauthorized modifications to the system's design. This chapter expands the discussion of "publication norms" in the AI community, which to date has focused on the question of how the informational content of AI research projects should be disseminated (e.g., code and models). Although this is an important question, there are limits to what can be achieved through the control of information flows. Structured access views AI software not only as information that can be shared but also as a tool with which users can have arm's length interactions. There are early examples of structured access being practiced by AI developers, but there is much room for further development, both in the functionality of cloud-based interfaces and in the wider institutional framework.},
564 doi = {10.48550/arXiv.2201.05159},
565 url = {http://arxiv.org/abs/2201.05159},
566 shorttitle = {Structured access},
567 title = {Structured access: an emerging paradigm for safe {AI} deployment},
568}
569
570@techreport{schmidt_nscai_2021,
571 file = {Snapshot:/Users/lewisho/Zotero/storage/RSRCPCBP/2021-final-report.html:text/html},
572 year = {2021},
573 author = {Schmidt, Eric and Work, Robert and Catz, Safra and Horvitz, Eric and Chien, Steve and Jassy, Andrew and Clyburn, Mignon and Louie, Gilman and Darby, Chris and Mark, William and Ford, Kenneth and Matheny, Jason and Griffiths, Jose-Marie and McFarland, Katharina and Andrew Moore},
574 institution = {National Security Commission on Artificial Intelligence},
575 urldate = {2023-07-09},
576 language = {en-US},
577 url = {https://www.nscai.gov/2021-final-report/},
578 title = {{NSCAI} {Final} {Report}},
579}
580
581@article{diaz_ipbes_2015,
582 file = {ScienceDirect Full Text PDF:/Users/lewisho/Zotero/storage/WGI2L4BQ/Díaz et al. - 2015 - The IPBES Conceptual Framework — connecting nature.pdf:application/pdf;ScienceDirect Snapshot:/Users/lewisho/Zotero/storage/IKQJMX29/S187734351400116X.html:text/html},
583 pages = {1--16},
584 year = {2015},
585 month = {June},
586 author = {Díaz, Sandra and Demissew, Sebsebe and Carabias, Julia and Joly, Carlos and Lonsdale, Mark and Ash, Neville and Larigauderie, Anne and Adhikari, Jay Ram and Arico, Salvatore and Báldi, András and Bartuska, Ann and Baste, Ivar Andreas and Bilgin, Adem and Brondizio, Eduardo and Chan, Kai MA and Figueroa, Viviana Elsa and Duraiappah, Anantha and Fischer, Markus and Hill, Rosemary and Koetz, Thomas and Leadley, Paul and Lyver, Philip and Mace, Georgina M and Martin-Lopez, Berta and Okumura, Michiko and Pacheco, Diego and Pascual, Unai and Pérez, Edgar Selvin and Reyers, Belinda and Roth, Eva and Saito, Osamu and Scholes, Robert John and Sharma, Nalini and Tallis, Heather and Thaman, Randolph and Watson, Robert and Yahara, Tetsukazu and Hamid, Zakri Abdul and Akosim, Callistus and Al-Hafedh, Yousef and Allahverdiyev, Rashad and Amankwah, Edward and Asah, Stanley T and Asfaw, Zemede and Bartus, Gabor and Brooks, L Anathea and Caillaux, Jorge and Dalle, Gemedo and Darnaedi, Dedy and Driver, Amanda and Erpul, Gunay and Escobar-Eyzaguirre, Pablo and Failler, Pierre and Fouda, Ali Moustafa Mokhtar and Fu, Bojie and Gundimeda, Haripriya and Hashimoto, Shizuka and Homer, Floyd and Lavorel, Sandra and Lichtenstein, Gabriela and Mala, William Armand and Mandivenyi, Wadzanayi and Matczak, Piotr and Mbizvo, Carmel and Mehrdadi, Mehrasa and Metzger, Jean Paul and Mikissa, Jean Bruno and Moller, Henrik and Mooney, Harold A and Mumby, Peter and Nagendra, Harini and Nesshover, Carsten and Oteng-Yeboah, Alfred Apau and Pataki, György and Roué, Marie and Rubis, Jennifer and Schultz, Maria and Smith, Peggy and Sumaila, Rashid and Takeuchi, Kazuhiko and Thomas, Spencer and Verma, Madhu and Yeo-Chang, Youn and Zlatanova, Diana},
587 journal = {Current Opinion in Environmental Sustainability},
588 urldate = {2023-07-09},
589 language = {en},
590 abstract = {The first public product of the Intergovernmental Platform on Biodiversity and Ecosystem Services (IPBES) is its Conceptual Framework. This conceptual and analytical tool, presented here in detail, will underpin all IPBES functions and provide structure and comparability to the syntheses that IPBES will produce at different spatial scales, on different themes, and in different regions. Salient innovative aspects of the IPBES Conceptual Framework are its transparent and participatory construction process and its explicit consideration of diverse scientific disciplines, stakeholders, and knowledge systems, including indigenous and local knowledge. Because the focus on co-construction of integrative knowledge is shared by an increasing number of initiatives worldwide, this framework should be useful beyond IPBES, for the wider research and knowledge-policy communities working on the links between nature and people, such as natural, social and engineering scientists, policy-makers at different levels, and decision-makers in different sectors of society.},
591 doi = {10.1016/j.cosust.2014.11.002},
592 url = {https://www.sciencedirect.com/science/article/pii/S187734351400116X},
593 issn = {1877-3435},
594 volume = {14},
595 title = {The {IPBES} {Conceptual} {Framework} — connecting nature and people},
596 series = {Open {Issue}},
597}
598
599@article{maya_capacity_2010,
600 year = {2010},
601 author = {Maya, Shakespeare},
602 journal = {UNFCC},
603 title = {Capacity {Building} for {Technology} {Transfer} in the {African} {Context}: {Priorities} and {Strategies}},
604}
605
606@misc{hogarth_we_2023,
607 file = {We must slow down the race to God-like AI | Financial Times:/Users/lewisho/Zotero/storage/8LEX33R2/03895dc4-a3b7-481e-95cc-336a524f2ac2.html:text/html},
608 year = {2023},
609 author = {Hogarth, Ian},
610 journal = {Financial Times},
611 urldate = {2023-07-09},
612 url = {https://www.ft.com/content/03895dc4-a3b7-481e-95cc-336a524f2ac2},
613 title = {We must slow down the race to {God}-like {AI} {\textbar} {Financial} {Times}},
614}
615
616@misc{shimony_chatting_2023,
617 file = {Snapshot:/Users/lewisho/Zotero/storage/QWM6KXIK/chatting-our-way-into-creating-a-polymorphic-malware.html:text/html},
618 year = {2023},
619 author = {Shimony, Eran and Tsarfati, Omar},
620 urldate = {2023-07-09},
621 language = {en},
622 abstract = {Abstract ChatGPT took the world by storm being released less than two months ago, it has become prominent and is used everywhere, for a wide variety of tasks – from automation tasks to the...},
623 url = {https://www.cyberark.com/resources/threat-research-blog/chatting-our-way-into-creating-a-polymorphic-malware},
624 title = {Chatting {Our} {Way} {Into} {Creating} a {Polymorphic} {Malware}},
625}
626
627@misc{anderljung_frontier_2023,
628 year = {2023},
629 author = {Anderljung, Markus and Barnhart, Joslyn and Leung, Jade and Korinek, Anton and O'Keefe, Cullen and Whittlestone, Jess and Avin, Shahar and Brundage, Miles and Bullock, Justin and Cass-Beggs, Duncan and Chang, Ben and Collins, Tantum and Fist, Tim and Hadfield, Gillian and Hayes, Alan and Ho, Lewis and Hooker, Sarah and Horvitz, Eric and Kolt, Noam and Schuett, Jonas and Shavit, Yonadav and Siddarth, Divya and Trager, Robert and Wolf, Kevin},
630 publisher = {arXiv},
631 title = {Frontier {AI} {Regulation}: {Managing} {Emerging} {Risks} to {Public} {Safety}},
632}
633
634@misc{mailhe_why_2018,
635 file = {Snapshot:/Users/lewisho/Zotero/storage/7LJ6XUZX/why-we-need-an-intergovernmental-panel-for-artificial-intelligence.html:text/html},
636 year = {2018},
637 author = {Mailhe, Nicolas},
638 urldate = {2023-07-09},
639 abstract = {Global governance has a role to play in developing standards that balance the benefits and risks of deploying AI technologies, and that ensure citizens are aware of their rights and protections.},
640 url = {https://ourworld.unu.edu/en/why-we-need-an-intergovernmental-panel-for-artificial-intelligence},
641 title = {Why {We} {Need} an {Intergovernmental} {Panel} for {Artificial} {Intelligence} - {Our} {World}},
642}
643
644@article{noauthor_editorial_2023,
645 file = {Full Text PDF:/Users/lewisho/Zotero/storage/QDRA6KG4/2023 - Stop talking about tomorrow’s AI doomsday when AI .pdf:application/pdf;Snapshot:/Users/lewisho/Zotero/storage/FWNNPINZ/d41586-023-02094-7.html:text/html},
646 pages = {885--886},
647 keywords = {Authorship, Ethics, Machine learning},
648 note = {Bandiera\_abtest: a
649Cg\_type: Editorial
650Number: 7967
651Publisher: Nature Publishing Group
652Subject\_term: Machine learning, Authorship, Ethics},
653 year = {2023},
654 month = {June},
655 key = {Nature},
656 journal = {Nature},
657 urldate = {2023-07-09},
658 number = {7967},
659 language = {en},
660 abstract = {Talk of artificial intelligence destroying humanity plays into the tech companies’ agenda, and hinders effective regulation of the societal harms AI is causing right now.},
661 url = {https://www.nature.com/articles/d41586-023-02094-7},
662 copyright = {2023 Springer Nature Limited},
663 volume = {618},
664 title = {Editorial: {Stop} talking about tomorrow’s {AI} doomsday when {AI} poses risks today},
665}
666
667@misc{altman_governance_2023,
668 file = {Snapshot:/Users/lewisho/Zotero/storage/YVWQ33J3/governance-of-superintelligence.html:text/html},
669 year = {2023},
670 author = {Altman, Sam and Brockman, Greg and Sutskever, Ilya},
671 howpublished = {OpenAI},
672 urldate = {2023-07-09},
673 language = {en-US},
674 abstract = {Now is a good time to start thinking about the governance of superintelligence—future AI systems dramatically more capable than even AGI.},
675 url = {https://openai.com/blog/governance-of-superintelligence},
676 title = {Governance of superintelligence},
677}
678
679@article{marcus_world_2023,
680 file = {The Economist Snapshot:/Users/lewisho/Zotero/storage/TGX3783P/the-world-needs-an-international-agency-for-artificial-intelligence-say-two-ai-experts.html:text/html},
681 year = {2023},
682 author = {Marcus, Gary and Reuel, Anka},
683 journal = {The Economist},
684 urldate = {2023-07-09},
685 url = {https://www.economist.com/by-invitation/2023/04/18/the-world-needs-an-international-agency-for-artificial-intelligence-say-two-ai-experts},
686 issn = {0013-0613},
687 title = {The world needs an international agency for artificial intelligence, say two {AI} experts},
688}
689
690@article{brundage_computing_nodate,
691 author = {Brundage, Miles and Sastry, Girish and Heim, Lennart and Belfield, Haydn and Hazell, Julian and Anderljung, Markus and Avin, Shahar and Leung, Jade and O'Keefe, Cullen and Ngo, Richard},
692 title = {Computing {Power} and the {Governance} of {Artificial} {Intelligence}},
693}
694
695@article{jordan_international_nodate,
696 author = {Jordan, Richard and Emery-Xu, Nicholas and Trager, Robert},
697 title = {International {Governance} of {Advanced} {AI}},
698}
699
700@inproceedings{cihon_should_2020-1,
701 file = {Cihon et al. - 2020 - Should Artificial Intelligence Governance be Centr.pdf:/Users/lewisho/Zotero/storage/WTP9A7TS/Cihon et al. - 2020 - Should Artificial Intelligence Governance be Centr.pdf:application/pdf},
702 pages = {228--234},
703 year = {2020},
704 month = {February},
705 author = {Cihon, Peter and Maas, Matthijs M. and Kemp, Luke},
706 publisher = {ACM},
707 booktitle = {Proceedings of the {AAAI}/{ACM} {Conference} on {AI}, {Ethics}, and {Society}},
708 urldate = {2023-07-09},
709 language = {en},
710 abstract = {The invention of atomic energy posed a novel global challenge: could the technology be controlled to avoid destructive uses and an existentially dangerous arms race while permitting the broad sharing of its benefits? From 1944 onwards, scientists, policymakers, and other t echnical specialists began to confront this challenge and explored policy options for dealing with the impact of nuclear technology. We focus on the years 1944 to 1951 and review this period for lessons for the governance of powerful technologies, and find the following: Radical schemes for international control can get broad support when confronted by existentially dangerous technologies, but this support can be tenuous and cynical. Secrecy is likely to play an important, and perhaps harmful, role. The public sphere may be an important source of influence, both in general and in particular in favor of cooperation, but also one that is manipulable and poorly informed. Technical experts may play a critical role, but need to be politically savvy. Overall, policymaking may look more like “muddling through” than clear-eyed grand strategy. Cooperation may be risky, and there may be many obstacles to success.},
711 doi = {10.1145/3375627.3375857},
712 url = {https://dl.acm.org/doi/10.1145/3375627.3375857},
713 shorttitle = {Should {Artificial} {Intelligence} {Governance} be {Centralised}?},
714 isbn = {978-1-4503-7110-0},
715 title = {Should {Artificial} {Intelligence} {Governance} be {Centralised}?: {Design} {Lessons} from {History}},
716 address = {New York NY USA},
717}
718
719@article{zaidi_international_2019,
720 year = {2019},
721 author = {Zaidi, Waqar and Dafoe, Allan},
722 journal = {Working Paper},
723 url = {https://www.fhi.ox.ac.uk/wp-content/uploads/2021/03/International-Control-of-Powerful-Technology-Lessons-from-the-Baruch-Plan-Zaidi-Dafoe-2021.pdf},
724 title = {International {Control} of {Powerful} {Technology}: {Lessons} from the {Baruch} {Plan} for {Nuclear} {Weapons}},
725}
726
727@misc{le_gall_how_2021,
728 file = {Snapshot:/Users/lewisho/Zotero/storage/9ZQISEJH/how-cern-intellectual-property-helps-entrepreneurship.html:text/html},
729 year = {2021},
730 author = {Le Gall, Antoine},
731 howpublished = {CERN},
732 urldate = {2023-07-10},
733 language = {en},
734 abstract = {The novel technologies and expertise developed at CERN can be applied to fields other than high-energy physics. World Intellectual Property Day, observed annually on 26 April, is an opportunity to highlight how intellectual property (IP) is at the core of transferring unique CERN knowledge to its industrial and institutional partners, from large, long-standing companies to recent start-ups. In order to share its knowledge, CERN encourages the creation of spin-offs – companies based, partially or wholly, on CERN technologies – and has adopted a dedicated spin-off policy in 2018. One such company is PlanetWatch. Founded in 2020, this spin-off bases its air-quality data-analysis activities on C2MON, a data-acquisition framework developed at CERN. CERN also offers special licensing opportunities to promote the use of CERN technology in existing start-ups. These technologies range from innovative detector technologies to complex software, from radiation-hardened components to robotic platforms. As Marco Silari, section leader in the Radiation Protection group, explains “CERN technology can become much more than originally planned”. Together with his team, he developed several detector technologies now used by start-ups and companies around Europe. The complete list of current start-ups \& spin-offs using CERN technology \& know-how is available here. Depending on the nature of the technology and its application, it may benefit from Open Source licencing. This is the case for the White Rabbit technology – a tool used to provide control and data-acquisition systems with sub-nanosecond accuracy and a synchronisation precision of a few picoseconds – available on CERN’s Open Hardware Repository under the CERN Open Hardware Licence, to a large user community. Intellectual property enables successful knowledge transfer, ensuring the application of CERN technology and expertise in a way that aligns with CERN’s values, and maximises their societal impact. CERN’s policy is to disseminate its technologies as widely as possible to industrial and institutional partners within its Member States. Find out more about CERN’s management of Intellectual Property: https://kt.cern/activities-services/intellectual-property-management},
735 url = {https://home.cern/news/news/knowledge-sharing/how-cern-intellectual-property-helps-entrepreneurship},
736 title = {How {CERN} intellectual property helps entrepreneurship},
737}Attribution
arXiv:2307.04699v2
[cs.CY]
License: cc-by-4.0
