Existential Hope Podcast: Joe Carlsmith | Infite Ethics and the Sublime Utopia

The Foresight Institute Podcast

Joe Carlsmith is a writer, researcher, and philosopher. He works as a senior research analyst at Open Philanthropy, focusing on existential risk from advanced artificial intelligence. He also writes independently about various topics in philosophy and futurism and has a doctorate in philosophy from the University of Oxford.

Much of his work is about trying to help us orient wisely towards humanity’s long-term future. He delves into questions about meta-ethics and rationality at the foundation, feeding into questions about ethics (and especially about effective altruism), which motivate concern for the long-term future.

Session Summary
Join us to explore Joseph Carlsmith’s insights into his ongoing work and thoughts on issues including AI alignment, lesser-known future risks, infinite ethics and digital minds, and the sublime utopia.

Carlsmith shares his concerns in ensuring that advanced AI systems behave beneficially for humanity. However, he emphasizes the importance of broadening the horizon to identify and address other critical factors beyond technical AI alignment. For instance, he delves into infinite ethics, addressing the ethical considerations involving infinite impacts and numbers of people – necessary if we are going to meet future realities.

Despite the challenges, this episode is devoted to the exploration of utopia. Beyond the lesser boundary of a concrete utopia, Carlsmith envisions the sublime utopia: a realm of aspirational goals and visions. Although filled with vulnerabilities tied to hoping for extraordinary and unprecedented outcomes, Carlsmith lays out the essential nature of such a pursuit.

Full transcript, list of resources, and art piece https://www.existentialhope.com/podcasts

Existential Hope was created to collect positive and possible scenarios for the future, so that we can have more people commit to the creation of a brighter future, and to start mapping out the main developments and challenges that need to be navigated to reach it. Find all previous podcast episodes here, always featuring a full transcript, artwork inspired by the episode, and a list of recommended resources from the podcast. Existential Hope is a Foresight Institute project.

The Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.

Allison Duettmann is the president and COO of Foresight Institute. She directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, Fellowships, Prizes, and Tech Trees, and shares this work with the public. She founded Existentialhope.com, co-edit

---

Hosted on Acast. See acast.com/privacy for more information.