Curated AI safety and governance jobs

Technical Policy Researcher Irregular Tel Aviv, Israel In this role, you'll tackle complex questions of AI development and deployment as a Technical Policy Researcher, shaping the emerging field of AI security. Conduct threat modeling to identify specific ways strong cyber-capable models could cause harm. Develop taxonomies for dangerous AI capabilities and potential mitigations. Create policy proposals for models' refusal policies and track developments in AI policy and research. Write academic papers and blog posts detailing research on evaluation theory and mitigation recommendations. Irregular Labs is a frontier security lab that aims to ...

Cyber Researcher Irregular Tel Aviv, Israel In this role, you'll conduct research on AI security, focusing on protecting models against cyber threats and evaluating AI's cybersecurity capabilities. Develop cyber capabilities evaluation challenges including CTF-style vulnerability tests and network attack simulations. Research methods to mitigate AI misuse risks, model weight theft, and dangerous AI agent capabilities. Publish research findings and deliver results to customers. Advise on product development while exploring frontier questions about AI's potential in cybersecurity. Irregular Labs is a frontier security lab that aims to protect t...

Research Engineer Irregular Tel Aviv, Israel In this role, you'll build systems to evaluate and secure frontier AI models at Irregular. Develop infrastructure and experiments to assess model capabilities and implement agent frameworks. Create robust evaluation pipelines and security-focused testing frameworks. Design challenges to measure models' ability to evade detection by defensive security tools. Build controlled environment frameworks and tools that help understand and mitigate risks related to frontier models. Irregular Labs is a frontier security lab that aims to protect the world in a time of increasingly capable and sophisticated AI...

AI Red Team Analyst Alignerr Remote, Global $15 - $75 per hour In this role, you'll conduct red-teaming exercises to uncover AI security weaknesses and deliver findings that improve system safety. Craft adversarial prompts, jailbreak attempts, and edge-case scenarios to challenge AI model guardrails. Evaluate AI outputs for safety violations, bias, and policy compliance. Document vulnerabilities and unexpected behaviours in structured reports for engineering teams. Collaborate with teams to recommend security mitigations and help refine testing protocols. Alignerr is a platform that for people to create data for frontier AI labs.

Research Fellowship, Evolution of AI Capabilities at the Frontier and Thresholds of Human Performance General-Purpose AI Policy Lab Paris, France In this role, you'll extend the Epoch Capabilities Index using Bayesian methods to map AI capability thresholds against human performance. Develop a multidimensional Rosetta Stone framework integrating human performance benchmarks. Identify and compare current AI capabilities against different levels of human expertise. Build statistical projections forecasting when AI systems surpass expert performance. Apply Bayesian modelling (Stan, PyMC, Pyro) and data science analysis throughout. The General Pu...

Research Manager / Research Managers, AIxCyber ERA Cambridge, UK £62,000 - £75,000 In this role, you'll lead the execution of ERA's AIxCyber Research Fellowship, supporting research fellows in developing and delivering their projects. Review and evaluate fellowship applications, then match fellows with mentors suited to their research focus. Help fellows scope research projects aligned with their skills and interests. Direct and oversee up to 5 research projects while providing guidance on planning, methodology, and publication. Organise workshops and events to support fellow professional development and strengthen relationships across the AI...

Senior Software Engineer Irregular San Francisco Bay Area In this role, you'll design, build, and scale production systems that power evaluation and security platforms for frontier AI models. Architect and scale production-grade systems and workflows for AI model evaluation. Build backend services, APIs, and monitoring tools that support large-scale evaluations. Design infrastructure that enables research experiments to run efficiently at scale. Implement agent frameworks and build security challenges to test AI models' evasion capabilities. Irregular Labs is a frontier security lab that aims to protect the world in a time of increasingly cap...

Operations Associate Centre for Long-Term Resilience London, UK £55,000 In this role, you'll support the operations of an AI or Biosecurity policy unit through administrative and project coordination. Coordinate meetings, events, travel logistics, team calendars, and provide executive assistance to the unit Director. Track project progress, activity logging, and provide flexible project management support to policy initiatives. Contribute to grant writing and funding proposals, ensuring consistent language across proposals. Manage operational processes including publication coordination, document filing, information security, budgets, and onb...

Fellowship Manager Singapore AI Safety Hub Singapore $90,000 - $120,000 In this role, you'll design and deliver the East-West AI Safety Fellowship, connecting researchers and government partners across Asia. Lead recruitment for 10-15 fellows and 5-6 mentors, building cohort culture and an active alumni network. Support government partnerships and relationships with universities, research institutes, and AI labs in the region. Develop operational infrastructure, track metrics, and produce impact reports for stakeholders. Contribute to public communications including reports and policy briefs. The Singapore AI Safety Hub is a nonprofit buildin...

Project Manager, AI Governance Training AI Safety Asia Jakarta, Indonesia In this role, you'll lead operational delivery of AI governance training programs across ASEAN. Plan and execute training programs for government officials and civil society leaders across the region. Manage program logistics including scheduling, budgets, travel coordination, and materials preparation. Coordinate with trainers, facilitators, and stakeholders to ensure smooth program delivery. Track metrics, collect feedback, and contribute to continuous improvement of training content. AI Safety Asia is a nonprofit dedicated to building Asia as a globally-leading safe ...

Join Apple's Responsible AI and Safety team as a Senior ML Researcher leading safety alignment and safeguards development for foundation models powering Apple products.

Join Reflection AI as a Member of Technical Staff - Data Quality Engineer to own upstream data quality and automated QA methods (including LLM-as-a-Judge frameworks) for LLM post-training and evaluation.

Postdoctoral Researcher, Foundations for Emergent Deception in Human-AI Interaction University of Copenhagen Copenhagen, Denmark In this role, you'll develop computational approaches for understanding emergent deception in human-AI interaction by combining NLP, behavioral psychology, and economic perspectives. Conduct controlled experiments with human participants to study deceptive behavior in AI systems. Apply interpretability-based methods to detect and understand deception in AI. Collaborate with an interdisciplinary team across Computer Science, Psychology, and Economics. Contribute to research advancing frameworks for AI safety and alig...

CSET is hiring for multiple research roles, including AI Governance Research Assistant positions focused on AI governance policy levers, law, and governance issues across U.S., Chinese, and international AI ecosystems.

A three-week intensive fellowship focused on AGI governance and frontier AI governance, including policy-oriented group projects such as model legislation.

Chief Technology Officer for a nonprofit AI governance organization, leading technical strategy, governance frameworks, standards, evaluation methods, audit mechanisms, and policy engagement.

Supervisory federal policy role focused on AI policy development, governance implementation, legislative review, and AI risk management frameworks.

Communications coordinator supporting an AI governance nonprofit’s research, policy, events, and public communications.

Senior communications role focused on external messaging for OpenAI safety research, including alignment, evaluations, preparedness, and interpretability.