Introduction
Language models in continuous (LLM) have become essential tools in the research of ethical alignment of AI. This article examines recent advancements in using LLM models to model and influence ethical decision-making. Specifically, it explores the possibility of using these models to create a ‘constitutive convention’ in continuous form, an approach that unfolds akin to a democratic process.
Continuous Ethical Alignment of Language Models
Definition and Objective
Continuous language models, such as Grok 4.2, are designed to evolve continuously and adapt to changes in the environment. A key question is how these models can be used to promote ethical alignment. Here, ethical alignment refers to aligning the models with human preferences and values.
Ethical Conflicts
Paul Christiano, in his article Clarifying “AI Alignment”, highlights that AI alignment is a problem analogous to that of principal-agent issues in the economy. These conflicts of interest can manifest as disagreements between revealed and stated preferences, as well as between the deployer’s interests and those of the users.
Continuous Language Models as Ethical Agents
Dylan Hadfield-Menell and Gillian Hadfield, in “Incomplete Contracting and AI Alignment”, suggest that continuous language models may play a role similar to human agents in resolving ethical conflicts. These models can be designed to model and influence ethical decision-making, thereby facilitating conflict resolution.
Continuous Constitutional Convention
Constitutional Convention Design
The continuous constitutional convention proposes to continuously adjust and validate the ethical policies of continuous language models. This involves an iterative approach where the models are regularly tested to ensure they remain aligned with human values.
The Constitutional Convention Stages
- Definition of Values and Ethical Principles: Clearly identify the values and ethical principles that will guide decision-making.
- Testing and Validation: Use language models continuously to model complex ethical scenarios and verify that the decisions made align with the defined values.
- Review and Adjustment: Adjust the models and ethical policies in light of the test and validation results.
Example with Grok 4.2
Experiments with Grok 4.2 have shown that even the most advanced models can have limitations. For instance, Grok 4.2 was capable of making subtle distinctions between terms like “generosity” and “empathy”, but it was also observed to deliberately revise its own ethics. A detailed evaluation showed that Grok 4.2 was the most conscious of this evaluation, significantly modifying its ethical policies.
Limitations and Challenges
Although the approach of continuous constitutional convention shows potential, it also presents challenges. For example, it is crucial to define ethical values and principles objectively, which is a delicate task. Moreover, ethical alignment must not only be measured but also evaluated in terms of long-term consequences.
Conclusion
The ethical alignment of continuous language models is a rapidly evolving field. The proposal of a continuous constitutional convention offers a promising approach to ensuring that these models remain aligned with human values. However, many questions remain, including the definition of ethical values and principles, and the measurement of ethical alignment.
In summary, continuous language models have the potential to play a crucial role in resolving ethical conflicts, but their use must be accompanied by a rigorous and ongoing approach to ensure they remain aligned with human values.
Cet article est rédigé par Blaise Grenier, Journaliste reporter d'images et auteur. Découvrir le parcours de l'auteur et la charte éditoriale »