Overview

Lesson 2.2: Ethical Frameworks for AI in Research

This lesson teaches research leaders how to develop comprehensive ethical review frameworks for AI-assisted research. You'll learn to adapt existing ethics review processes (IRBs) to address AI-specific concerns, design ethics committees with appropriate expertise, balance principles-based and rules-based approaches, and cultivate ethical culture in AI research.

Title

Lesson 2.2: Ethical Frameworks for AI in Research

Purpose

This lesson teaches research leaders how to develop comprehensive ethical review frameworks for AI-assisted research. You'll learn to adapt existing ethics review processes (IRBs) to address AI-specific concerns, design ethics committees with appropriate expertise, balance principles-based and rules-based approaches, and cultivate ethical culture in AI research.

The Ethics Gap: Where Existing Frameworks Fall Short

Modern research ethics rests on foundations built in response to historical abuses of human subjects in research. The Belmont Report (1979) established three foundational principles, respect for persons (autonomy), beneficence, and justice, that became the backbone of IRB review processes across the United States. The Declaration of Helsinki (1964, revised most recently in 2013) established international standards for medical research involving human subjects. Professional codes, the APA Ethics Code, the American Statistical Association's Ethical Guidelines for Statistical Practice, the ACM Code of Ethics, establish conduct standards for specific research communities.

These frameworks are well-designed for the problems they were designed to solve. But they were built for a world where research harm flows from intentional human decisions or identifiable failures in research design: a drug trial that doesn't adequately protect subjects, a researcher who misrepresents data, a study that exploits a vulnerable population. AI systems introduce a fundamentally different category of potential harm: diffuse, emergent, and often unintentional harm that arises from the behavior of complex systems operating at scale, learning from data, making predictions, and acting on those predictions in ways that compound across millions of interactions.

Consider three scenarios that illustrate where existing ethics frameworks struggle. First: a research team trains a natural language processing model on a corpus of clinical notes. The corpus reflects historical patterns of care that include documented racial disparities in pain treatment. The model, trained to predict appropriate pain management, learns these biased patterns and encodes them. No researcher made an intentional decision to build a biased model, the harm emerged from data that reflected systemic inequities. Belmont's justice principle requires fair selection of research subjects, but it doesn't provide clear guidance for this kind of data-embedded harm.

Second: a research team develops an AI system for detecting signs of psychological distress in social media posts, with the intention of enabling mental health outreach. The system works at scale, analyzing public posts of millions of people who have never consented to being research subjects. The research exemption under 45 CFR 46 likely applies because public data is being used, but the ethical implications of large-scale behavioral surveillance for mental health purposes go beyond the formal consent question. Third: a research team releases an AI model for scientific literature summarization. The model is later used by others to generate scientific misinformation at scale. The researchers had no intent to enable this harm, and it emerged years after the research was completed. What are the researchers' ongoing ethical obligations?

These scenarios reveal that AI research ethics requires a framework that goes beyond consent, beneficence, and justice as traditionally defined: one that addresses systemic harm, distributed harm, data-embedded harm, and extended responsibility across the research lifecycle.

Adapting the Principlist Framework for AI Research

Rather than abandoning established bioethics principles, a productive approach adapts them to the specific characteristics of AI research. The principlist approach, developed by Beauchamp and Childress in 'Principles of Biomedical Ethics', translates usefully to AI contexts when each principle is re-specified for AI's distinctive features.

Autonomy, in traditional bioethics, centers on informed consent: individuals should be able to make informed decisions about their participation in research. In AI research, autonomy maps to transparency and explainability: individuals who are affected by AI decision-making systems have a legitimate interest in understanding how those systems operate and what data about them is used. This doesn't always require the same consent framework as clinical research: but it does require that AI systems affecting individuals be explainable, that data use be disclosed, and that meaningful opt-out mechanisms exist where feasible. For research that deploys AI systems affecting individuals, transparency about the AI nature of the system and the basis for its outputs is an autonomy obligation.

Beneficence, the obligation to maximize benefit, takes on specific operational meaning in AI research. It requires that researchers: design AI systems with the intended beneficial use cases specified and built-in; evaluate whether the AI system actually achieves its intended benefit (not just whether it achieves technical performance metrics); consider who benefits and whether benefits are distributed equitably; and assess the counterfactual (does the AI research create benefits that would not otherwise be achievable?). Beneficence in AI research also includes the obligation to make beneficial AI research available, restrictive intellectual property practices that prevent beneficial AI tools from reaching the populations that need them raise beneficence concerns.

Non-maleficence, avoiding harm, requires AI researchers to proactively identify potential harms before they occur, rather than responding reactively. This includes: systematic bias auditing before deployment; red-team exercises to identify misuse potential; assessment of downstream harms to third parties not directly involved in the research; and ongoing monitoring for harm after deployment. Non-maleficence in AI research must also address harms to communities and populations, not just individuals. This is a significant expansion from the individual-centered harm framework of traditional bioethics.

Justice, fair distribution of burdens and benefits, is particularly salient for AI research because AI systems can encode and amplify existing social inequities. Justice in AI research requires: ensuring that the populations whose data trains AI systems are among those who benefit from the AI; conducting disparity analysis as a standard research output requirement; considering the justice implications of AI research priorities (who decides what AI is developed for?); and engaging affected communities in research design, not just as data sources. The concept of algorithmic justice, that AI systems should not discriminate along protected characteristics, is a specific application of the justice principle to AI.

Bias and Fairness as Ethical Imperatives

Bias in AI research is not a technical failure. It is an ethical failure that requires ethical accountability. Understanding where bias enters AI systems and what fairness requires is foundational for research leaders who want to establish ethical standards in their institutions.

Bias enters AI systems through multiple pathways. Historical bias in training data is the most discussed: training data represents the past, including past patterns of discrimination, exclusion, and inequity. A hiring algorithm trained on historical hiring data will learn patterns that reflect who was historically hired: patterns that may encode gender, racial, and educational biases. A medical AI trained on clinical data will reflect historical patterns of care, including documented disparities in diagnosis, treatment, and follow-up for racial and ethnic minority populations. The AI system does not create these biases. It learns and operationalizes them.

Measurement bias occurs when the variables used in AI systems do not accurately measure what they purport to measure, and where measurement errors are systematically related to group membership. A classic example is the use of healthcare cost as a proxy for healthcare need in clinical AI systems: because access to healthcare is unequal, cost is a worse proxy for need among populations with lower access, leading to systematic underestimation of need for those populations.

Aggregation bias occurs when models are trained on aggregate data that treats heterogeneous populations as uniform, and then applied to subpopulations that differ systematically from the aggregate. A model trained primarily on data from high-income populations may perform poorly for low-income populations even if low-income population data was included in training, if the training data was imbalanced, the model's learned patterns may not generalize.

Fairness metrics for AI systems are not a single thing. They are a family of criteria, each reflecting different values, and the technical impossibility of satisfying all simultaneously is one of the central insights of algorithmic fairness research. Demographic parity (the model's positive prediction rate should be equal across groups) reflects a different value than equalized odds (the model's true positive rate and false positive rate should be equal across groups) which reflects a different value than individual fairness (similar individuals should be treated similarly). Research institutions need to decide which fairness criteria are appropriate for which research contexts, not outsource that decision to a technical team.

The ethical imperative for research leaders is to require fairness auditing as a standard component of AI research that will affect individuals or populations, and to ensure that fairness analysis is reported in research outputs. Research that demonstrates strong aggregate performance metrics while producing disparate impacts on minority populations is not ethically complete, even if it is technically sophisticated.

Consent Frameworks for AI Research

Consent in AI research is complicated by the nature of how AI systems are trained and deployed. Traditional informed consent frameworks assume a bounded research interaction: a subject participates in a specific study, consents to that specific participation, and the research relationship is defined and time-limited. AI research often involves data collected for one purpose being used to train systems for another purpose, at scale, affecting people who never interacted with researchers directly.

Secondary use of existing data for AI training raises the question of whether original consent covers AI training use. If a patient consented to their clinical data being used for 'medical research,' does that cover training a commercial diagnostic AI system? If a student consented to anonymized participation in an educational study, does that cover training an AI that will be deployed in educational settings beyond the original study? The answer is frequently ambiguous, and research institutions need both legal guidance and ethical frameworks for making these determinations.

FDA guidance on secondary use of real-world data for AI training in medical contexts has moved toward requiring that consent for secondary use be specific to the nature of AI training, including disclosing commercial applications if applicable. NIH's data management and sharing requirements include consent considerations for secondary use. Research leaders should ensure that their IRB and research consent templates are updated to address AI training use specifically, consent language developed before the AI era will frequently be ambiguous on this point.

Opt-out versus opt-in for AI research represents a values choice, not just a technical decision. Opt-in (requiring affirmative consent) provides stronger individual autonomy protection but may introduce selection bias into AI training datasets, the people who opt in may be systematically different from those who don't, and the resulting AI system may not generalize to the broader population. Opt-out (including all data unless individuals actively exclude themselves) may produce more representative training datasets but provides weaker autonomy protection. The appropriate choice depends on the sensitivity of the data, the potential for individual harm, and the research purpose, and it is an ethical decision that should be made transparently.

Dynamic consent models are emerging as a technologically-enabled alternative that may resolve some of these tensions. Dynamic consent platforms allow individuals to set and revise their data sharing preferences over time, see what their data has been used for, and withdraw consent for future use. For longitudinal AI research programs that maintain ongoing participant relationships, dynamic consent offers a path to both broad data use and strong autonomy protection. Several UK Biobank-style research programs are exploring dynamic consent implementation.

Synthetic data as a consent-avoiding strategy is increasingly discussed in AI research, if you generate synthetic data that preserves the statistical properties of real data without including any real individuals' data, you may be able to train AI systems without requiring consent for the training data. This is a promising approach but not a complete solution: synthetic data that too closely mirrors real data can enable re-identification; the statistical patterns in synthetic data still reflect historical inequities; and the quality of AI systems trained on synthetic data may not match those trained on real data for high-stakes applications.

Dual-Use AI Research

Dual-use research refers to research with legitimate scientific value that could also be used for harmful purposes. In biosecurity, 'dual-use research of concern' (DURC) has a formal policy framework at NIH, NSF, and most research institutions. AI research presents analogous dual-use challenges that are less institutionally established but equally important.

The challenge of AI dual-use is that the line between research that enables beneficial capabilities and research that enables harmful capabilities is often impossible to draw clearly in advance. Consider deepfake detection research: developing tools to detect manipulated media requires understanding how to create manipulated media convincingly, the same research enables both detection and creation. Consider adversarial robustness research: testing AI systems' vulnerability to adversarial attacks requires creating effective attacks, knowledge that enables both defense and offense. Consider LLM safety research: understanding how to jailbreak large language models is necessary for developing safeguards, but also enables the jailbreaks themselves.

Autonomous systems research presents particularly acute dual-use challenges. Research on autonomous navigation, target recognition, and decision-making in uncertain environments has clear military applications even when conducted in civilian research contexts. ITAR and other export control frameworks address some of these concerns, but the ethical dimensions go beyond what regulatory compliance captures, a research institution may have legal clearance to conduct research that raises serious ethical questions about contributing to autonomous weapons development.

Institutional frameworks for dual-use AI review should be modeled on, but distinct from, existing DURC review frameworks. Key elements: a definition of AI research categories subject to dual-use review (autonomous systems, large-scale influence tools, identity and surveillance AI, AI for biological research with potential weapons applications); a review body with expertise in both AI and the relevant harm domains; a decision framework that weighs potential benefits against potential harms with attention to probability, severity, and reversibility; and policies on publication: whether dual-use AI research should be published without restriction, published with specific information withheld, or not published.

The research community is actively debating whether 'responsible disclosure' norms from cybersecurity should be adopted more broadly for AI research, notifying developers and relevant authorities about vulnerabilities before public disclosure to allow time for mitigation. Research leaders who are building institutional dual-use frameworks should engage with these community discussions rather than developing institutional policies in isolation.

Reproducibility as Ethical Obligation

Reproducibility is a scientific value, but for AI research it is also an ethical obligation, because AI research that cannot be reproduced cannot be validated, and AI research that cannot be validated should not be deployed in consequential decisions affecting real people.

The replication crisis in AI research is well-documented. Studies have found that many published AI results cannot be reproduced even with access to the original code and data, due to uncontrolled randomness in training, hardware differences, software version dependencies, and underspecified methods. A 2021 analysis found that fewer than 15% of ML papers provided code, fewer than 5% provided data, and the reproducibility rate for claimed performance improvements was well below what would be expected if results were accurate. This is not just a scientific quality problem. It is an ethics problem, because unreproducible AI research creates false confidence in systems that may not perform as claimed when deployed.

Model cards are a structured documentation format for AI models proposed by Mitchell et al. (2019) at Google that has become a de facto standard for responsible AI model disclosure. A model card specifies: the model's intended use and out-of-scope uses, training data characteristics, evaluation results (with disaggregated performance metrics for subpopulations), limitations and known failure modes, and ethical considerations. Research institutions that release AI models, on Hugging Face, GitHub, or through research publications, should require model cards as a condition of release. Journals publishing AI research should require model cards as a condition of publication.

Datasheets for datasets, proposed by Timnit Gebru and colleagues (2018), provides analogous structured documentation for training and evaluation datasets. A datasheet documents: the dataset's motivation and creation process, composition (what data types, how collected, who collected it), collection process (who was involved, consent processes used), pre-processing steps, distribution and licensing, maintenance, and ethical considerations (potential harms, known biases). Research institutions that create and release datasets, including benchmark datasets used for evaluating AI research, should require datasheets as a condition of release.

Code availability and reproducibility requirements are increasingly mandated by journals and funders but inconsistently implemented. Research institutions should establish norms (ideally policies) that AI research code be deposited in institutional repositories or public archives at time of publication, with sufficient documentation to enable reproduction. This is a resource commitment, preparing code for public release takes significant effort, and research groups need support (from research software engineers, from library support, from computing resources) to meet reproducibility standards.

Environmental Ethics of AI Research

The environmental cost of AI research is an ethical issue that research institutions have been slow to grapple with, but that increasingly demands explicit consideration at the institutional level.

The carbon footprint of large-scale AI training is substantial and has been quantified in published research. Training GPT-3 (released 2020) was estimated to produce approximately 552 metric tons of CO2 equivalent, roughly equivalent to 60 transatlantic round-trip flights. More recent foundation models are substantially larger and correspondingly more carbon-intensive, though improved hardware efficiency has somewhat offset scale increases. Fine-tuning large models and running inference at scale also contribute meaningfully to carbon footprints. An AI research institution that runs significant training workloads is making material environmental choices through its research decisions.

Green AI is an emerging research and practice area that argues for making computational efficiency a primary research criterion alongside accuracy and performance. Key green AI principles for research institutions: (1) Report compute costs, every AI research publication should disclose the computational resources used, including GPU/TPU hours, energy consumption, and estimated carbon footprint, in the same way clinical research reports sample sizes and statistical power; (2) Consider efficiency as a research objective, optimization for computational efficiency should be valued in research funding and evaluation, not treated as a constraint after performance is maximized; (3) Use renewable energy, research computing infrastructure should be powered by renewable energy sources, and the carbon intensity of cloud computing resources should factor into provider selection.

The reporting requirements for AI energy use are evolving. The EU AI Act requires that providers of general-purpose AI models report energy consumption. Some journals and conferences (NeurIPS, ICML) have begun requiring or requesting energy use reporting in submissions. NIST's AI Risk Management Framework includes environmental impact as a consideration in responsible AI development. Research institutions that proactively develop energy reporting practices for AI research will be better positioned for emerging regulatory requirements and better able to make genuine commitments to environmental responsibility.

The equity dimensions of AI energy use add another ethical layer: the carbon costs of AI research are disproportionately borne by populations already most vulnerable to climate change, typically in low-income countries and communities, while the benefits of AI accrue disproportionately to high-income countries and populations. This is a justice consideration as well as an environmental one. Research institutions developing large-scale AI research programs should engage with this equity dimension as part of their ethical framing, not treat energy use purely as a cost or efficiency consideration.

AI Ethics Governance Structures

Translating ethical principles into institutional practice requires formal governance structures: bodies with defined authority, expertise, and processes for making ethics-relevant decisions about AI research.

AI Ethics Review Boards or Committees are being established at leading research institutions to provide oversight for AI research with significant ethical implications. These are distinct from IRBs, which review human subjects research under federal regulatory frameworks; AI ethics committees have a broader and less formally defined mandate. Effective AI ethics committee composition includes: a research ethicist (with AI ethics expertise specifically), domain experts relevant to the institution's AI research focus areas, legal counsel familiar with AI regulation, a community representative (someone who represents populations affected by AI, not affiliated with the institution), a technologist with AI expertise, a research administrator, and a graduate student or early-career researcher representative. The breadth of expertise is essential, an ethics committee composed only of researchers from the AI field will have blind spots that outside perspectives can address.

Review triggers for AI ethics committee consideration should be specified clearly, distinguishing what requires full committee review from what can be handled through expedited review or consultation. Triggers for full review might include: AI research expected to be deployed in high-stakes decision-making (healthcare, criminal justice, financial services, education); AI research involving sensitive population groups (children, people with disabilities, racial or ethnic minorities as the primary affected population); AI dual-use research; AI research with significant potential for large-scale harm; and AI research in partnership with organizations that have significant conflicts of interest in the research outcomes.

Advisory versus binding authority is a design choice for AI ethics committees that has significant practical implications. Advisory committees provide recommendations that researchers are free to accept or reject. Binding committees provide approvals or rejections that researchers must comply with. Advisory authority maximizes researcher flexibility and avoids the governance complexity of a binding process, but may be ineffective if researchers routinely override recommendations. Binding authority provides real governance but requires appeal processes, clear criteria, and significant institutional commitment to the committee's legitimacy. Many institutions are starting with advisory authority and evolving toward binding authority as the committee establishes credibility and develops clear criteria.

Community engagement as an ethics governance mechanism goes beyond formal committee review. Meaningful engagement with the communities most likely to be affected by AI research, including before research design is finalized, is an emerging best practice that some institutions are formalizing into their ethics frameworks. This is challenging to operationalize but may be the most important governance mechanism for ensuring that AI research serves the populations it purports to help.

Researcher Responsibility and Professional Ethics

Ethical AI research is not just an institutional governance question. It requires individual researchers to develop and exercise ethical judgment, and research leaders have a responsibility to cultivate ethical culture in their teams and departments.

Professional codes of conduct provide one foundation for individual researcher responsibility. The ACM Code of Ethics and Professional Conduct (updated 2018) is the most directly applicable to computer science and AI researchers. It establishes obligations to: contribute to society and human well-being acknowledging that all people are stakeholders in computing; avoid harm; be honest and trustworthy; be fair and take action not to discriminate; respect privacy; honor confidentiality; and assess the potential impacts of computing systems on society. The IEEE Code of Ethics similarly commits members to protect public health, safety, and the environment and to be honest and impartial in making ethical decisions. Research leaders should ensure that their graduate students and postdocs are familiar with the professional ethics code of their field, not as a compliance exercise but as a genuine foundation for ethical reasoning.

Whistleblower protections for AI ethics concerns are an important component of ethical culture that institutions need to develop explicitly. Researchers who observe AI practices they believe are unethical, biased systems being deployed without adequate testing, research data being used without proper consent, dual-use risks being ignored, need channels to raise concerns and protection against retaliation for doing so. Most institutional whistleblower policies cover research misconduct broadly but were not designed with AI ethics concerns in mind. Research leaders should ensure that AI ethics concerns can be raised through existing confidential reporting channels and that reporters are protected.

Collective responsibility versus individual responsibility is a philosophically and practically important distinction for AI research ethics. Large AI projects involve many people, data collectors, annotators, model developers, evaluators, deployment engineers, and the potential harms from AI systems are often attributable to decisions made collectively or in sequence rather than by any single individual. This distribution of responsibility can create collective action problems where each individual concludes that the harm is not their responsibility because so many others are also involved. Research leaders must actively cultivate a culture of collective responsibility, where every member of a research team understands their role in preventing potential harms and feels empowered to raise concerns, rather than allowing diffusion of responsibility to create ethical gaps.