PRIVACY INTERESTS
Editor: Tara Whalen, twhalen@pobox.com

The Menlo Report
Michael Bailey | University of Michigan
David Dittrich | University of Washington
Erin Kenneally | University of California, San Diego
Doug Maughan | Department of Homeland Security

R

esearch on rapidly advancing information and communication technology (ICT)
has exposed gaps between what
researchers could do and what they
should do, thereby creating ethical
challenges. Even seemingly benign
research involving botnet infiltration for observation purposes,
empirical analysis of fraud, or
behavioral research on anonymity
network users puts researchers at
an ethical crossroads.
To understand significant
Internet threats, researchers might
actively participate in a botnet by
relaying or responding to commands instructing compromised
bot machines to send malicious
email, initiate a distributed denialof-service attack, or propagate malicious software to other hosts. Such
active engagement lets researchers
discover the botnet code’s vulnerabilities, which enables the development of remote cleanup programs
that execute unauthorized commands on end users’ computers.
Or, it might stimulate publication
of the botnet code—instructions
that permit the manipulation
of hosts ranging from personal

computers to proprietary process
control devices, electronic voting machines, or medical devices.
Although researchers’ intent might
be to empirically prove that harm
in the virtual environment can
manifest as harm in the physical
environment, the potential harm
from improper disclosure could
be immediate and life threatening
if malicious attackers exploit such
knowledge before the vulnerabilities are remediated. Furthermore,
researchers often gain access to
drop zones—servers containing
sensitive stolen data, such as trade
secrets, bank account credentials,
personal communications, and
login credentials.
Experiments aimed at understanding Internet fraud dynamics,
or phishing, might require that users
be unaware they’re being studied
to observe their typical behavior.
Obtaining informed consent prior
to a study can defeat this purpose,
so deception might be necessary
to accurately simulate real fraudulent behavior. Researchers might
redirect users from an unsuspecting malicious website to a benign
site that they control and monitor.

1540-7993/12/$31.00 © 2012 IEEE

Copublished by the IEEE Computer and Reliability Societies

When informed consent is waived
and deception is involved, debriefing after the study’s completion
is typically required; however,
this might increase rather than
decrease harm to subjects because
their knowledge of being tricked
can cause shame or decrease their
trust in researchers’ actions.
Experiments intended to understand Internet use or measure
certain networks’ traffic characteristics require access to network
traffic. If researchers find it difficult to access the target network,
they might choose to leverage usersupported networks such as Tor
(the onion router), setting up entry
or exit nodes from which they can
collect data locally (including sensitive payload data). Such monitoring
might violate the network’s terms
of use, contravene network users’
expectations, and raise legal risks
about communication privacy.
Exploiting weaknesses in such
networks to gain access to data or
selecting subjects to avoid restrictions raises further questions.
In addition, it’s sometimes
unclear whether researchers consider the could or should courses
of action, and if so, whether their
actions are affected by opinions
about the gray applications of laws or
the relevance of institutional review
boards to research beyond the traditional behavioral and biomedical
realm. A variety of reasons attempt
to explain why Internet research
considers ethics less than other
fields do, including a deficiency of
shared values among this research
community, a shortage of individual expertise in formal ethical
decision-making, the inconsistent
March/April 2012

71

PRIVACY INTERESTS

application of principles, and a lack
of agreement on enforcement.1
How should we address
these shortcomings? The logical approach in the face of this
dilemma is to revisit first-order
ethical principles and applications.
Thus, the US Department of Homeland Security (DHS) Science and
Technology Directorate launched
an effort that resulted in the Menlo
Report, a document containing
ethical principles guiding ICT research.2

two volumes of appendices. 3,4 The
Belmont Report was republished
on its own in the Federal Register
the following year. This is a formal
mechanism for officially announcing a proposed rule or policy to
both government agencies and the
US public as well as a means for
soliciting comments before government actions.
The Menlo Report was created
under a less formal, grassroots

issues illuminated by the Menlo
Report. In this respect, the Belmont and Menlo reports both live
within larger bodies of work.

The Core Menlo Report

The working group determined
that the core report should reflect
the Belmont Report’s simple and
elegant structure, length, use of
language, minimal use of references, and focus. The Belmont
Report has only three
main sections: the differThe beneficence principle maintains
ence between research
Origins of the
and practice in the biothat researchers should avoid harm,
Menlo Report
medical setting, a set of
The Menlo Report builds
maximize probable benefits and minimize three basic principles,
on the Belmont Report
and the associated appliprobable harms, and systematically
and shares some procecations of those princidural heritage, such as
assess both risk of harm and benefit.
ples. There are only three
publication in the Federal
references, no definiRegister. However, the
tions of terms, and little
two processes aren’t equivalent.
process that was catalyzed by the background on the issue space. FolThe Belmont Report was the cul- narrower ethical issues raised in lowing this model, the core Menlo
mination of years of working meet- ICT computer security research. Report required more detail, backings by the National Commission Discussions at conferences and in ground information, case studies,
for the Protection of Human Sub- public discourse exposed growing assistive questions, and references
jects of Biomedical and Behavioral awareness of strong ethical debates to elucidate the nuanced language
Research, which began as part of in computer security research and and explain how ICT complicates
the National Research Act defined that existing US oversight authori- the interpretation and application
by Congress and signed into law by ties might have been unaware of— of the principles in the Belmont
President Richard Nixon in 1974. or might not have believed they Report.
Although the Belmont Report had a mandate for reviewing—
The Menlo Report briefly covstands alone in many researchers’ some of this research.
ers motivation and background. It
and ethics reviewers’ minds, it’s
“The Menlo Report: Ethical restates the Belmont Report’s prinpart of a larger body of supporting Principles Guiding Information ciples in light of changes brought
documents. During its multiyear and Communication Technol- about by advances in ICT. It then
deliberation process, 26 papers ogy Research” is the core docu- describes the application of those
were written that provide back- ment stemming from the series of principles in the ICT research
ground, scientific observations, and working group meetings that context. Specifically, the Menlo
recommendations on topics includ- broached these issues (see the Report details four core ethical
ing basic ethical principles relating “Menlo Report Working Group” principles: three from the original
to research involving human sub- sidebar). 2 Similar to the National Belmont Report—respect for perjects, boundaries between research Commission’s approach, the DHS- sons, beneficence, and justice—and
and practice, risk/benefit crite- supported working group meetings an additional principle—respect
ria, and informed consent. At the considered previously published for law and public interest. The
same time that the Department of and unpublished documents when report explains each of these in the
Health, Education, and Welfare developing the Menlo Report. This context of ICT research. Respect
first published this concise state- process inspired the production for persons requires that
ment of ethical principles in 1978, and dissemination of other docuit also published the hundreds of ments intended to supplement and ■ research subject participation
pages of papers created by and for clarify some of the more complex
is voluntary and follows from
National Commission members in and controversial aspects of the
informed consent,
72

IEEE Security & Privacy

March/April 2012

■ individuals are treated as autonomous agents, and their rights to
determine their own best interests are considered and protected,
■ the interests of individuals who
are impacted by but not targets
of research are guarded, and
■ individuals
with
diminished autonomy and decisionmaking capabilities are afforded
protection.
The beneficence principle maintains that researchers should

Menlo Report Working Group

T

he US Department of Homeland Security hosted a two-day workshop on 26–27 May
2009 in Washington, DC, which brought together ethicists, institutional review boards,
researchers, and lawyers to discuss ethical issues in information and communication technology (ICT) research. The desired outcome of this meeting was a set of ethical guidelines that,
though anchored off the original Belmont Report framework, reflects the unique questions
facing ICT researchers. The subset of participants charged with creating this report became the
Menlo Report working group, which subsequently held meetings over a 16-month period to
create the report. The Menlo Report working group participants are
■
■
■

■ avoid harm,
■ maximize probable benefits and
minimize probable harms, and
■ systematically assess both risk of
harm and benefit.

■
■
■
■
■
■

Justice entreats that

■
■

■ each person deserves equal consideration in how to be treated;
■ research benefits should be distributed fairly according to
individual need, effort, societal
contribution, and merit; and
■ subject selection should be fair,
and burdens should be allocated equitably across impacted
subjects.
The principle of respect for law
and public interest, similar to the
Belmont Report’s conception of
beneficence, explicitly charges
researchers to
■ engage in legal due diligence,
■ be transparent in methods and
results, and
■ be accountable for actions.
To sufficiently address all the
principles and their applications,
the report stresses the need to
understand the relevant stakeholders. These include but aren’t
limited to ICT researchers, at-risk
humans (be they research subjects, nonsubjects, or simply ICT
users), malicious actors, network/
www.computer.org/security

■
■
■
■

Michael Bailey, University of Michigan
Aaron Burstein, University of California, Berkeley
KC Claffy, University of California, San Diego
Shari Clayman, Department of Homeland Security
David Dittrich, University of Washington
John Heidemann, University of Southern California
Erin Kenneally, University of California, San Diego
Doug Maughan, Department of Homeland Security
Jenny McNeill, SRI International
Peter Neumann, SRI International
Charlotte Scheper, RTI International
Lee Tien, Electronic Frontier Foundation
Christos Papadopoulos, Colorado State University
Wendy Visscher, RTI International
Jody Westby, Global Cyber Risk

platform owners and providers,
government and law enforcement,
and society at large. Such differentiation facilitates balancing benefit
and harm, which is more complicated than simply maximizing all
benefits and minimizing all harms.
Part of this process involves balancing the competing interests of
stakeholders who all deserve consideration and protection.

The Menlo Report
Companion

The Menlo Report’s major appendage is “Applying Ethical Principles
to Information and Communication Technology Research: A
Companion to the Department
of Homeland Security Menlo
Report.”5 At 37 pages, the companion document is still significantly less information than the
Belmont Report’s two volumes of

appendices, but its content, references, and case studies similarly
reflect working group deliberations and provide crucial adjunct
details about a complex topic.
The companion document wasn’t
published in the Federal Register;
however, it accompanies the core
Menlo Report on the DHS website.
The companion goes into greater
depth on the Menlo Report’s history and relationship to the Belmont
Report. It expands on the application of principles, including providing assistive questions that help
design and evaluate research in conformance with the stated principles.
It uses a single synthetic case study
to illustrate a broad range of ethical
issues and contains four appendices
with reference material supporting
both the companion and core documents, including a representative
set of case studies and references to
73

PRIVACY INTERESTS

source materials covering industry
and academic research.

Other Supporting
Activities

In January 2010, an early version
of the Menlo Report’s principles
and applications introduced an
ethical impact assessment (EIA)
framework intended to facilitate
ethical ICT research design and
evaluation.6 In addition, working
group members have participated
in multiple panels, workshops,
and presentations at which the
Menlo Report and related topics
were discussed. These include the
2010 Workshop on Ethics in Computer Security Research; the 2010
Symposium on Usable Privacy
and Security; the 2010 Network
and Distributed System Symposium; the Public Responsibility
in Medicine and Research’s 2011
Social, Behavioral, and Education Research Conference; the 1st
International Digital Ethics Symposium; the 2011 Honeynet Project Public Day; the 2011 Annual
Computer Security Applications
Conference; and the 2011 AntiPhishing Working Group Meeting.
Feedback from these and other
interim publications have helped
frame the Menlo Report and its
companion document, contributed
insight for the evolving EIA, and
spurred iterative engagement with
the community to advance dialogue in this issue space.
After internal discussion and
two rounds of reviews by a representative population of ethicists,
practitioners, academics, and
industry, the core document was
published in the Federal Register
on 28 December 2011,7 the first
at-large public release for comments on the proposed principles
and applications.

Moving Forward

The Menlo Report attempts to
lower the barrier to entry for
74

IEEE Security & Privacy

researchers and oversight entities dealing with computer security research ethics. As such, it’s
an important first step, but it represents only part of the needed
community response to existing
ethical challenges.
The security research community and the larger ICT research
domain (for example, network measurement, computer-human interface, and software engineering)
lack shared community values,
those guiding principles around
which we can assess, systematize,
influence, and justify research
conduct.1 The growth and persistence of debate among relevant
conference program committees
over the ethical propriety of certain
research exemplify this disharmony.
Although agreement on core principles might not be uniform, the
larger challenge lies in galvanizing
the principles into coherent applications and implementations.
In addition, the community
faces a dearth of domain guidance
and technical enablers to translate
the abstract and theoretical ethics
principles into practicable actions.
Specifically, there is a lack of formal
institutional and ad hoc peer guidance in ethical decision-making,
thereby reinforcing the vacuum in
which first-order ethics principles
are embraced at the community
level. Further, assuming the existence of guidance, researchers
are in want of tools that embed,
consistently reproduce, and scale
such expert ethics advice. An
ethics-by-design strategy for
computer security researchers
demands tools that operationalize the defining principles.
Finally, there is a shortage of
forcing functions for implementing the applications of ethics
principles. Specifically, although
institutional review boards have
largely shouldered the mandate to
ensure ethics in research involving
human subjects, many question

their relevance and capabilities
in computer security research.
And it’s unclear the extent to
which other institutions, such as
conference program committees
or funding agencies, are able or
willing to provide the oversight
and quality control necessary to
ensure that ethics issues are identified, applied, and evaluated in
research endeavors—let alone to
do so consistently.

T

he Menlo Report and its companion document attempt to
galvanize ethics principles and
their applications. This path forward is one recipe to enable the community to embrace a self-regulatory
approach to embedding ethics in
research by way of a more mature
and community-built notion of
what is ethically defensible.
An alternative is to wait for
an unfortunate event to trigger
hasty, top-down mandates that
won’t likely reflect this community’s input.
References
1. D. Dittrich, M. Bailey, and S. Dietrich, “Building an Active Computer
Security Ethics Community,” IEEE
Security & Privacy, vol. 9, no. 4,
2011, pp. 32–40.
2. “The Menlo Report: Ethical
Principles Guiding Information
and Communication Technology Research,” D. Dittrich and E.
Kenneally, eds., US Dept. Homeland Security, 2011; www.
c yber.st.dhs.gov/w p-content/
uploads/2011/12/MenloPrinciples
CORE-20110915-r560.pdf.
3. Nat’l Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, “The
Belmont Report: Ethical Principles
and Guidelines for the Protection
of Human Subjects of Research,”
appendix vol. 1, Dept. Health,
Education, and Welfare Publication no. (OS) 78-0013, 1979;
March/April 2012

http://v ideocast.nih.gov/pdf/
ohrp_appendix_belmont_report
_vol_1.pdf.
4. Nat’l Commission for the Protection of Human Subjects of Biomedical and Behavioral Research,
“The Belmont Report: Ethical Principles and Guidelines
for the Protection of Human
Subjects of Research,” appendix vol. 2, Dept. Health, Education, and Welfare Publication
Publication no. (OS) 78-0014,
1979; htt p://videocast.nih.gov/
pd f/ohr p _append i x _bel mont
_report_vol_2.pdf.
5. “Applying Ethical Principles
to Information and Communication Technology Research:
A Companion to the Department of Homeland Security
Menlo Report,” D. Ditt rich and
E. Kenneally, eds., US Dept.

Homeland Security, 2011; www.
c yber.st.dhs.gov/w p-content/
upload s/2 012/01/Men lo
PrinciplesCOMPA NION-2012
0103-r731.pdf.
6. E. Kenneally, M. Bailey, and D.
Maughan, “A Tool for Understanding and Applying Ethical
Principles in Network and Security Research,” Workshop Ethics in Computer Security Research
(WECSR 10), Springer, 2010, pp.
240–246.
7. “The Menlo Report: Ethical
Principles Guiding Information
and Communication Technology
Research,” Federal Register, vol.
76, no. 249, 28 Dec. 2011, p. 81517.
Bailey is an assistant
research scientist at the University of Michigan. Contact him at
mibailey@eecs.umich.edu.

David Dittrich is a senior secu-

rity engineer and researcher at
the University of Washington.
Contact him at ditt rich@u.
washington.edu.

Erin Kenneally is an information

technology law specialist at the
Cooperative Association for
Internet Data Analysis, University of California, San Diego.
Contact her at erin@caida.org.

Doug Maughan is division direc-

tor, Cyber Security Division, at
the Department of Homeland
Security Science and Technology Directorate. Contact him at
douglas.maughan@hq.dhs.gov.

Michael

Selected CS articles and columns
are also available for free at
htt p://ComputingNow.computer.org.

Richard E. Merwin Student Scholarship
IEEE Computer Society is offering $40,000 in student scholarships from $1,000 and up
to recognize and reward active student volunteer leaders who show promise in their
academic and professional efforts.

Who is eligible? Graduate students, and those in the final two years of an undergraduate program in electrical or
computer engineering, computer science, information technology, or a well-defined computer related field. IEEE
Computer Society membership is required. Applicants are required to have a minimum grade point average of 2.5
over 4.0, and be a full-time student as defined by his or her academic institution during the course of the award.

APPLY NOW — APPLICATION DEADLINE IS 30 APRIL!
www.computer.org/scholarships
For more information, see the above link or send email to:

jw.daniel@computer.org

Current IEEE students can join IEEE Computer Society
for as low as $4.00 USD. Go to
ieee.org/join, select IEEE Society Memberships

www.computer.org/security

75