Implementing Ethics for a Mobile App Deployment

IMPLEMENTING ETHICS FOR A MOBILE APP DEPLOYMENT

JOHN ROOKSBY, PARVIN ASADZADEH, ALISTAIR MORRISON, CLAIRE MCCALLUM, CINDY GRAY, MATTHEW CHALMERS

These are slides for a “GIST” seminar at the University of Glasgow (17 / 11 / 2016).

Pre-conference talk about OzCHI paper.

Full paper here: johnrooksby.org/publications.html

http://johnrooksby.org/publications.html

JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016


IN THIS TALK

▸ Case study of “implementing” or “operationalising” ethics in a deployment-based study.

▸ No claim that our work is perfect. Instead, an exploration of the problem of ethics:

▸ There is uncertainty and complexity in research ethics for deployment studies.

▸ Research ethics are enmeshed with system design in deployment studies.

▸ I will conclude that a “in action” view of research ethics is appropriate.


IN-ACTION ETHICS▸ “The realities of conducting the study can unexpectedly differ from

what the researchers have planned for”

Munteanu et al. Situational Ethics: Re-thinking Approaches to Formal Ethics Requirements for Human-Computer Interaction. Proc. CHI ’15.

▸ “While technology design has become explorative, situated and responsive, the accompanying ethics processes largely remain static and anticipatory.”

Frauenberger et al. In Action Ethics. Interacting With Computers 2016.

▸ Responsible HCI research “is not an absolute virtue but an attitude and practice”

Grimpe et al. Towards a closer dialogue between policy and practice: responsible design in HCI. Proc. CHI ’14.


DEPLOYMENT BASED RESEARCH

▸ Characteristics:

▸ A technology released “in the wild” (outside of a controlled environment), over a prolonged period.

▸ Beta/advanced prototype.

▸ Uptake is not necessarily via formal recruitment, and often there is little interaction between participants and researchers.

▸ Data is collected during use of the system.


DEPLOYMENT BASED RESEARCH

▸ Problems:

▸ People may be unaware they are participating in a study.

▸ Limited opportunity for communication between participants and researchers.

▸ It is difficult to anticipate and manage problems outside of a controlled environment.



QUPED

▸ Designed to encourage walking. Uses three behavioural change strategies:

▸ Self monitoring (step counts)

▸ Automated goal setting

▸ Social comparison


ETHICS OF QUPED

▸ Formal review processes

▸ Institutional review and Apple review

▸ Supporting autonomy

▸ Informed consent and withdrawal

▸ There are other ethical issues

▸ Risk - covered in the paper but not the talk

▸ Multiple other issues in ethics and responsible research

FORMAL REVIEWS

JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT

INSTITUTIONAL REVIEWS

▸ Until recently, adherence to ‘codes of ethics’ was the norm.

▸ Now, many institutions and funding bodies mandate a formal ethical review for any project involving human subjects.

▸ “Ethical Creep”

Brown et al. Five Provocations for Ethical HCI Research. Proc. CHI ’16.

JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT

INSTITUTIONAL REVIEWS

▸ ACM has long had a Code of Ethics

▸ ACM Conferences (such as CHI) moving towards mandating formal, institutional reviews:


INSTITUTIONAL REVIEW

▸ At the University of Glasgow, research with (adult, non-vulnerable) human participants is reviewed by a college ethics committee. There are four colleges, including:

▸ College of Science and Engineering.

▸ College of Social Science.

▸ The application forms/processes for the colleges are different.

▸ Science and Engineering has an orientation to lab studies, Social Science to field studies. Neither directly accommodates deployment research.

▸ Neither specifies what the ethical issues are.


INSTITUTIONAL REVIEW

▸ For Quped, as a multi-disciplinary team we had a choice of committee. We chose Social Science:

▸ Deployments more akin to fieldwork.

▸ Previous experience with applications showed an application to social science would be easier.

▸ We told Claire it would be a “good experience” for her to apply.


APPLE REVIEW

▸ App Store Apps are also reviewed by Apple, with their own criteria, e.g.:

▸ “Apps conducting health-related human subject research must obtain consent from participants or, in the case of minors, their parent or guardian. Such consent must include the (a) nature, purpose, and duration of the research; (b) procedures, risks, and benefits to the participant; (c) information about confidentiality and handling of data (including any sharing with third parties); (d) a point of contact for participant questions; and (e) the withdrawal process.”

▸ Two separate, independent and overlapping review processes.


FORMAL REVIEW: SUMMARY

▸ Multiple, partially conflicting guidelines and processes.

▸ Guidelines must be translated to specific study.

▸ Reviews are late in the day - the objective is to “pass” the review, not to consider ethical issues at this point.

▸ Different organisations have jurisdiction over ethics.

AUTONOMY


AUTONOMY

▸ The term autonomy comes from an ancient Greek term translating as “self law” (i.e. to be a law to oneself).

▸ In the context of research ethics, it means that an individual should be able to make an informed and free decision to participate in a study.

▸ Often seen as key for research ethics, however:

▸ Consent as severance from data. Luger and Rodden Luger, E. and Rodden, T. An Informed View on Consent for UbiComp. UbiComp ’13.

▸ Consent as limitation of legal liability.

Cairns, P. and Thimbleby, H. The diversity and ethics of HCI. Computer and Information Science ’03.


CONSENT

▸ For Quped, main recruitment mechanism is the app store. It is not possible to ensure people are informed about the study before the install.

▸ Cannot frame app as a research study in the app store.

▸ Cannot specify the app is for adults.

▸ Installation cannot be considered consent.

▸ Therefore, we need in-app mechanisms to gain consent.


CONSENT▸ Consent integrated with on-boarding process


CONSENT▸ In app information sheet appears after

description.

▸ No data uploaded until after ‘accept’

▸ A ‘decline’ button (just sends you back)

▸ Scrollable in textarea to log how far people view the text

▸ Under 18 switch

▸ Changed in an update to be ‘switch off is you are under 18’.


CONSENT▸ In app information sheet appears after

description.

▸ No data uploaded until after ‘accept’

▸ A ‘decline’ button (just sends you back)

▸ Scrollable in textarea to log how far people view the text

▸ Under 18 switch

▸ Changed in an update to be ‘switch off is you are under 18’.


CONSENT▸ Apple also requires the app to ask for

permissions.

▸ Pop up, ‘just in time’ permission requests.


CONSENT

▸ Information sheet also available in-app.


CONSENT▸ Scaffolded approach to

collecting demographic data

▸ This data not required to use the app, only for social comparison.

▸ Second opportunity for identifying non-adults

▸ If a user specifies under 18, they are excluded from the study.


WITHDRAWAL

▸ People should also be able to freely withdraw from a research study

▸ Withdrawing from a deployment somewhat different from with drawing from a lab study.

▸ Stopping using the app: people can do this whenever they wish.

▸ Pulling out of the study (i.e. data not to be included): we need to support this in-app.


WITHDRAWAL


DEBRIEFING

▸ Debriefing is seen as good practice in research studies

▸ Another opportunity to give information and ask questions

▸ However, with a deployment such as Quped there is no end.

▸ If a user deletes the app, there is no way to contact them (unless we collected extra data such as email address).

▸ Left debriefing as a design problem to address later.

▸ We can turn off logging after some point (12 months?)

▸ We could create a strategy based on whether the intervention does/doesn’t work.


AUTONOMY: SUMMARY

▸ Standard approaches to autonomy problematised in deployment research.

▸ Automation, with limited opportunity for discussion

▸ Changing nature of withdrawal and debriefing

▸ Consent and withdrawal part of the app design.

▸ Design changes were made after release, and some decisions put off until future.

DISCUSSION


UNCERTAINTY IN ETHICS

▸ No clear set of ethical rules and procedures, but multiple, diverging review processes and guidelines for us to navigate.

▸ The deployment is built upon opaque infrastructure that sits between the participants and us, which constrains and to an extent dictates what we can do.

▸ As HCI researchers we have a concern for naturalistic, ecologically valid data. This produces tensions with ethical concerns for informing people about the research.

▸ For long term deployments, some issues are clarified during the deployment. Therefore it can be appropriate to revisit ethical decisions.


ETHICS AS DESIGN

▸ Research ethics had to be worked out in anticipation of the ethical reviews during the design process. Problems with reviews may have entailed major changes to our design.

▸ We needed to create mechanisms for consent, withdrawal and data management that were a coherent part of an overall user experience. These needed to be present in the app, easy to understand and not off-putting.

▸ App design is iterative, and updates and changes can have implications for our ethics procedures.


CONCLUSION

▸ “While technology design has become explorative, situated and responsive, the accompanying ethics processes largely remain static and anticipatory.”

Frauenberger et al. In Action Ethics. Interacting With Computers 2016.

▸ Implications

▸ HCI should share and discuss design knowledge relevant to research ethics

▸ Ongoing support research ethics in the planning/design phases and post-release

▸ Reviews are valuable (in principle) but better support needed for deployment based research