The data ethics emergency drill (DEED) is principallly similar to a fire drill. However, instead of understanding roles when evacuating a building, data and AI team play out a scenario of a fictional issue with their data science/ML/AI system that has ethical implications and develop an understanding of their responsible AI practice. For example, a data ethics emergency could involve a hiring algorithm that, having been exclusively tested on young individuals, demonstrates discriminatory behavior against older demographics once deployed. How would the team discover, remediate and address the problem? The DEED’s main goal is to help technologists reflect on the ethics of their applications in a safe space and practice having conversations around ethical concerns and doing this in a way that is relevant to the context of the specific system they are building. This project is part of Vanessa Hanschke’s PhD research at the University of Bristol. You can read our CHI paper to find out more about how we designed and evaluated the DEED.
Craft a scenario
Brainstorm and draw up what a data ethics dilemma would look like for your team with the help of our workshop template.
Conduct the drill session
Get a group of people together in a meeting and expose them to your scenario.
Reflect on the session
Think about the experience, what you learnt and what changes you would like to see in your team processes, your role or in yourself.
The Toolbox
What does the DEED toolbox include?
A PowerPoint slide deck and Miro template for crafting your drill scenario
Directions on how to successfully run a drill in your company
We will be happy to assist you in your DEED journey
FAQs
How much time will these activities take?
The activity can be split into two workshops of at least an afternoon each:
(1) A workshop to craft a drill scenario specific to your system (min one person)
(2) A workshop with different team members to act out the drill in a meeting (min 3 people)
All activities can be extended on request depending on the context. After these each participant will be required to reflect on their experience with a short survey.
Does this need to be in person?
No, it can be conducted entirely online but can also be done in person.
What does a team get out of this?
Lots! They get to explore what problems their current processes might have, uncover difficulties they have not anticipated, discover roadblocks which need to be removed or find possible solutions to be prepared for in case of an actual ethics emergency. It also makes for a great team-building activity.
Why is it called a drill?
Telling someone about the evacuation plan for a fire emergency is not the same as testing the plan in a fire drill of the building. A fire drill can give you a clearer idea of how long it will take for people to react and if they will know how to move as a group with a sense of urgency. This is the experience we are trying to recreate in our data ethics emergency drills. Unlike common responsible AI tools such as frameworks and checklists, the DEED is embedded into the workplace context to give you a safe space to test out your responsible AI practice.
What will the researcher get out of this?
I am interested in how the participants’ interaction shapes solution finding and the general types of problems and solutions that come up. I know that poking at the internal processes can be a sensitive topic, however, I am not interested in the specific issues of a participating team. Rather I am interested in the effectiveness of the method and how to improve the design and its replicability.
Contact Us
We are looking for teams to test our toolbox.
Register your interest via this form
or
ask us any question via email at deed-study@bristol.ac.uk