CUTGroup Detroit

Agenda

Meet the teamEstablishing CUTGroup DetroitRecap of first CUTGroup test

Review of questionsProctor feedback

Next steps

Meet the Team!• Smart Chicago: Sonja Marziano, CUTGroup Director and Project

Coordinator• Microsoft Chicago: Shelley Stern Grach, Director of Civic

Engagement, and Ivoire Morrell, Civic Tech Fellow• City of Detroit: Garlin Gilchrist, Detroit Director of Innovation &

Emerging Technology, and Joel Howrani-Heeres, Director of Open Data and Analysis, Department of Innovation and Technology

• Data Driven Detroit (D3): Erica Raleigh, Executive Director, Noah Urban, Project Lead & Senior Analyst, Kibichii Chelilim, Data Manager & Programmer, Boitshoko Molefhi, MSU InnovateGov Summer Intern, Meghin Mather, Operations Coordinator, and Ayana Rubio, Data Analyst

Quick Recap of CUTGroup Test

Pre-Game!

Establishing CUTGroup Detroit: Website Build• Targets users of all technical

backgrouds• Simple form allows Detroit

residents to sign up for CUTGroup Detroit

• Captures addresses for gift card mailings

• http://www.datadrivendetroit.org/cutgroup/

CUTGroup Detroit Recruitment

Dual Recruitment Strategy: Street & Web

Street Team: Traveled to three different Detroit communities and distributed fliers to residents in those communities. Mapped out organizations to target for recruitment, formed a team, and assigned different roles to the team (social media manager, photographer).

Social media: Provided an intimate experience of the team’s recruitment efforts through live tweeting, Facebook posting, and blogging. Partnered with @D3detroit’s social media team to promote the initiative on social media.

Recruitment Week 1: Southwest Detroit

Recruitment Week 2: Midtown

Recruitment Week 3: Downtown Detroit

Recruitment Week 4: Downtown Detroit

Recruitment Results

• 110 testers after one month of recruiting

• 4 blogs on adventures in recruiting• Prominent social media presence:

135 Twitter followers• Location secured for first test: Ford

Research & Engagement Center• Currently: 233 CUTGroup testers

Session Recruitment byMailchimp & Wufoo

Mailchimp and Wufoo for Signups

Two invitations, each linked to an availability form.

Emailed 10 and 6 days before the test.

CUTGroup Test : City of Detroit’s

Commercial Property Tool

Brief Recap13 testers participated throughout

the evening in 1-on-1 proctored user tests.

For all testers & proctors this was their very first CUTGroup test!

85% of testers most often connect to the Internet using a smart phone.

38% of testers had never searched for information about a commercial

property or business.

Brief RecapThings that went really well• Diverse group• Most testers showed up in their confirmed time slot• All testers were excited about the property tool• Useful feedback on both mobile and desktop devices

Things that happened that we weren’t expecting• Not a single “no-show”, which caused it to be busy at

times• One tester’s concerns with privacy led to no recording• Proctoring on a mobile device was difficult

Brief Recap• City of Detroit Non-Residential Property Tool• Developed by Joel Howrani Heeres• Gives Detroit residents information about the

current status of a non-residential properties including city department ticketing, ownership, demolition, and other essential information

Why use CUTGroup?

“We would like to use the CUTGroup to help us determine which aspects of the Commercial Property tool users like and don't like and which features it doesn’t have. The objectives for the application would be

that people who like to know the current status of a given building as regards

city department ticketing, ownership, demolition, and other processes can find

out what they need to find out.”-- Joel Howrani Heeres, Manager,

Director of Open Data and Analysis in Detroit’sDepartment of Innovation and Technology

Capturing User FeedbackTo capture feedback, a Wufoo questionnaire was formulated based on the following areas:

• Background Questions: Questions targeted to gain a better understanding of how the tester would use the developing application   

• Usability Testing: Questions that are based around the application’s usability (how it works, what should be changed with the user interface, features to add/remove)

• Card Sorting: Questions to gain insight on the features and content desired most by testers

Commercial Property Tool Question Review

First Impressions - ReviewLaptop Testers – 9 total

• Several testers did not recognize Detroit City Council District boundaries, and were confused by them.

• 4 testers said the spatial navigation was self-explanatory and felt familiar.

Mobile Testers – 4 total

• 2 testers weren’t able to load the page on the first attempt, a third switched to a tablet from a phone because the tool was loading so slowly.

• 2 testers noted the familiarity of maps as a tool, and felt that the spatial navigation functions were intuitive.

Ease of Use in TasksHow easy was it to find a property with multiple inspections?

5 – Very Easy 46% (6)4 – Easy 15% (2)3 – Neutral 15% (2)2 – Difficult 23% (3)1 – Very Difficult 0% (0)

How easy was it to find the the closest property to Pinegrove Park that is scheduled for demolition?5 – Very Easy 38% (5)4 – Easy 15% (2)3 – Neutral 8% (1)2 – Difficult 23% (3)1 – Very Difficult 15% (2)

Overall Review – Ease & Fit

Yes46%

No54%

Do you think this type of property tool is targeted to

you?

Very Dif -ficult15%

Difficult23%

Neutral8%

Easy15%

Very Easy38%

Overall, how easy do you think it is to use this property

tool?

Q: Why not?Typical A: Seems better for planners,

developers, people who work in office environments.

Commercial Property Tool Improvements

Landing Page

“It's a little disorienting because I'm looking at the borders of what I'm guessing are the neighborhoods and they're easy to get confused with the streets without looking at the names…Things are kind of blending in the background and there's just a lot of lines.”

Unfamiliarity with Detroit City Council Districts was a hurdle for several testers. Others appreciated the familiarity of the map

format and the ease with which they could spatially navigate by using the mouse.

Search for a property…Search results were inconsistently generated when…

- Testers attempted to search by keyword or business name- Testers did not enter “Detroit, MI” at the end of a query

Testers were confused by…- Point data: several expected to see polygons- The frame on the right: it wasn’t clear to them that they

could click inside the frame to discover additional information

Testers appreciated…- The amount of data available on a given property- Street view confirmation of what they were looking at

Design ElementsTesters were confused by:

• Data point icons that decreased in size when zooming in

• The color codings of various data points

• Internal navigation within the records pane

• Expectations around polygons vs points

Proctor Feedback

Proctor Feedback

This was a new experience for Proctors, too!

A questionnaire can be developed for the next test, to be completed at the end of every user test session.

Next Steps

User-recommended updatesMake navigation more user-friendly- A “how-to” slideshow with screenshots - Instructions/faq for searching- Index of property-related terms - Key or Legend

Increase visual distinctions- Delineate neighborhoods/zip codes - Include street views for all properties- Visually mark bldgs, not data entries - Highlight property boundaries when

selected

Card Sorting: Data Sets of Interest

Blight Violations11%

Building Characteristics14%

Building Licenses3%

Demolitions5%

DFC Future Land Use5%

Environmental Testing Results8%Building Inspections

11%

Insurance5%

Ownership19%

Priority Neighborhoods & Commercial Corridors

5%

Tax Payment Status8%

Zoning5%

Final ReportSynthesis and analysis of test results.

Assembling the report has informed our perspectives on test protocol.

Next Steps

❏ Publish final report

❏ Continue building the tester pool

❏ Evaluate and plan for future tests

Thank you!