Upload
leonardo-madariaga
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
Leonardo A. Madariaga MS Human Factors Engineering
Committee: Daniel Hannon - Mechanical Engineering-Tufts UniversityChris Rogers - Mechanical Engineering - Tufts UniversityEthan Danahy - Computer Science- -Tufts University
of the physical environment
in Makerspaces:exploratory study
Implications for the d
esign and
user evaluation of a n
ew workstation-‐syste
m
April 2015
Discussion
Question
Background
Method
Results
AgendaExploratory Study of the Physical Environment in Makerspaces: Implications for the design and user evaluation of a new workstation-system
Problem solving and innovationBackground
Top Down
Bottom Up
People/Places
Events
Vision Resources
Technologies
Some examplesBackground
(Photo by Jonathan Dietz, used with permission)
MIT Museum Makerspace
Background Some examples
(Photo by Jonathan Dietz, used with permission)
Malden High School
Background Some examples
(Photo by Jonathan Dietz, used with permission) NUVU Studio
Background
(Photo by Jonathan Dietz, used with permission)
Some examples
NUVU Studio
Background
(Photo by Author)
Some examples
Artisans Asylum
“Maker spaces support learning in an informal, play-focused environment that can cultivate an interest in science, technology, and design.”Scott Nicholson, associate professor in the School of Information Studies, Syracuse University, NY, 2012
“Despite a flurry of interest and activity around designing and creating Makerspaces, we still know little about the content and processes of learning in Makerspaces”Hakverson & SheridanHARVARD EDUCATIONAL REVIEW, 2014
CommunityShops
LibraryMakerspaces
MuseumMakerspaces
HomeShops
HobbyClubs
CommercialMakerspaces Making
EducationalMakerspaces
MachineWoodShops
Science Labs
Computer Labs
Collaborative spaces
VideoConference
DesignThinking
Educational
50´s 60´s 80´s - 90´s 00´s
A brief timelineBackground
Market analysis: WorkstationsBackground
LocusStanding DeskUS$1400
Ergotron WorkFit-DSit-Stand DeskUS$670
Jesper Office Height-adjustable Standing DeskUS$399
RefoldCardboard standing desk$250
D-School/ Stanford UniversityPeriodicTableUS$493
Market analysis: Storage & Organization systemsBackground
Really GoodStudent BinOrganization rackUS$180
i-RoverIPT102 7 Device Tech CartUS$320
D-SchoolStanford UniversityMobile StorageUnitUS$923
Copernicus3D PRINTINGStationUS$486
CopernicusSTEM MAKERStationUS$599
Paper/Book Author Findings Physical Environment/Usability
The Philosophy of Educational Makerspaces Part 1 of Making an Educational Makerspace
Kurti, Kurti and Flemming (2014)
Flexible by providing open spaces with elements that can capture users attention the first time they visit a space
A Design Case Examining Learning in the d-lab Leigh et al. (2013)
Transparency achieved through visual and acoustical “access” to activitiesEnhanced interconnectivity and socialization spaces inviting collaborationSpatial plasticity through the movability of furnishings
A Multi-disciplinary Design Environment
Carlson and Sullivan (2006)
Providing team storage lockers to organize tools and projects supplies. Larger and flexible storage areas would be desirable features.Architectural concepts, such as flexibility and visibility, should inform most of the design decisions of design lab.
Development of a Multidisciplinary Engineering Learning Center Batill and Gedde (2001)
A flexible and adaptable space. (b) Work areas and benches for project fabrication tools, test and assembly. (c) Storage space for students (d) Movable, student operated equipment (e) Project development and set-up area
Ergonomics product development of a mobile workstation for health care
Toivonen et al (2011) Two Usability tools were used: VAS (Visual Analogue Scales) and System Usability Scale. (SUS). (N=8)The study was able to define 19 design criteria for improvement of the product.
A usability evaluation of workplace-related tasks on a multi-touch tablet computer by adults with Down syndrome
Kumin (2012) Well-defined tasks can be used to account for user error, time to completion and rating of difficulty performing the task with specific devices. (N=10)
Flexibility/Mobility/Display/Storage/Project focused
Literature review
Usability testing/ Well-defined Tasks
Background
How can a Makerspace be more effective for managing larger amounts of projects and people at
the same time?
With limited (shared) spaceWith limited time (class 1/class2..)
With limited number of people to manage the space
Physical Environment
Question
Physical Structure
Symbolic artifacts
Physical stimuli
Open VS Closed OfficesSitting Vs Standing
Formal Meeting Space Vs CasualNatural Vs Artificial Lighting
Incoming email/notes on a wallSecurity signsSmell of coffee
Color of the wallsStyle of furnishing
Logotypes / Images
(Davis, 1984)
ObservationWorkflow& Design
GoalsPrototype
Tufts Tisch/CEEO Jumbo´s Makerstudio/Arts & Crafts Center/
Cambridge Friends/Malden High School
Modelling a general sequential
worklfowDefining design goals
and specifications
DIY-DesignExperimental unit
Contrast with Market
Usability Testing
SimulateSet Up
Design activityClean UpStorage
Method
Classroom needs / College / School
• How do people use the physical environment of Makerspaces?• How does the physical environment help Makerspace users achieve their goals?
Qualitative research method, such as shadowing were taken into account for note taking and understanding actions of users in context (McDonald, 2005)
ObservationMethod
(Photos by author)
Enter
Check on-going activitiesRead announcements
Leave personal belongings near work spotRetrieve on-going project from storage space
Set-up
Gather supplies/tools
Start design
prototyping
Mobility around space
Clean-up
Leave
Functional perimeter
with tools and materials
Other central
workstations
Clean work surfaceDiscard scrap
Recycle
Storage
Retrieve materialsCheck availability of toolsCheck availability of equipment
Leave on-going project and materialsin process in storage area
Bring personal belongings
Collaborationwith peers
Discussion ofproblems
SequentialWorkflow
MakerspaceUsage
User
Initial concept
The on-going project is the center of the making process
Shift from “stationary tables” to a “dynamic set of surfaces”
Allow to free up space
Improve storage / organization / set up / clean up
Method
(Photos by author)
Product goals, specification and prototypingMethod
(Photos by author)
Product goals, specification and prototypingMethod
(Drawings by author)
Below Market average (US$ 642)Cost per tray US$25
Does not consider Design/Assembly hours
Product goals, specification and prototypingMethod
Below Market average (US$ 350)Does not consider Design/Assembly hours
(Drawings by author)
Experimental prototypeMethod
(Drawings and Photo by author)
Tray RACKMovable Workstation
Researcher
Usability testing
Tray Rack
Materials rack and 3D printer
EntranceDisposal
Can
Workplace
Setting A: Baseline condition
Setting B: Experimental
Experimental layout
Method
•30 Participants (17Female/13Male)
•Age: M=24.6, SD=7.9
•18 sessions, spread over 9 days
•24 in groups of 2
•6 individually
•Performed a series of task using both
settings, two design activities
•Repeated Measures Design(Stangor, 2004)
•System Usability Scale(Brooke, 2996)
•User Experience Questions(Toivonen, 2011)
Usability testingMethod
Study was approved by Tufts University Institutional Review Board Nº 1501002
Task workflow
Usability testingMethod
PredictionsMethod
Increased System Usability Score (SUS)
Easier Storage & Organization
Adjective scale SUS SD
Worst Imaginable 12.5 13.1
Awful 20.3 11.3
Poor 35.7 12.6
OK 50.9 13.8
Good 71.4 11.6
Excellent 85.5 10.4
Best Imaginable 90.9 13.4
Adjective Scale(Bangor et Al, 2009)
Experimental (B)Standard (A)
Standard (A)
Experimental (B)
Faster Exitfrom Makerspace
Time<4 min
Subjective Eval.Subjective Eval.
Set Up/Clean Up
Usability testingResults
Open ended answersResults
Worksurface was equal in both settings (986 sq inches)Sitting still preferred for some detailed prototyping
Familiarity: Baseline condition is present in the majority of Makerspaces
(Coding according to Qualitative Data Analysis, Miles and Huberman, 1994)
Open ended answersResults
Lack of storage spaceShifting positionAccess to powerClean up: Male
Projector size didn´t free-up space
8%#
31%#
8%#
31%#
15%#
8%#
0%# 0%# 0%#
8%#
0%#
41%#
0%#
41%#
0%#
12%#
6%# 6%#
12%#
18%#
0%#
5%#
10%#
15%#
20%#
25%#
30%#
35%#
40%#
45%#
Limited#Space#
Backpack#storage#
Materials#rolling#off#
Shi@#posiAon#
Clean#up#
Access#to#power#
Difficult#collaboraAon#space#
SiIng#
Projector#connecAon#
Projector#size#
Most%difficult%aspects%of%se0ng%A%
MALE#
FEMALE#
Results Open ended answers
Storage directly in the project-trayFree up worksurface with small projector
Improved mobility: gather materials, supervise equipmentDevice-centered users require easy access to power
Results Open ended answers
The project-tray felt “less spacious”Middle separation was perceived as a “separator”
Standing positionThe amount of devices can provoke issues with cords
15%$
8%$ 8%$ 8%$
38%$
8%$ 8%$
0%$ 0%$
8%$
0%$
35%$
12%$ 12%$
47%$
6%$
0%$
6%$ 6%$
18%$
0%$5%$10%$15%$20%$25%$30%$35%$40%$45%$50%$
Tangled$cords$
Middle$separa:on$
Needs$seat$
Tray$access$
Spaciousness$
Locking$Mechanism$
Weight$resistance$
SetEup$
Backpack$support$
Standing$
Most%difficult%aspects%of%se0ng%B%
MALE$
FEMALE$
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Setting A can be qualified as between “OK” and
Good”, whereas setting B can be qualifies as between
“Good” and “Excellent”. (Bangor et al, 2008)
Usability testingResults
Usability testingResults
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Both groups scored setting B with significantly higher scores. Less than 1 year: : t (15)=-3.0365, p=0.0042
More than 1 year: t (13)= -2.0812, p= 0.0289
People working individually (N=6)did not evaluate significantly the workstation as more usable.
People working in groups (N=24) displayed significant higher evaluation of experimental
workstation
Usability testingResults
Gender-wise difference in evaluation.
Male participants scored significantly higher the new system, t(12)=-4.6209 p= 0.0006.
Female participants didn´t reveal any difference with statistical significance, t (16)=-1.6753,
p=0.1133.
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Usability testingResults
Nº Statement Male Female Diff. P-value(M diff F)
1 I think that I would like to use this system frequently. 4.54 3.94 0.597 0.0305
2 I found the system unnecessarily complex. 2.15 2.24 0.0814 0.789
3 I thought the system was easy to use. 4.31 3.76 0.5429 0.075
4 I think that I would need the support of a technical person to be able to use this system.
1.94 1.54 0.402 0.161
5 I found the various functions in this system were well integrated 4.31 4.24 0.072 0.817
6 I thought there was too much inconsistency in this system 1.94 1.62 0.32 0.252
7 I would imagine that most people would learn to use this system very quickly.
4.61 4.11 0.497 0.055
8 I found the system very cumbersome to use. 1.69 1.941 0.2488 0.305
9 I felt very confident using the system. 4.384 3.823 0.561 0.037
10 I needed to learn a lot of things before I could get going with this system.
1.69 2 0.307 0.188
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Participants within the average group (N=17) scored setting B with a higher
difference in terms of its Usability. (Makes sense: design for average)
Participant in the “extremes” (N=13) with either low or high standing elbow height
just reported a 1.1 points lower score in terms of Usability.
Usability testingResults
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
User experience: storage & organization
Hypothesis 2: Better storage and organization of projects
Expected value for the workstation system was higher than expected (Statistically significant difference: t (29)= -16.0709, p<0.001)
No gender-wise difference in evaluationStrong perception that the system would make it easier to store and
come back to a project
Results
User experience: storage & organization
Statistically significant , t(29)= -5.3866, p<0.001.
Positive evaluation of the user experience organization of projects
No gender-wise difference in evaluation
Results
Hypothesis 2: Better storage and organization of projects
Usability testingResults
No statistical significance in differences between timings
Set UP Clean UP
Usability testing
Is it faster to exit a Makerspace with Setting B?
Significant difference between means
t(29)= -10.9958, p<0.001
Results
Results Selected quotes
“I really liked the storage idea. For many Makerspaces, students come and go as they need to, the
hours are less regular. It's so crucial to have a place that is all my own that I can access very quickly
and easily. Having the shelves turn into work stations is a great solution to that. / / Also, in this
environment, when a maker is moving from machine to machine or back to their computer for
adjustments or checking the various projects going on, they are very active. It is much more practical
to be standing up at the standing up at the workstation so moving around and switching tasks is
easier”
Valued aspects: Storage/Mobility/Standing/Projector size and loca6on
Usability study met expectations. Further research is needed to understand why female scores did
not display significant differences. Adoption rate of the new workstation could be more dynamic, not
depending on the level of expertise.
Set up and clean up times did not display significant differences, although subjective perception
supports the idea that exit can be faster, storage and organization easier.
Sense of spaciousness was lower in experimental setting, even though both work areas were the
same. The presence of “edges” and “central division” is probably related to this
Most Valued aspects:
Mobility in the Makerspace: The project-tray decompresses the central zones allowing users to
move around using a functional perimeter.
Storage and organization of materials and projects: During the design and prototyping process
materials and/or projects won´t get mixed up, because the project-tray allows each user to
manage his/her own working surface.
Results Summary
Simplification of the design task due to: time constraints of the experiment, fixed set of materials.
Results Limitations
Carryover effects: Difficult to isolate influences over users when they used one setting and then the other
Practice and fatigue: Continuous assessment in a limited period of time
Define tasks that necessarily require the use of different sizes of materials
Usability evaluation just after testing each setting / Think aloud
Discussion
Developed a design for workstation-system based on current observation of workflow and needs in educational makerspaces
Tested an experimental method comprised of repetitive tasks related to Makerspace operation.
No previous findings related to clean up, set up and storage tasks when using workstations.
Replicate the study with different type of seats
Compare workstation performance against other commercially available standing workstations (adjustable and non-adjustable)
Survey: Perceived usefulness and limitations among teachers
Acknowledgements
Friends
CommitteeFamily
CEEO Administrative Staff
Graduate StudentsParticipants
Questions?
Leonardo [email protected]
MS Human Factors Engineering
Advisor: Daniel J.Hannon
Tufts UniversityDepartment of Mechanical Engineering
April 2015
of the physical environment
in Makerspaces:exploratory study
Implications for t
he design and
user evaluation o
f a new workstatio
n-‐system