Open Professional Development as a Sociotechnical Design Challenge

Abstract It is increasingly common for individuals to seek to enter into and advance in a technical profession by acquiring knowledge and skills through open and independent online learning. Missing from the ecosystem of tools and platforms to provision free or low-cost learning opportunities at scale is an infrastructure to support the kinds of learning, development, and acculturation into a field of practice that are commonly afforded by participation in the social context of a profession; i.e., a professional community. This note introduces and conceptualizes Open Professional Development: A sociotechnical concept model for coupling open, online learning practices with acculturation into a field of practice through participation in a professional community. Challenges to the implementation of this vision are discussed, and a prototype model is presented for capturing participation data using trace data extracted from advertised descriptions of meetup.com events.

Author Keywords Communities of Practice, Technology-Enhanced Learning, DesignX, Sociotechnical Systems.

ACM Classification Keywords K.3.1. Computers and Education: General.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. CHI'17 Extended Abstracts, May 06-11, 2017, Denver, CO, USA Copyright is held by the owner/author(s). ACM 978-1-4503-4656-6/17/05. http://dx.doi.org/10.1145/3027063.3053143

Warren S. Allen Florida State University Tallahassee, FL 32303 USA wsallen@fsu.edu

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1478

Introduction As the 21st century approached, computers increasingly substituted for labor in middle-wage technology jobs and complemented high skilled labor, leading to a polarization of jobs into low-skilled, routine work and high-skilled, high-wage jobs [1,4,6,7]. Job growth in this “hourglass economy” [10] is asymmetrical, with growth in the highly-skilled upper tier outpacing the moderate growth of low-skilled, low-wage routinized technology jobs [18]. As a result, well-paying technology careers are disproportionately available to individuals who have access to resources necessary to move upward from low-wage, routinized technology work, and into a shrinking middle-tier of technology jobs. Online education and alternative career pathways drive hopes and strategies to mitigate the funneling effect by democratizing and expanding access to these resources.

The disruptive potential of educational innovations – e.g., Massive Online Open Classes (MOOCs) and related learning infrastructure – intend to “bring the best education in the world to the most remote corners of the planet, help people in their careers, and expand intellectual and personal networks” [13]. However, the value and future of MOOCs is debated [see 5 for review], and open education for (would-be) technology professionals remains, in practice, an online phenomenon constrained by the limitations of conventional learning theories and the concept of learner as a receptacle of (taught) knowledge.

Missing from this vision is an infrastructure to support the kinds of learning, development, and acculturation into a field of practice that are commonly afforded by participation in the social context of a profession; i.e., a

professional community. The openness and accessibility of regional and local technology communities affords “off-line” opportunities for participation in informal associations in which “groups of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” [16]. In such environments, expertise beyond taught knowledge is developed through the exchange of narratives among experts with diverse experience and knowledge [3,9], and the practice of networking – i.e., “building, maintaining, and using professional relationships” [17] is conducted. By acquiring, practicing, and reflecting on the knowledge to conduct oneself in such environments, the student continues the learning pathway beyond those facilities provided by open, online learning platforms.

This note introduces Open Professional Development (OpenPD), a sociotechnical concept model for coupling open, online learning practices with acculturation into a field of practice through participation in a professional community. A prototype technical infrastructure is described, designed to extend emerging standards for capturing learning experiences that currently support only online modes of learning. Data mined from meetup.com event descriptions are used to construct an initial set of terms for a controlled vocabulary for OpenPD, a necessary first step in designing a system with which learners will describe their activities. Findings suggest that the data-mining method requires non-trivial human involvement to translate meetup.com data into a useful vocabulary, but the results of the effort produce a feasible set of terms for scaffolding the construction of a full OpenPD vocabulary. The paper concludes with a brief discussion of the OpenPD vision

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1479

and proposed infrastructure as a response to a sociotechnical “DesignX” problem.

Towards a Model for OpenPD This note describes initial efforts to design for the effective capture and representation of activities and experiences that constitute OpenPD. Here, OpenPD refers to the application of reflective practice (“a challenging, focused, and critical assessment of one’s own behavior as a means towards developing one’s own craftsmanship” [12]) to the engagement with open resources, referring to both online and in face-to-face settings. Figure 1 presents a sociotechnical concept model consisting of the following:

• The status quo: Participation and learning in face-to-face environments are possible, but extant and emerging components for capturing and representing those experiences remain focused on technology-mediated, online modes of learning and interaction. Emerging specifications for “microcredentials” (e.g., “digital badges” [21]) remain either tied to online learning environments, coupled with traditional K-12 or higher education settings, or are delivered at cost by for-profit education and software vendors.

• OpenPD vision: A more holistic vision of capturing learning experiences requires the integration of trace data about online learning experiences (2a) with data about community participation (2b). In order to achieve this vision, two technical artifacts need implementation: an interface (2c) between physical learning environments and digital learning record systems designed to facilitate an individual capturing learning experiences, and (2d) a vocabulary for describing the practice of OpenPD.

Capturing OpenPD Experiences Capturing trace data from learning activities is not new; the practice is central to the still-growing domain of Learning Analytics. However, existing systems and models for learning analytics are almost exclusively concerned with the capture of data from of online, technology-mediated learning platforms. The proposed component of the technical infrastructure for OpenPD is a mobile application to provide a user interface for capturing learning experiences such as (but not limited to) offline learning experiences. Furthermore, while mobile applications for capturing experiences abound – from simple note-taking and diary apps to apps for guided mindfulness – none exist that are designed to capture or produce learning data based on an individual’s learning experiences, and in a manner which affords integration with existing learning analytics infrastructure.

Design and implementation of an interface to capture OpenPD activities first requires developing “systematic approaches to obtain[ing] evidence of authentic student achievement” [20], which, coupled with technical interoperability with a broad ecosystem of learning tools and platforms, means that eLearning standards ought to be central to the OpenPD model. To that end, OpenPD relies on and implements the emerging Experience API (xAPI, formerly the “Tin Can” API) [19]. Current efforts on this specification for creating trace data about learning activities are exclusively focused on online modes of learning. The OpenPD model will describe a controlled vocabulary (a restricted list of terms used to achieve consistency in the description of content objects and to facilitate retrieval [15]) and constitutes a novel extension of xAPI in design and application.

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1480

Initializing the OpenPD vocabulary An empirical basis is required for the vocabulary that would describe the activities and entities involved in the conduct of OpenPD. Trace data grounded in the descriptions of real technology community events was constructed by extracting activities and topics from event descriptions from meetup.com, a popular website for organizing face-to-face group events. The meetup.com API was used to collect descriptions of publicly-available events of four highly-active Python-related groups. Further analysis followed the guidance found in [15] for controlling a vocabulary, consisting of: (1) Defining the scope, or meaning, of terms; (2) Using the equivalence relationship to link synonymous and nearly synonymous terms; and (3) Distinguishing among homographs.

Results The descriptions were parsed into 8636 unique sentences, from which 617 dynamic verbs and 3955 noun-phrases (NP) were extracted. 8637 verb-NP combinations based on sentence co-occurrence were constructed, and for each combination, up to 20 randomly-selected sentences were used to interpret the verb in context and generate an xAPI verb definition. Table 1 displays the resulting vocabulary elements.

Discussion: OpenPD as a DesignX Problem The OpenPD vision is a complex sociotechnical one. Among many others [e.g., 2,8,14], Norman and Stappers [11] draw attention to design challenges presented by complex sociotechnical systems. Their “DesignX” paradigm entails tackling problems involving a complex, non-linear mixture of people and technology embedded in political, economic, cultural,

Figure 1: Sociotechnical concept model for OpenPD integrating online trace data with data about community participation.

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1481

organizational, and structural problems. It is the assemblage of elements including (but not limited to) the technology labor force, educational systems, and the commercial industry of training and credentialing that are brought into the scope of the work described in this note.

The OpenPD vision inserts the professional community into the arrangement in order to implement positive changes in the economy of workforce development. In times of increasing career fluidity and instability, the professional community may be the particular structure to establish the kind of learning pathways to expertise that once ran through social arrangements such as union-supported workplace apprenticeships. Among the prerequisites for inserting the professional community where once there were unions is an infrastructure that establishes the community as a place to perform the kinds of professional development that is expected of middle- and high-skilled workers.

It is a long and winding road from concept models and vocabularies to realizing participation in the professional community as a viable career pathway. Such is the reality of DesignX problems, for which “the solution should be reached through modularity, and the introduction of numerous small, incremental steps” [11]. The event descriptions analyzed in this note include many activity statements (dynamic verbs coupled with noun-phrases) resulting in a small set of viable verbs for an OpenPD vocabulary. Constructing basic verb definitions from a convenience sample of and may not scale well to the development of a full vocabulary profile. A larger dataset from a more diverse range of communities and an improved language-parsing design will produce a richer dataset;

however, automated methods alone are unlikely to generate a useful vocabulary.

Additionally, a gap in the vocabulary will result if only event descriptions were used in vocabulary construction. Simply put, describing an event for advertising purposes on meetup.com is substantially different from describing what a person does, learns, and experiences at the event. Finally, the vocabulary constructed in this note describes a set of possible activities, but has no terminology for capturing the reflection on those activities. Therefore, the next steps in developing the OpenPD vocabulary will focus on the following: (1) improving the quality of automated processes for vocabulary construction, and (2) designing and conducting human-centered research to develop terminology describing actions, experiences, and reflection.

Conclusion This note introduced Open Professional Development (OpenPD), a sociotechnical concept model for coupling open, online learning practices with acculturation into a field of practice through participation in a professional community. The OpenPD model represents a future in which the reach and accessibility of online learning innovations are coupled with the accessibility and participation in local and regional face-to-face professional events, networks, and communities. The automated extraction of terms from found data produced by meetup.com event coordinators is shown to be a viable source of a preliminary and extensible OpenPD controlled vocabulary. With such a vocabulary, learners can describe (and information systems can capture) “offline” learning activities and experiences for integration with online learning data.

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1482

Proposed xAPI Verb Verbs in Data Example NP Pairs Analysis Summary

Presented. Indicates an actor has executed the performance of a structured presentation to an audience.

Talk (1217) Present (108) Show (100)

Tech talk (26) Django talk (12) Python abstract (12) Python code (12) Office hours (8)

“Talk” is found in the dataset both as a noun (e.g., giving a “talk” about a topic) and less-frequently as a verb. Therefore, instances of giving a talk are interpreted the same as the verb “present” and “show” which also frequently occur.

Practiced. Indicates an actor has performed tasks the intention of which is specifically or mostly to improve their performance of subsequent, similar tasks.

Work (482) Use (433)

Open source (9) Python projects (9) Learning python (6) Web framework (4) Echo nest apis (3)

Events offer opportunities to learn about projects, platforms, etc. Events also offer opportunities to practice working on and using languages, tools, APIs, etc.

Built. Indicates an actor has performed tasks the intention of which is specifically to create a particular kind of artifact.

Make (252) Write (242)

Back end (3) Basic process (3) Django app (2) Python code (2)

Similar to “working” and “using” tools, certain events focus on learning by producing specific (types of) artifacts; e.g., writing code, or making a Web application.

Helped. Indicates an actor has assisted another actor or group of actors in a skill- or knowledge-building context.

Help (456) Python basics (17) Language basics (14) Python projects (6) Practice projects (5) Open source (5)

Instances of “help” refer to both the giving of help (“volunteers will help you set up your development environment”) and getting help (“bring a project or a problem and get help from fellow pythonistas”). Both actions should be available for learners to report, but they are two different actions and experiences requiring two verbs.

Table 1: Controlled vocabulary elements derived from meetup.com event description data. While these vocabulary elements do not fully constitute an OpenPD vocabulary, the automated extraction of terms from found data is a viable source of vocabulary elements with which learners and information systems can describe learning activities and experiences.

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1483

References 1. Daron Acemoglu and David Autor. 2011. Skills,

Tasks and Technologies: Implications for Employment and Earnings. In Handbook of Labor Economics, David Card and Orley Ashenfelter (ed.). Elsevier, 1043–1171. https://doi.org/10.1016/s0169-7218(11)02410-5

2. Mark S. Ackerman. 2000. The intellectual challenge of CSCW: The gap between social requirements and technical feasibility. Human–Computer Interaction 15, 2-3: 179–203. https://doi.org/10.1207/s15327051hci1523_5

3. Mark S. Ackerman, Volkmar Pipek, and Volker Wulf. 2003. Sharing Expertise: Beyond Knowledge Management. MIT Press. https://doi.org/10.1524/icom.4.1.65.61678

4. Adrian Adermon and Magnus Gustavsson. 2015. Job Polarization and Task-Biased Technological Change: Evidence from Sweden, 1975–2005. The Scandinavian Journal of Economics 117, 3: 878–917. https://doi.org/10.1111/sjoe.12109

5. Abram Anders. 2015. Theories and Applications of Massive Online Open Courses (MOOCs) : The Case for Hybrid Design. The International Review of Research in Open and Distributed Learning 16, 6. https://doi.org/10.19173/irrodl.v16i6.2185

6. David H. Autor, Lawrence F. Katz, and Melissa S. Kearney. 2008. Trends in U.S. Wage Inequality: Revising the Revisionists. Review of Economics and Statistics 90, 2: 300–323. https://doi.org/10.1162/rest.90.2.300

7. David H. Autor, Frank Levy, and Richard J. Murnane. 2003. The Skill Content of Recent Technological Change: An Empirical Exploration. The Quarterly Journal of Economics 118, 4: https://doi.org/10.1162/003355303322552801

8. Lisl Klein. 2014. What do we actually mean by “sociotechnical”? On values, boundaries and the problems of language. Applied Ergonomics 45, 2, Part A: 137–142. https://doi.org/10.1016/j.apergo.2013.03.027

9. Jean Lave and Etienne Wenger. 1991. Situated learning: legitimate peripheral participation. Cambridge University Press, Cambridge [England]; New York. https://doi.org/10.1017/cbo9780511815355

10. Douglas S. Massey and Deborah S. Hirst. 1998. From Escalator to Hourglass: Changes in the U.S. Occupational Wage Structure 1949–1989. Social Science Research 27, 1: 51–71. https://doi.org/10.1006/ssre.1997.0612

11. Donald A. Norman and Pieter Jan Stappers. 2015. DesignX: Complex Sociotechnical Systems. She Ji: The Journal of Design, Economics, and Innovation 1, 2: 83–106. https://doi.org/10.1016/j.sheji.2016.01.002

12. Karen F. Osterman. 1990. Reflective Practice: A New Agenda for Education. Education and Urban Society 22, 2: 133–152. https://doi.org/10.1177/0013124590022002002

13. Laura Pappano. 2012. Massive Open Online Courses Are Multiplying at a Rapid Pace. The New York Times. Retrieved December 30, 2015 from http://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html

14. Katie Shilton, Jes A. Koepfler, and Kenneth R. Fleischmann. 2013. Charting Sociotechnical Dimensions of Values for Design Research. The Information Society 29, 5: 259–271. https://doi.org/10.1080/01972243.2013.825357

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1484

15. National Information Standards Organization (US) and others. 2005. Guidelines for the construction, format, and management of monolingual controlled vocabularies. NISO Press. http://www.niso.org/apps/group_public/download.php/12591/z39-19-2005r2010.pdf

16. Etienne Wenger, Richard A. McDermott, and William Snyder. 2002. Cultivating Communities of Practice: A Guide to Managing Knowledge. Harvard Business Review Press. https://doi.org/10.1108/bl.2002.17015bae.001

17. Hans-Georg Wolff and Klaus Moser. 2009. Effects of networking on career success: A longitudinal study. Journal of Applied Psychology 94, 1: 196–206. https://doi.org/10.1037/a0013350

18. Erik Olin Wright and Rachel E. Dwyer. 2003. The patterns of job expansions in the USA: a comparison of the 1960s and 1990s. Socio-Economic Review 1, 3: 289–325. https://doi.org/10.1093/soceco/1.3.289

19. 2014. adlnet/xAPI-Spec. GitHub. Retrieved January 16, 2016 from https://github.com/adlnet/xAPI-Spec

20. 2016. Higher Education Quality: Why Documenting Learning Matters. A Policy Statement from the National Institute for Learning Outcomes Assessment. National Institute for Learning Outcomes Assessment. eric.ed.gov/?id=ED567116

21. mozilla/openbadges-specification. GitHub. https://github.com/mozilla/openbadges-specification

Late-Breaking Work CHI 2017, May 6–11, 2017, Denver, CO, USA

1485