The Evolution of the Social Programmer Social Media and Software EngineeringMargaret-Anne (Peggy) StoreyKeynote for MSR 2012, Zurich, Switzerland

University of Victoria, Victoria, BC Canada

22 Acknowledgements

3

CHISEL group, UVic, Canada: – Christoph Treude– Brendan Cleary– Fernando Figueira Filho– Jamie Starke– Gargi Bougie– Peter Rigby– Lars Grammel

Chris Parnin, Georgia Tech, USA

Leif Singer, Leibniz Universität, Germany

Ohad Barzilay, Tel-Aviv University, Israel

Daniel German, UVic, Canada

Arie van Deursen, TU Delft, the Netherlands

Li-Te Cheng, IBM Research

Software repositories

Goals

“Software repositories such as source control systems, archived communications between project personnel, and defect tracking systems are used to help manage the progress of software projects. Software practitioners and researchers are

recognizing the benefits of mining this information to support the maintenance of software systems, improve software design/reuse, and empirically

validate novel ideas and techniques.” MSR CFPs 2004-2012

RoadmapBroaden goals of MSR

Redefine software repository to include

social media

Explore the impact of social media on software engineering

Suggest how future MSR research may play a role in emerging practices and

software ecosystems

Broadening

Goals

Beyond maintenance activities and

developer actions…

The emergence of software ecosystems

and communities of practice

Redefining Software

repositories

Historical importance of tools!Integrated development environments:

Software forges:

Trailblazers and early adopters of many communication tools in software development...

PlaceSpace

P. Dourish and V. Bellotti. Awareness and Coordination in Shared Workspaces. Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW'92).

“Architecture of participation” Tim O’Reilly

“Architecture of participation” Tim O’Reilly

"We shape our tools and thereafter our tools shape us", Laws of Media by Marshall McLuhan

"We shape our tools and thereafter our tools shape us", Laws of Media by Marshall McLuhan

http://www.youtube.com/watch?v=A7GvQdDQv8g

McLuhan Quotes:

It is the framework which changes with each new technology and not just the picture within the frame. 1955

It is the framework which changes with each new technology and not just the picture within the frame. 1955

The medium is the message. 1958The medium is the message. 1958

There are many reasons why most people prefer to live in the age just behind them. It's safer. To live right on the

shooting line, right on the frontier of change, is terrifying. 1970

There are many reasons why most people prefer to live in the age just behind them. It's safer. To live right on the

shooting line, right on the frontier of change, is terrifying. 1970

Medium

What role is social media playing in Software

Engineering?

What role is social media playing in Software

Engineering?

M.-A. Storey, C. Treude, A. van Deursen and L.-T. Cheng. The Impact of Social Media on Software Engineering Practices and Tools. In FoSER ’10: Proceedings of the FSE/SDP workshop on Future of software engineering research.

Social Media Channels in Software Engineering

Wikis, social networking, etc.

Tagging

MicrobloggingQuestion &

Answer Websites

Reputation

Blogging

Source codecomments

Research methods used

Studies to inform tool designs and software practices

Mixed methods:– Mining and analysis of software artifacts

– Ethnographic observations– Interviews– Surveys

Wikis, social networking, etc.

Tagging

MicroBloggingQuestion &

Answer Websites

Reputation

Blogging

Source codecomments

Source code comments

Source code comments

How programmers use source comments for communicating with developers?

The role of annotations in program

comprehensioni.e. for human

usage

e.g. comments, bookmarks, tasks, etc

Source Code Comments: Graffiti or Information?

Fermat: "I have a truly marvellous proof of this proposition which this margin is too narrow to contain."

Fermat: "I have a truly marvellous proof of this proposition which this margin is too narrow to contain."

Marginalia, by H. J. Jackson

2001

Marginalia in source codeDevelopers co-opt source code comments for navigation and task management

– But can’t be shared and how they are used varied according to developers’ sophistication with tools

M.-A. Storey, L.-T. Cheng, J. Singer, M. Muller, D. Myers, J. Ryall. 2007. How Programmers can Turn Comments into Waypoints for Code Navigation. In Proceeding of: Software Maintenance, 2007. ICSM 2007.

Wikis, social networking, etc.

Question & Answer Websites

Reputation

Tagging

Source codecomments

MicroBlogging

Blogging

Social TaggingSocial Tagging

Inspired by how social tagging is used on the web

Tagging on the web Social bookmarking,

folksonomies

TagSEA: Tagging “waypoints” in source code and gathering into “tours”

M.-A. Storey, J. Ryall, J. Singer, D. Myers, L.-T. Cheng, M. Muller, 2009. How Software Developers Use Tagging to Support Reminding and Refinding. IEEE Transactions on Software Engineering (TSE), 2009.

Tagging inStudied introduction and adoption of tags by several teams for work items

C. Treude and M.-A. Storey. Work Item Tagging: Communicating Concerns in Collaborative Software Development. In IEEE Transactions on Software Engineering 38, 1 (January/February 2012). pp. 19-34.

Tagging in

Findings: – Categorization (cross cutting concerns)

– Organization– Finding and refinding– Team work practices emerged

C. Treude and M.-A. Storey. Work Item Tagging: Communicating Concerns in Collaborative Software Development. In IEEE Transactions on Software Engineering 38, 1 (January/February 2012). pp. 19-34.

Tags used in

ConcernLines

WorkItemExplorer: ICSE, Friday 10:45am

Wikis, social networking, etc.

Question & Answer Websites

Reputation Tagging

Source codecomments

MicroBlogging

Blogging

MicrobloggingMicrobloggingWhy do developers tweet?

MicrobloggingSoftware engineers tweet actively (share) facts about software engineering topics and technology

G. Bougie, J. Starke, M.-A. Storey and D. German. Towards Understanding Twitter Use in Software Engineering: Preliminary Findings Ongoing Challenges and Future QuestionsIn Proceedings of the 2nd International Workshop on Web 2.0 for Software Engineering. 2011.

Should microblogging be integrated in the IDE for the enterprises?

A. Guzzi, M. Pinzger, A. van Deursen. Combining micro-blogging and IDE interactions to support developers in their quests. ICSM 2010.

W. Reinhard, Communication is the key – Support Durable Knowledge Sharing in Software Engineering by Microblogging. SENSE 2009.

Wikis, social networking, etc.

Question & Answer Websites

Reputation Tagging

Source codecomments

MicroBlogging

Blogging

BloggingBloggingWhy do developers blog?

Blogging (1)“Our internal blog (only readable by those at the company) is really a virtual "water cooler". We are expected to blog there at least once a week (spelling and grammar don't matter) just to keep others at the company updated on what we are doing. If you can't find something interesting to say at least once a week, then you're not doing enough interesting work. Blogs are everything from: I hit this really annoying bug, to I completed this new awesome feature.”

(Ian Bull, Software Engineer, EclipseSource)

“Our internal blog (only readable by those at the company) is really a virtual "water cooler". We are expected to blog there at least once a week (spelling and grammar don't matter) just to keep others at the company updated on what we are doing. If you can't find something interesting to say at least once a week, then you're not doing enough interesting work. Blogs are everything from: I hit this really annoying bug, to I completed this new awesome feature.”

(Ian Bull, Software Engineer, EclipseSource)

Blogging (2)

Determining requirements through blogs [Park and Maurer, CHASE 2009]

How developers blog: high-level concept discussion and requirements

[Pagano and Maalej, MSR 2011]

Blogs play a role in documenting APIs [Treude and Parnin, Web2SE 2011]

Wikis, social networking, etc.

Question & Answer Websites

Reputation Tagging

Source codecomments

MicroBlogging

Blogging

\

Question and Answer WebsitesQuestion and

Answer WebsitesWhat role do Question and Answer

websites play in software engineering?

Stackoverflow

Over 92% of the questions on Stackoverflow are answered, and for those 92% the median answer time is 11 minutes

L. Mamykina, B. Manoim, M. Mittal, G. Hripcsak, and B. Hartmann. Design lessons from the fastest q&a site in the west. CHI 2011.

Stackoverflow

How-to questions prevalent, and used frequently by novices

C. Treude, O. Barzilay and M.-A. Storey. How do Programmers Ask and Answer Questions on the Web? NIER/ICSE 2011.

Linking Stackoverflow data with API usage

C. Parnin, C. Treude, L. Grammel and M.-A. Storey. Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow”. Under submission, May 25 2012, related blog (15,000 hits so far).

Stackoverflow as Crowd DocumentationCoverage of API documentation: 77% of the Java API classes & 87% of Android API classes

Speed of coverage:

C. Parnin, C. Treude, L. Grammel and M.-A. Storey. Crowd Documentation: Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow”. Under submission, May 25 2012, related blog (15,000 hits so far).

Impact on documentation tools? Automatically generating documentationVisualizing crowd documentation

http://latest-print.crowd-documentation.appspot.com/?api=android

Wikis, social networking, etc.

Question & Answer Websites

Reputation

Tagging

Source codecomments

MicroBlogging

Blogging

ReputationReputationWhy do developers and recruiters use reputation

features in social networking sites?

Social Coding in GitHubGitHub supports transparency Management of profiles and their visibility important for project successExplicit self promotion not valued

L. Dabbish, H.C. Stuart, J. Tsay and J. Herbsleb. Social coding in github: transparency and collaboration in an open software repository. CSCW 2012.

Masterbranch and Coderwall

Developer motivations

L. Singer, F. F. Filho, B. Cleary, C. Treude, M.-A. Storey, K. Schneider. Mutual Assessment in the Social Programmer Ecosystem: An Empirical Investigation of Developer Profile Aggregators Under submission, June 2012.

“As a software developer, you need to learn a lot. It's a constant challenge to keep up with technologies. […] It's the proof that you have that kind of mindset, if you do it [learning, keeping up] anyway in your free time”

“[When] I look at repos around this topic [...], I may be interested in seeing the coder footprint of people that

work in this area [...] their favorite languages, the topics they write code about, what they work on”

Recruiter motivations

Social connections for finding candidates that are passionate, learn quickly and with diverse skills

Mutual Assessment in the Software Ecosystem

L. Singer, F. F. Filho, B. Cleary, C. Treude, M.-A. Storey, K. Schneider. Mutual Assessment in the Social Programmer Ecosystem: An Empirical Investigation of Developer Profile Aggregators Under submission, June 2012.

Wikis, social networking, etc.

Question & Answer Websites

Reputation

CommunityPortals

Tagging

Source codecomments

MicroBlogging/Blogging

Wikis etc...Wikis etc...

• Wikis useful for documentation, requirements engineering, knowledge sharing

• Impact of social networking in software engineering (Codebook, Github) - can also follow software artifacts!

• Crowdsourcing of coding (TopCoder) and testing (e.g. Google’s A/B testing approach)

• End-user involvement in closed, open source and mixed initiative projects

• Community portals in software communities

Making sense of the social media ecosystem (the social era) in software engineering

Wikis, social networking, etc.

Tagging

MicrobloggingQuestion & Answer

Websites

Reputation

Blogging

Source codecomments

Social Media

Enhances

Task management, coordination

Awareness, transparency

Learning

Personal portfolio development, expertise finder, recruitment

Social Media

Retrieves??

Programmer “rock stars”Oral culture (talkbacks on blogs)End-user programmersPortfolios

“On Twitter, I follow a few prominent software developers. For example, Kelly Sommers from Canada, she’s constantly trying new things. I don’t think she ever sleeps. So she’s a great source of inspiration.” (From the Reputation study)

Social Media

Reverses??

Geek cultureReliance on searchInterruptionsSecurity holesSpaghetti code

“Google as the most important member on your programming team”, Brendan Cleary

Social Media

Obsolesces??

Formal documentationIn-house expertise, certain jobsNeed for co-locationClassroom educationEmail listsCVs

"It's always good to document a widget, but it's more important in many cases to document a process [...]. It's the context of how you use the widget that's much more important."

(Distributed) Community formation, awareness, transparency, knowledge curation,learning, reuse, reputation

Community fragmentation,

informal processes, geek culture,

reliance on search,security concerns,

interruptions, advertisements

Programming gurus,end users as developers,verbal discussions, portfolios,communities of practice

In-house expertise/jobs, formal documentation,classroom education,

CVs, email lists, need for co-location

Social Media

7 Burning Questions about Social Media use in

Software Engineering...

Wikis, social networking, etc.

Tagging

MicrobloggingQuestion & Answer

Websites

Reputation

Blogging

Source codecomments

Q1: Towards the “Social Programmer”?• What makes a good developer?

– Ability to write good code… or…– Ability to search for good code and to network? Do social skills matter?

• Can you assess a programmer’s ability independently of the larger community?

Q2: Classroom education still relevant? Knowledgeable -> Knowledge-able

http://www.academiccommons.org/commons/essay/knowledgable-knowledge-able, by Michael Wesch

Less than 13% of Wikipedia content is authored by women, fewer than 9% of editors are women

Why does this matter?Less female oriented contentFewer opportunities to gain expertise, build portfolios, reputation

Q3: Gamification and marginalization?

“Define Gender Gap? Look up Wikipedia’s Contributor List. New York Times, January, 2011.

http://blog.20sb.net/2011/10/changing-the-ratio-on-wikipedia.html

“obsessive web browsing can cause attention spans to drop to as little as nine seconds—equivalent to a goldfish”, Ted Selker, MIT 2002

“obsessive web browsing can cause attention spans to drop to as little as nine seconds—equivalent to a goldfish”, Ted Selker, MIT 2002

Q4: Information overload?

Q5: Impact on design and documentation?

What are the risks of using social media for requirements

gathering/elicitation?

Does the use of social media lead to a

“laissez-faire” documentation approach?

Q6: Impact on software quality? Does social media use lead to:Spaghetti code and brittle integrations? More (viral) bugs? Security concerns? Undesirable clones? More license violations? Poor code ownership?

Q7: Impact on mining methods?Can mining of social media lead to improved predictions, detections and recommendations?

Challenge!! mining an ecosystem of media!

Concluding remarks...

Mining community takeawaysThe social era in software engineering begs us to ask different kinds of questions

Social media ecosystems as an integral component of software repositories (or vice versa?)

Abundant and exciting mining opportunities! Combine with other research methods

The social era in software engineering begs us to ask different kinds of questions

Social media ecosystems as an integral component of software repositories (or vice versa?)

Abundant and exciting mining opportunities! Combine with other research methods

Software repositories

“If we understand the revolutionary transformations caused by new media, we can anticipate and control them; but if we continue in our self-induced subliminal trance, we will be their slaves.” Marshall McLuhan, 1974

“If we understand the revolutionary transformations caused by new media, we can anticipate and control them; but if we continue in our self-induced subliminal trance, we will be their slaves.” Marshall McLuhan, 1974

Goals

MSR 2012-...

Goals first...

http://www.thechiselgroup.orghttp://margaretannestorey.wordpress.com

/ @thechiselgroup, @margaretstorey

on Twittermstorey@uvic.ca

Funded by NSERC/DRDC/IBMAcknowledgements for feedback on my talk: C. Treude, B. Cleary, I. Richardson, J. Aranda, B. Fu, J. Starke, C. Petrachenko, and to C. Parnin and L. Singer

IEEE Software special issueBridging Software Communities through Social Networking

Papers due June 25th, 2012

http://www.computer.org/portal/web/computingnow/swcfp1

Editors:Jan Bosch, Chalmers University of Technology, SwedenMargaret-Anne Storey, University of Victoria, CanadaAndrew Begel, Microsoft Research, USA

(Distributed) Community formation, awareness, transparency, knowledge curation,learning, reuse, reputation

Community fragmentation,

informal processes, geek culture,

reliance on search,security concerns,

interruptions, advertisements

Programming gurus,end users as developers,verbal discussions, portfolios,communities of practice

In-house expertise/jobs, formal documentation,classroom education,

CVs, email lists, need for co-location

Social Media

Additional ReferencesC. Treude and M.-A. Storey. Effective Communication of Software Development Knowledge Through Community Portals. ESEC/FSE ’11.

Communities of practice: http://www.ewenger.com/theory/

See the following two links for other references on social media use in software engineering:

M.-A. Storey, C. Treude, A. van Deursen and L.-T. Cheng. The Impact of Social Media on Software Engineering Practices and Tools. In FoSER ’10: Proceedings of the FSE/SDP workshop on Future of software engineering research.

Christoph Treude’s Blog: http://www.ctreude.ca/

See also this year’s and last’s MSR proceedings for some new work on this topic.