Teaching Software Product Lines: A Snapshot of Current Practices and Challenges

Teaching Software Product Lines A Snapshot of Current Practices and Challenges

Presented at SIGCSE 2018

Repository: http://teaching.variability.io/ TOCE article: https://hal.inria.fr/hal-01522779 SPLTea workshops: http://spltea.irisa.fr Education and SPL in the CfP of SPLC 2017 and 2018: http://splc2018.net/call-for-papers/call-for-research-papers/

2018 | Acher, Lopez-Herrejon, Rabiser (TOCE article: SPL/variability and teaching) https://teaching.variability.io 3

(Software)ProductLines

2018 | Acher, Lopez-Herrejon, Rabiser (TOCE article: SPL/variability and teaching) https://teaching.variability.io

Vari-ability in Software

“Industrial reality shows […] the number of variation points, variants […], and dependencies […] easily reaches staggering levels [...] often resulting in a situation in which no one […] has a comprehensive overview of the available variability.”

•  impacts all phases of development

and organizational structures •  affects not only code, but also

documentation, test cases, … •  emergent or planned •  results from diverse decisions

made to address requirements

4

(c) Charles Krueger, BigLever


3 ways to build a (software) product

Indepently

5

„Clone&Own“ „Shared“(reusable)Assets

SoftwareProductLinesProductConfigurationVariabilityModelingComponentsDomain-specificLanguagesGeneratorsPreprocessorsDesignPatterns…

(credits:ThorstenBerger’sslide)


Or

Xor

Mandatory

Optional

not, and, or, implies

Variability Models (feature models)

Variants of code (e.g., Java ou C) Variants of user interfaces Variants of video sequences Variants of models (e.g., UML or SysML) …


Software Product Line and Variability well-established in research and industry

Body of knowledge growing

7

12th International Workshop on Variability Modelling of Software-Intensive Systems,


What about teaching and software product line?

• Teach or perish! What’s the point of doing research in this field if you’re unable to disseminate your results and train the engineers of tomorrow?

•  Education has a key role to play in disseminating the SPL knowledge •  Research can also benefit from teaching

• We knew there are many initiatives here and there but unclear to find out eg:

•  what is the material available •  what are the possible gaps and difficulties faced •  what is needed by industry

• Teaching SPLs is challenging •  Relatively young discipline •  SPLE encompasses a variety of topics (SE, PL, AI)

8


In a nutshell, there is a central question

How to teach software product lines and variability?

9


Research Method Stakeholders of Software Product Line Teaching

Survey on Teaching of Software

Product Lines

SPLTea Workshops on Software Product

Lines and Teaching

Survey on Learning about Software

Product Lines

Researchers Educators Practitioners Students

Our Initiatives

Analyses of the State of Practice

Curriculum (e.g., length of course, topics covered, audience, …)

Material (e.g., textbooks, tools used, ...)Experience Reports

Challenges

Recommendations

Suggestions to improve the practiceKey messages and concrete actions

Analyses and Discussions

Our Results

#1 Survey on Teaching of Software Product Lines


Survey Design •  91 contacts

•  pool of SPL researchers we know teach

•  search for SPL courses online •  review SPLC and VaMoS papers

• On-line questionnaire (http://www.surveygizmo.com/s3/1342346/Teaching-Software-Product-Lines)

•  13 enumerative and 5 open questions

•  Feedback by colleagues to refine questionnaire

•  34 complete responses and 15 partial

•  Decision: only analyze complete responses

12


Survey Results Respondents and their institutions •  34 responses from 13 countries •  Avg. SPL experience 10yrs, avg. SPL teaching experience 6yrs

•  Half say their research exp. = their teaching exp. •  Institutions

•  15 research-focused •  9 teaching-focused (colleges) •  7 have both focuses •  3 industrial institutions

•  Departments •  CS (13), SE (9), IT (2) •  others (10) -- all CS-related

•  16 full and self-contained SPL courses •  18 teach SPL topics as part of other courses

13

Austria, 2 Canada, 1

Colombia, 1

France, 2

Germany, 12

Israel, 2 Italy, 1

Japan, 1

Korea, Rep. Of, 1

Spain, 3

Switzerland, 1

USA, 6

Portugal, 1


Survey Results Literature used • 28 respondents use these books • 25 use research papers, selected based on personal

preferences • 12 use case studies, e.g., by van der Linden et al. 2007;

BigLever; Renault, STAGO, LINUX, and SPLOT; and own case studies

14

0 1 2 3 4 5 6 7 8 9

10

Apel et al. 2013

Clements and

Northrop 2001

Czarnecki and

Eisenecker 2000

Gomaa 2005

Pohl et al. 2005

van der Linden et al. 2007

Weiss and Lai 1999


Survey Results Tools used

SPL Tool used byNone 8FeatureIDE 6BigLever's GEARS 4FeatureHouse 3pure system's pure::variants 3AHEAD 2CIDE 2CVL 2Feature Modelling Plug-in (FMP) 2DOPLER, EasyProducer, FaMa, Familiar, FeatureMapper, Munge, SPLAR, SPLOT, VARIAMOS, Varmod

1 each

C++, Metaprogramming, Software Generators, Xtext, MPS 1Different tools for creating feature diagrams and UML-based models 1Feature modeling UML tools extended for SPL 1Haskell-Embedded Variation DSL 1Own research prototypes 1

15

also see the survey by Berger et al. VaMoS 2013


Survey Results Length, audience

• Most courses are held over a semester (26) •  3 "on demand", 2 as week-long courses, quarter (1), 1-2 days

seminars (1), and with a flexible schedule (1) •  Average course length 24 hours (min: 1h; max: 120h)

16

0 2 4 6 8

10 12 14

Only Undergrads

Only Grads

Only Industry People

Undergrads and Grads

Grads and Industry people

Undergrads students and

Industry people

Undergrads, Grads, and Industry people


Survey Results Practical time, SPL lifecycle coverage

•  Practical time/overall course time 65% in average

17

Topic

DepartmentCS SE OT

Total

Requirements engineering 7 4 12 23Testing 3 1 4 8Modelling 9 8 13 30Implementation 9 8 8 25Maintenance & Evolution 9 3 5 17Reverse engineering & SPL Adoption 7 2 1 10Processes 5 6 8 19Other 5 2 3 10

CS: Computer Science, SE: Software Engineering, OT: Other departments


Survey Results Challenges of teaching SPL

•  Lack of and availability of well-documented real-world examples and case studies suitable for teaching (16)

•  Complexity of the subject and required background knowledge (9)

•  Lack of and availability of SPL tools (7)

•  Acceptance on the curriculum and opening the mind of students (4)

•  "Even software engineering can be hard to teach as developing large-scale systems does not connect to students hands-on experience of developing rather small solutions. Teaching SPL means SE for many systems, this does even less relate to students' experiences."

18


Survey Results How to improve the state of teaching SPL

•  Improved textbook examples and case studies (13)

•  Better tools for students (7)

•  Broaden the focus of teaching SPLs (3)

•  SPL is already mainstream! (2)

• Other ideas •  videos of experts to motivate SPLs and explain key concepts and ideas •  incorporate the notions and terminology of SPLs more in other courses •  develop a standard curriculum and evaluation scheme for teaching SPLs

19


Survey Results Impact of teaching SPL

• On research •  Student participation in research evaluations (11) •  Finding research personnel (8) •  Feedback on and discussion about tools, examples, case studies (6) •  Connecting with industry (5) •  Discussion of open research issues with students (2)

• On practice •  Teaching SPLE = preparation of introduction of SPLE in industry (13) •  Makes students aware of the topic à students eventually end up in

industry or even are already working in industry à increases awareness of SPL in industry and trains future practitioners (7)

•  Teaching SPLs can be the start of industry-academia collab. projects (2)

20


Survey Results Other comments

•  Teaching SPLs together with model-driven development •  very interesting from a teaching perspective

•  Having a common, clearly defined basis of terminology and concepts taught at the majority of the institutions

•  would help a lot

•  Real-world SPLE is very different from research •  Variability models seem to be the key topic in research •  In practice it often is just one technique for some roles •  Industry often successfully manages variability w/o dedicated models

•  Our survey was not a perfect fit for SPL consultants and trainers •  True; we have to address this in our future work

•  Invite consultants/trainers in a workshop

21

#2 SPLTea Workshops 2014 and 2015

22


SPLTea’14: Experience Reports and Industry Panelist

• 30+ attendees at SPLC 2014 (Florence, Italy) • 3 experience reports • Industry Panel

• Dr. Paul Clements, VP of customer success at BigLever (a leading commercial SPL engineering tools provider)

• Dr. Martin Becker, head of Embedded Systems Development at Fraunhofer IESE

• Dr. Steve Livengood, Software Architect at Samsung Research America

• Dr. Juha Savolainen, (at that time) SW Director at Danfoss Power Electronics


Industry Panel

24


Key points discussed at the industry panel • How should SPLs be taught so that students going to industry have the required knowledge?

• SPL infuses additional complexity in the software engineering problems – students should be aware of it.

• What is the required SPL knowledge for industry, e.g., how relevant are variability models?

•  treat variability in the same way as other basic functional and non-functional properties are treated

• What are the challenges of teaching SPLs in the light of the needs of the industry?

•  “technology is easy, people are not” 25


Key points discussed at the industry panel

• How can the state of teaching SPLs be improved? • Making the courses more “realistic” • Elevating variability as core concepts in standard SE curricula

• How could industry be better involved in teaching SPLs? •  Industry involvement in the teaching of SPLs should be seen as

a mutually beneficial activity • Product line tools used in industry can/should be used

• Scenario: SPLs going mainstream? •  In the past, companies were unwilling to publicly acknowledge

their use of SPL. Now, they instead not only publicly acknowledge their use but go as far as to advertise it.

Teaching can be a catalyst for this scenario to materialize. 26


Experience reports and insigths

• Variability in… teaching experiences: •  Seidl and Domachowska presented their experiences on ’Teaching

Variability Engineering to Cognitive Psychologists’ • McGregor reported about ’Ten Years of the Arcade Game Maker

Pedagogical Product Line’ •  Collet et al. presented their ’Experiences in Teaching Variability

Modeling and Model-driven Generative Techniques’

• Other discussions at the workshops • Need for case studies • Curriculum and different ways to teach SPL knowledge • Populating a repository of teaching material

27


SPLTea’15 (Nashville, USA, SPLC 2015)

•  Brainstorming about a repository of teaching material for product lines and variability

• Questions like: •  What could be the desirable features, services, and underlying technology

of such a repository? How to collect more resources (including slides, lecture notes, lab instructions, tools)? How to involve industry? How to advertise the existence of the repository?

•  For which: educators only? researchers? students? practitioners? both? •  How to build a network of SPL educators? •  How to make it possible for people to browse and understand the SPL

knowledge of the repository?

• Result and ongoing effort: http://teaching.variability.io

28

#3 Student Survey

29


Survey on Learning About SPLs •  What did you expect from this course before it started? •  What did you actually learn from this course? •  What did you not learn and/or would have wanted to hear

more about? •  Please define in your own words what a Software Product

Line is. •  What do you think is the most challenging part of learning

about Software Product Line? • Suggestions to improve the state of the teaching of

Software Product Lines? • How can the teaching of Software Product Lines impact

industrial practice? • Any other comments

30


Courses and Countries

31


Students’ Expectations and Perceived learning effects

• They expected to learn: •  “managing a complex software system” •  “more about product management and testing” •  “handling the process of developing similar software” •  “different approaches how SPLs could be implemented”

• They learned: •  All students replied that they learned the basics of SPL engineering

and the involved processes (application and domain engineering) •  Eight students replied that besides the basics and theory of SPLs

they also learned how to apply SPL engineering in practice based on presented case studies and implementation examples

•  “I learned several approaches how SPLs could be implemented, the advantages & disadvantages of them and got live experience of how much programming effort each approach takes”

32


Missed student interests • More practical examples and tools or more about recent case studies or experience reports from SPLs

• They wanted but did not learn about the barriers for adopting SPL concepts in a real software organization and how to introduce the SPL culture in a company

• Would have wished for more links between model-driven engineering and SPL engineering

•  “SPL engineering often seems to be presented as a (too) ideal approach to develop software but does not (enough) take into account the reality that existing (legacy) software still needs to be maintained and/or refactored into an SPL”

33


Challenges students perceive • Seven students (all without industry job) think that “it is very

challenging to understand the general concept of (large-scale) reuse as a student”

• Understanding the basic concepts of SPL engineering, especially “imagining how it will work in practice is difficult”

• A lot of background knowledge is required to understand SPLs, especially a lot of experience in software engineering

• Difficulties understanding “cross-cutting concerns” and “feature interactions”

• Other challenges: •  visualization of variability; understanding the mapping of abstract

features to concrete software artifacts • maintenance of the many variants created (derived) based on a

product line

34


Suggestions to improve teaching by students •  Include more practical activities and discussions and details

on how to transfer the theoretical knowledge to the real world (e.g., “providing detailed solutions to practical exercises”)

• Let students try out commercial SPL tools

• Compare SPL engineering with other recent approaches such as agile development

•  “It would be a good idea to work on one big, concrete project throughout the semester to experience the development of a SPL from scratch”

35

Wrap-up and Future

36

Stakeholders of Software Product Line Teaching

Survey on Teaching of Software

Product Lines

SPLTea Workshops on Software Product

Lines and Teaching

Survey on Learning about Software

Product Lines

Researchers Educators Practitioners Students

Our Initiatives

Analyses of the State of Practice

Curriculum (e.g., length of course, topics covered, audience, …)

Material (e.g., textbooks, tools used, ...)Experience Reports

Challenges

Recommendations

Suggestions to improve the practiceKey messages and concrete actions

Analyses and Discussions

Our Results


Recommendations (based on insights) • Curriculum integration: many possible scenarios

•  The design of a full course is not the only option for educators. Another realistic alternative is to teach about SPLs as part of existing courses.

•  Teach SPLs at the graduate levels •  Towards a baseline curriculum and evaluation scheme; customizable!

• From exercises to case studies •  Despite early efforts, encompassing all SPL activities within an unique

case study is an open and difficult challenge. •  Therefore it is more realistic to only consider a subset of SPL activities •  Tools are no longer a source of anxiety and an obstacle: educators

now have a collection of tools for modeling and implementing SPLs

•  Innovate, take risks, possibly deviate from current practices, and eventually report on failures or successes

http://teaching.variability.io

37


Wrap up • Software product line (SPL) and variability engineering: well-

established in research and industry

• Teach or perish! • How to teach SPL and variability?

•  Instance of a general problem faced by other specialized fields (e.g., software testing, model-driven engineering, software language engineering)

• Surveys and insights from researchers, industry, educators, and students

Repository: http://teaching.variability.io/ TOCE article: https://hal.inria.fr/hal-01522779 SPLTea workshops: http://spltea.irisa.fr Eduction and SPL in the CfP of SPLC: http://splc2018.net/call-for-papers/call-for-research-papers/

38