Upload
whitney-doyle
View
214
Download
1
Tags:
Embed Size (px)
Citation preview
Building and Managing an Effective K–12 Data Analytics Team
Adam Warner, Texas Education Service Center Region 10 (Dallas) Sharon Reddehase, Double Line Partners Ed Comer, Double Line Partners
NCES MIS Conference - February 15, 2012 - San Diego
Today we’re going to talk about why software development is hard…and some proven methods that can help
Stock photo. Release for web use of this photo on file.
Two Cases Studies with Similar Missions
Provide visualizations and analytics that resonate with educators
Extract, standardize, integrate, and unify data from the many districts sources in a timely manner
Dashboards and AnalyticsUnify K-12 Data from Sources
Ed-Fi
•Texas Student Data System
TSDS
•Automated Dropout Early Warning System
Reveal
All student information and photographs are fictitious.
Texas Student Data System (TSDS)
Standardized Test Results
(SAT/ACT/TAKS)
Student Information System (SIS)
District Connections
Database (DCD)
Other District Source Data
Campus & DistrictSource Systems &
Raw Data
InterchangeSchema
DistrictConnections
Database
UserInterface
This process must be completed for each district/campus
• Common infrastructure designed for scalability• Incremental development over a series of prototype and
limited production releases
Ed-Fi XML Interchanges
Ed-Fi Database Schema
All student information and photographs are fictitious.
Reveal Automated Dropout Early Warning System
Student Information System (SIS)
Analytics Database &
Engine
Campus & DistrictSource Systems
InterchangeSchema
Data Warehouse Analytics
Ed-Fi XML Interchanges
• Leverages same Ed-Fi data standard for scalability now and as a complement to the TSDS down the road• Fully automated, hosted solution that applies dropout research on
most common risk factors (poor attendance, behind cohort, behind credits, failing grades)
All student information and photographs are fictitious.
The Common Formula
Standards-based
architecture
Agile process
Multi-
function
teams
Software development, especially data analytics, is like constructing an office building….
…where you have a framework but really don’t know exactly what kinds of offices it will contain – their size, shape, function, paint scheme…
Stock photo. Release for web use of this photo on file.
The Problem(s) with Software Development
• Customers do not know exactly what they need or want
• Customers change their minds• Customers’ priorities change• It is difficult to solve the needs of many
stakeholders• We can never adequately specify requirements• We are lousy at estimating software tasks• Unpredictable “stuff” happens• It often takes several times to get “something”
right
• Users do not know what they want until they see it (but are good at telling us what they do not like)
• It is difficult to determine the 80:20; that is, what features are most important - before the product is built and deployed
• It is difficult to account for the unknown
Stock photo. Release for web use of this photo on file.
What to do? “Agile” is born
Agile ManifestoWe are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:• Individuals and interactions over processes
and tools • Working software over comprehensive
documentation • Customer collaboration over contract
negotiation • Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Beck, Kent; et al. (2001). "Manifesto for Agile Software Development". Agile Alliance.
“Agile” is not a process – it’s a value system
Principles behind the Manifesto1. Our highest priority is to satisfy the customer through early and
continuous delivery of valuable software. 2. Welcome changing requirements, even late in development. Agile
processes harness change for the customer's competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Principles behind the Manifesto
7. Working software is the primary measure of progress. 8. Agile processes promote sustainable development. The sponsors,
developers, and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10.Simplicity--the art of maximizing the amount of work not done--is essential.
11.The best architectures, requirements, and designs emerge from self-organizing teams.
12.At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Source: http://agilemanifesto.org/principles.html
Agile Implementations
• Scrum• Extreme Programming• Iterative development•Kanban•And many others
Case Study: Building the team
• Business people and developers must work together daily throughout the project.
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Relates to these principles:
It takes many roles to build a K-12 analytics system…
Education World Technology World
Researchers
BusinessAnalysts
Trainers
Support Analysts
Data Modelers
System Architects
Interface Designers Interface
Developers
Managers
QA Testers ETL Developers
It takes many roles to build a K-12 analytics system…
Education World Technology World
Researchers
BusinessAnalysts
Trainers
Support Analysts
Data Modelers
System Architects
Interface Designers Interface
Developers
Managers
QA Testers ETL Developers
We hired experienced private-sector technologists,
who showed adaptability to
different industries, to provide world-
class technical skills.
It takes many roles to build a K-12 analytics system…
Education World Technology World
Researchers
BusinessAnalysts
Trainers
Support Analysts
Data Modelers
System Architects
Interface Designers Interface
Developers
Managers
QA Testers ETL Developers
Teachers
Administrators
Data Specialists
We hired experienced, tech-savvy educators to
provide domain expertise as researchers,
analysts, trainers, testers, and
support specialists
Key Characteristics
• Have the “agile” mindset• Self-driven to quality and doing things right despite schedule
pressures• Able to work today’s tasks while considering the big picture• Not phased by change and adaptation• Can reliably execute a test-driven process• Proactive continuous improvement• Thrive in a collaborative environment
• Critical thinking skills to deal with deep data and business rule semantics• Experienced and expert in the underlying technologies• Deep experience in education data and applications
Reveal Team Building• Upon joining the project, an excellent multi-functional team
had already been assembled • But existing project management and team-building practices
were ineffective, and there were many divisive influences affecting the team
We addressed the divisive influences. However, there was still an “us vs. them” problem between the educators and technologists. We needed a way to manage the project that promoted teamwork.
Stock photo. Release for web use of this photo on file.
Agile Results on Reveal• We turned to Agile principles and Scrum to help solve the problem.• Cross-functional teams were formed. • The teams were given a goal and timeframe in regular intervals.• They were self-organizing - no pre-defined leader or roles.• We abandoned detailed specifications in favor of face-to-face
collaboration and communication. (Documentation was done after the fact, and reflected the software as-built.)
• We kept the focus on the end user, and educational outcomes.• They reflected on their progress and opportunities at regular
intervals.• Soon, the teams began to understand the challenges they each
faced, and how they could better collaborate to build great software quickly and effectively.
• A high-functioning team was born!
Case Study: Managing the team
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Working software is the primary measure of progress.• Welcome changing requirements, even late in development. Agile
processes harness change for the customer's competitive advantage. • The most efficient and effective method of conveying information to
and within a development team is face-to-face conversation.
Relates to these principles:
Scrum is the most popular agile process
http://agilescrum.biz/six-sigma-and-agile-software-development/
Kanban (カンバン ) + Scrum =“Scrum-ban”
Kanban• Specialized functional
groups• Track process steps• Continuous process with
periodic release cycles• Able to expedite stories• Limits for work in process
to identify bottlenecks• Goal is to minimize story
completion time
Scrum
• Multi-faceted, self-organizing teams• Backlog | In process | Done • Time-boxed sprints, each
potentially releasable• Stories frozen during a sprint• Unfinished stories placed on
backlog• Goal is to increase team
velocity
Kanban Board
Roadmap
Major Process Steps
On Deck
Stories in process
Stories that are done, done, done
New Bug or Fix
Original BreakdownStory of Epic
DLP photo. Release for web use of photo on file.
Scrum time at the Kanban Board
DLP photo. Release for web use of photo on file.
Collaborative development• High level of user input and collaboration• Users – extensive stakeholder engagements at the start, monthly follow-up focus
groups, demos, etc.• Customer/sponsor – weekly meetings plus participation in JAD sessions• Business analysts – the internal “voice of the users” with deep knowledge of
education data
• Experienced development team physically in the same room• Architecture and leads with experience with education data and applications • ETL developers, most with deep knowledge of education data• UI/application developers
• Support and deployment plugged in to the development team• Provided remotely by Education Service Center Region 10• Multiple touch points planned for each week• Common issue tracker across districts, Region 10, and development
Other techniques contributing to being “agile”
• Joint Application Development (JAD) sessions• Bring users and developers together on specialized issues where features
and implementations are closely intertwined
• Test driven development• ETL test harness• Automated unit testing
• Continuous integration with automated build and testing• Architecture focus on loose coupling and extensibility• Code refactoring• Capture requirements and knowledge in shared online tools• Metrics database, wiki, code plus docs in subversion
• Final QA accomplished by the business analysts with deep knowledge of education data• “Done-done-done”
Agile results on TSDSYear 1: Iterate and perfect different features• Rapid iterations with users and
customers• UI look and feel• Metric definitions• Data anomalies• Key features and drilldowns
• Evolved and proved the process, the architecture, and factory for development
Year 2: Build out the features, productize and scale• Incrementally delivered a
significant dashboard application• Several refactoring passes• Incrementally on-boarded new
LPR districts• Integrated user support into the
development process
Consistently met all major milestones under budgetExceeded the expectations of our customerReceived high praises from users for usability
How can this be applied to an SEA implementation?• Get the “agile” mindset – will require a change in values• Manage to a prioritized roadmap, not to a specification
• “Wrap” an agile process in a more conventional project management structure
• Focus on building the right team• Agile works best with small teams of highly competent, highly
motivated individuals• If an internal team without agile experience, invest in some
training• If selecting a vendor, make sure they have deep experience with
agile – and check their references• Staff for a highly collaborative process• Full-time attention will be required• Critical to involve end-users through the process
Wrap an agile process in a conventional project/product management process
• Requires continuous collaboration• Healthy “tension” between the project/product manager and the
development leads
More Info on Agile• en.wikipedia.org/wiki/Agile_software_development• www.infoq.com/agile - lots of articles, case studies, free e-
books, including Henrik Kniberg’s excellent e-books:• Scrum and XP from the Trenches (2007)• Kanban and Scrum – making the most of both (2009)
• Other books:• Agile Software Development with Scrum (2001), Ken Schwaber
and Mike Beedle• Scrum in Action (2011), Andrew Pham
These slides available for download at http://www.schoolviz.org/reveal
[email protected]@doublelinepartners.com