1. Assessment Plan - Four Column · 2020-01-30 · 1. Assessment Plan - Four Column PIE - Instruction: Research and Institutional Effectiveness Unit Narrative Reporting Year 2018-19

1. Assessment Plan - Four Column

PIE - Instruction: Research and Institutional Effectiveness Unit

Narrative Reporting Year2018-19Contact Person: BARBARA MCNEICE-STALLARDEmail/Extension: [email protected] 909-274-4109Program Planning Dialog: The RIE Office operates as a centralized office under the Instruction Team. All researchers on the campus are hired by and report directly to the RIEDirector. The Director assigns researchers to work with the project teams who funded their salaries or who ask for RIE services. In order to improve efficiencies, the sameresearcher will typically be assigned to the same project team. RIE staff work collaboratively with others in the office to improve their efficiencies and learn from each other.Progress on projects is tracked by the staff and manager. Special effort is made to closely track time and effort on federal grants. Time is allocated and subsequently re-allocated according to needs of the project team members.

The Director provides services to the College as a whole and endeavors to manage the approximately 300 activities and projects that are required by the multiple fundingsources including the District, Student Equity and Achievement Program (SEAP), Guided Pathways, and Grants (federal and state). The centralized model allows forefficiencies and institutional effectiveness.

For PIE work, RIE continues to use group meetings to provide the Team with an opportunity to engage in the PIE process. This allows for synergy of thoughts, open input, anddeeper discussions about the value of what we do and how we contribute toward the higher-level goals of the College.

External Conditions, Trends, or Impacts:

Theme: Data Driven Decision-making

• Institutional directives (AACC Pathways, Statway) as well as federal and state mandates (California Chancellor's Office Student Success Metrics (SSM)) provides anopportunity for RIE to learn about different types of data, different data reporting processes, and how to engage groups in higher-level data conversations. The dataconversations allow areas to reflect on the data and other aspects of the situation to make the best possible data-driven decision to improve student success. These projectsallow RIE to discuss College-wide goals on student achievement measures with committees as well as the Board of Trustees. Based on this continued improvement of ourwork, RIE is advocating for more Data Coaching. Data Coaching should be a faculty-led process that would allow RIE Team members to coach many on data issues -qualitative, quantitative (e.g., focus groups, outcomes assessment, course success).

• Federal accessibility requirements (ADA Compliance) requires many more hours of RIE's time per request. Over the past year, RIE started to master the processesinvolved and understood when it was necessary (e.g., publicly available reports) and when it was not (internal request with no special ADA requests).

11/08/2019 Generated by Nuventive Improve Page 1 of 39

• Ways to access data via Dashboards are becoming more and more requested. PIE has a more robust Dashboard this year. RIE spent time listening to feedback andimproving upon it. RIE works with Student Service areas (EOPS, Access, CalWorks) to build customized Dashboards both with and without MIS data. School of ContinuingEducation (SCE) is also building Dashboards with RIE's assistance. Other Dashboards that would help the College are still to be built including: (1) new funding formula, (2) AB705 (multiple measures), (3) Guided Pathways, and (4) Student Equity. RIE added a member to its team in July 2019 who has expertise in Dashboards.

Some useful links:AACC Pathways: https://www.aacc.nche.edu/programs/aacc-pathways-project/

Statway: https://carnegiemathpathways.org/statway/

Student Success Metrics: https://www.calpassplus.org/LaunchBoard/Student-Success-Metrics.aspx

Internal Conditions, Trends, or Impacts :

Theme: Staffing

• Staffing Assignments: In early July 2019, two vacant Senior Research Analyst positions were filled. These two positions will join M. Tsai in this role and begin to leadprojects and offer standardization ideas to the RIE team for their data warehouse projects. They will also advise the RIE staff in other areas and offer feedback to theDirector. Similar to last year, some staffing assignments (full-time and part-time) were changed to allow for improved customer service and to align with new mandates (e.g.,Student Equity and Achievement Program (SEAP)). Permanent staff are more specialized in their research assignments which provides for a continuity of assistance to theproject teams. Many projects are funded by more than one source and thus could have more than one researcher assigned. RIE is streamlining their staffing assignments sothat a project team only works with one researcher, when possible. This improves efficiencies and allows the researcher to understand the project better and improve theirresearch services offered. Project examples include the following: Basic Skills, Student Learning Outcomes, Student Equity, Grants, Institutional Review Board proposals,Enrollment Management, College-wide Integrated Planning, College-wide internal/external surveys, and Accreditation to mention a few.

Theme: Changing Landscape

The campus continues its engagement in data driven decision making as a result of both the campus culture requiring it as well as outside (external) agencies requiring it.Grants have always had some level of data required but over the past six years the data requirements have become more strategic and directive in what they need todemonstrate our continued eligibility for funding. The College's national research initiatives (AACC Pathways) continues to require a different and deeper review of studentprogress data which require guided, reasonable group discussion about the data (e.g., Pathways Workgroup). These changes also brought with them increasing emphasis onplanning and evaluation and its links to resource allocation. With more reporting needs come the desire to have better, on-demand access to data and thus changingtechnological demands are requiring training to augment skill sets within RIE (e.g., Power BI dashboards). With these changes, there is a need to adhere to the staff's jobclassification while trying to address the new technology needs. RIE paid a professional expert from another College to provide 1.5 days of guided, practical, on-demandtraining on Power BI dashboards which propelled our work creating dashboards. Coincidentally, this person is one of the new Senior Research Analysts beginning in July2019.

Critical Decisions Made by Unit:

Theme: Intrusive Research Support with Data Coaching


The RIE Team continues to be trained to be intrusive in their research support. A good analogy is if they were in a car, they consider themselves to be the GPS while thefaculty/administrator is the driver. Researchers help others recognize how their program/service is helping students. Researchers know the data to connect to the purposeof the project. Researchers strive toward helping others see the overall success of their project. Researchers bring a new perspective to look at and an overall purpose asthey focus on different research perspectives with a different lens. Researchers tend to ask others "How things are working for you?" or "How is your work aligned with whatthe college is doing?" Sometimes project teams have long-term goals (e.g., my students will earn their Associates Degree). In these cases, researchers will, of course monitordegrees earned, but given that it takes years for degrees to be earned, researchers will also add milestones to multi-year projects. These milestones inform the College andrespective programs on progress being made (e.g., number of degree credits earned) and where the congestion or blockage is occurring (e.g., students are less likely to takemath courses needed for degree completed). In the end, as the GPS, researchers will strongly advise, but it is the project leads' decision as to how to use the research foraction. Throughout the process, Researchers use their Data Coaching tools to help the leads' understand their data.

Mt. SAC submitted a Title V grant in July 2019 that includes Data Coaching. In order to expand upon RIE's work and to allow the campus to be more data savvy, it isproposed to have an "Informed for Action Day & Data Coaching" on March 13, 2019. The Day will provide an opportunity for 80-100 employees to attend conferencesessions on campus to engage in the following: (1) research being conducted on campus by our employees and others for their doctorate/master, (2) research beingconducted on campus by our employees to improve programs and services, and (3) research and evaluation tools (Qualtrics). The sessions will use Data Coaching andTraining From the Back of the Room techniques to engage attendees in the research and how to use the findings for action. Informed for Action Day will be part of a largerseries of events to engage the campus in reflective and actionable research using Data Coaching. Various project areas will be highlighted: Guided Pathways, SEAP, SCE, AB705, to mention a few. The Day is being led by B. McNeice-Stallard, L. Greenlee, S. Mestas.

Theme: Technology

• Maximizing Technology

Several critical decisions were made to improve RIE’s use of technology. The IT and RIE Departments jointly asked for a Data Engineer position to help with the complexsystem we use to query the College's millions of student records (i.e., ODSP). The College supported the request (Thanks Mt. SAC!) and the process is ongoing with HR tofinalize the job classification and to post it for hiring. The Data Engineer will help maintain the ODSP as well as to help with writing queries, data blocks, and reports includingprofessional development for staff. While the RIE team is working well doing the aforementioned, their job description does not include deep level monitoring of theintegrity of the ODSP or deep level monitoring the integrity of the data views and queries. Having the Data Engineer take on this role will also help with the new fundingformula as it will include maintaining the validity and reliability of the data elements and its translation into the MIS data which the Chancellor's Office uses for the fundingformula calculations. The Data Engineer position will allow the researchers more time-on-task with direct research endeavors. RIE looks forward to an incredible year ofchange to its efficiencies in 2019-20.

Theme: Mandates

Data Mandates: RIE is required to complete Federal, State, and Accreditation mandates. There will always be more work to do than person-hours available. TheDepartment's critical decision was to continue to ensure that mandates were done and to ensure that non-mandated project team leads were informed of their new projectdeadlines for completion, as necessary.

Theme: Increased Demand for Services


The more the campus understands what RIE can do for them, the more they request our services. This is fantastic as well as challenging. In 2018-19 there were 300 majorprojects plus many more minor projects that RIE supported. Examples of these projects include:-Federal, State reporting (IPEDS, GP, SEAP)-Program-specific evaluations (Paralegal)-Student Centered Funding Formula (SCFF)-Guided Pathways (GPS Website, Business Process Analyses, AB 7050-Enrollment management-Accreditation-Strategic Planning-Grants (Arise, Trio, Perkins)-US Dept of Education Cohort Tracking-Cohort Tracking of Services Students Use-National Projects (AACC Pathways)-Employment Survey (CTEOS)-Graduation Survey-Student Survey (CCSSE)

These projects were funded from different budgets with different requirements: District, Grants (Title V, Title III (Arise), BSSOT), Articulation, Nursing, Basic Skills, StudentEquity and Achievement Program (SEAP), SWP, and Guided Pathways. When researchers do excellent work they are then being asked to complete work on other projects.While the RIE Director assigns work, this requires strong project management to be able to adjust and re-adjust their projects to accommodate this effort (or find otherpersonnel. Given that there are 7.5 permanent researchers, one administrative assistant, and 3-4 professional experts, this is complex. A critical decision made this year wasto increase two vacant research positions to a higher-level research positions. This will allow the office to have three Senior Research Analysts. As these two newresearchers began in July 2019, this will allow RIE to extend its ability to leverage the "lead" efforts of the Senior Research Analysts to lead and coordinate projects with theother researchers, faculty, and administrators. This will allow the Senior Research Analysts to provide direction to improve the quality of the RIE Team's research efforts andit will allow them to have others help them in their complex research projects.

Notable Achievements for Theme A: To Advance Academic Excellence and Student Achievement:The RIE Office operates as a centralized office under the Instruction Team. All researchers on the campus are hired by and report directly to the RIE Director. The Directorassigns researchers to work with the project teams who funded their salaries or who ask for RIE services. In order to improve efficiencies, the same researcher will typicallybe assigned to the same project team. RIE staff work collaboratively with others in the office to improve their efficiencies and learn from each other. Progress on projects istracked by the staff and manager through Smartsheet. Special effort is made to closely track time and effort on federal grants. Time is allocated and re-allocated according toneeds of the projects and the priorities of the College.

Reviewing the work of the RIE Team, its Mission, and the Mission of the College, the Team believes that it is fulfilling its Mission. In particular it is achieving higher levelcollaborative inquiry processes and Data Coaching with its work with employees across campus.

INSTITUTIONAL EFFECTIVENESS:• Institutional effectiveness applies to many projects (e.g., AACC Pathways, Student Success Metrics, and Accreditation) and services (e.g., Office management,Project Tracking, and Networking within the College). Our data provides insights into the college on what works and what doesn't work to help keep key stakeholdersaccountable.


• There is a yearly review of College-wide and state-wide indicators (aka metrics) of student achievement, outcomes, and fiscal viability. These data are discussed withthe Institutional Effectiveness Committee as well as the Accreditation Steering Committee and the President's Advisory Council. The conversations revolve around theprogress the College is making on achieving its goals and how its progress compares to the state-level metrics. Examples of key metrics are Institutional-set Standards(ACCJC) and Local Goal Setting (Chancellor's Office).

CULTURAL CHANGE:• Cultural change is an ongoing process at Mt. SAC. As the College becomes more and more involved in measuring the impact of interventions and using thatinformation for improvement (aka use of results) there is more of an openness to keep improving using data-driven decision making. Research on the effectiveness ofinterventions provide support to various initiatives related to cultural change to address performance gaps. Major grants are aimed at promoting cultural change using data(e.g., Title V, Arise). RIE is proud to be part of this collaborative, complex change process.

ASSESSMENT (SLO and placement)• RIE provides support for student learning outcomes assessment. The Educational Research Assessment Analysts are trained to work with faculty on outcomesassessment. They also work closely with the Outcomes Coordinator and Outcomes Committee on many related projects including qualitative review of the College's coursesdue for their 4-year review.

• Changing to a course placement assessment discussion, RIE also provides support for examining the efficacy of course placement practices and provides evidence forfaculty to use to decide if a change in how students are placed should be considered (e.g., English, Math). The RIE Team worked on multiple measures projects including areview of the placement levels of students if we had had their self-reported versus High School Transcripts. This project was for the English and Mathematics Departments.The data was used by faculty as one piece of the puzzle toward deciding how to change the course placement rules. The RIE Team worked closely with IT to support thiseffort as well as with many others across campus. April 2, 2018 marked the beginning of placing students based on their self-reported high school grades and GPA. RIE alsocreated a Multiple Measures Research Action Plan and collaborated with faculty on its nuances. In 2018-19, the Multiple Measures Research Action Plan was executed. TheSenior Research Analyst's time was devoted 99% of the time to this complex project in collaboration with faculty, IT, and administration. The project was nominated for astate-wide IT Award.

ACADEMIC EXCELLENCE:• Cohort (or group) tracking of students are the major projects the RIE Office oversaw in 2018-19 and it will be continuing into 2019-20. These studies provide long-term reports that shed light as to changes/improvements over time in the program and its impact on students (e.g., Basic Skills and Tutoring, Bridge Long-Term CohortProject, Student Equity, AB 705 results of new Assessment Questionnaire placement process). A new Cohort project focuses on measuring students' use of services oncampus (e.g., Tutoring, Arise). Periodic reports are given to President's Cabinet about the status of the project. The project is expanding to be intentional about students'well-being and how programs and services provide safe places for students to be and to learn.

MANDATES:• Our office completes mandated data requests for the federal and state governments including accreditation. RIE's work supports the College's continued efforts todemonstrate the worth of the millions of dollars of federal funding it receives.

Notable Achievements for Theme B: To Support Student Access and Success:


RIE grouped most of its work into Theme A. Theme A and B overlap.

Notable Achievements for Theme C: Secure Human, Technological, & Financial Resources:

DATA INTEGRITY:• RIE ensures data integrity by providing reports based on the existing data to the program faculty and managers who are able to assess the accuracy of the data; theprocess of validating the data is quite extensive. There is data used by various campus groups that is only in existence because of our shadow databases; however, the RIETeam is part of the discussions with the College President's team working to review and improve upon the tracking systems so that all programs/services that students usewill be tracked in Banner (Cohort Tracking Project). This will provide the RIE Team with the opportunity for higher-level analyses of the myriad of places that students areusing in order to improve their academic performance.

RESOURCE ALLOCATION:• One of the outcomes of our RIE research and evaluation reports is that the data can be used as one piece of the puzzle to justify resources for other Teams. We areexcited to be able to provide this service.• Managing the funding for the operations of the RIE Department has been more complex again this past year. We were thankful to receive funding from outside ofour District budget to support our many employees' needs such supplies and conference monies.

PROFESSIONAL DEVELOPMENT:• Given that there are 8.5 permanent classified staff (researchers, administrative) and 3-4 part-time professional experts, RIE continues to provide more time forstructured professional development and norming exercises to ensure consistency in data integrity, research methodology and analyses, reporting, and use of collaborativeinquiry. It will be efforts from the three Senior Research Analysts to improve upon these efficiencies.• Professional development opportunities to present at conferences and opportunities to attend conferences provide the RIE Team with the ability to bring the Mt.SAC research message to the US and with opportunities to learn from others. Examples of conferences attended include: RP Group's Strengthening Student Success (SSS), RPGroup's Researchers Conference; Hispanic Serving Institution.• In early 2019, all RIE staff were required to attend PD related to Training from the Back of the Room. This provided the staff with an opportunity to engage in brain-based learning and to apply that learning immediately. The team used it to help the Student Equity Committee and the Student Preparation and Success Council tounderstand the Chancellor's new and evolving Student Success Metrics and the metrics for the Student Equity Plan.

Notable Achievements for Theme D: To Foster an Atmosphere of Cooperation and Collaboration:

DIALOGUE:RIE Team provides many opportunities for dialogue about research and evaluation needs, results, and uses for the Mt. SAC employees. The RIE Team provides ongoingdiscussions to understand the project team's needs and visions. Examples of some of this work can be seen in Guided Pathways, Student Equity, Tutoring, MultipleMeasures, Arise, Outcomes, and Accreditation, to mention a few.

RIE Team members serve on governance, advisory, and operational committees to promote communication across divisions, units, and departments. This positions the RIETeam to be at the table for many discussions which allows the Team to broaden its understanding of College-wide issues and allows the staff to provide professionaldevelopment to the College. In some cases, the meeting does not begin until the researcher is present - this says a lot about the value placed by others on the researcher.


ROLE ON CAMPUS• RIE is strengthening its role on campus. It continues to use collaborative inquiry to allow the end-user to be a key part of the conversation in all steps of theresearch process including presentation and use of the results. RIE acts as both an advisor as well as a doer. Examples of these projects include Arise and Title V. The RIETeam created consistent reporting templates that allowed for more uniformity as to what is said, how it is said, and how we are able to demonstrate that the projects areclosing the student equity gap. This work was done jointly with grant directors and project faculty.

• The use of RIE's services for directive, constructive enrollment management work provides the College with an opportunity to be more strategic in its research andreporting requests and to able to ask for this information at a moment's notice - as is needed with enrollment management.

• Committees:RIE team members have excellent opportunities to engage in College-wide and project-specific decision making through committees and initiatives as noted below. The RIETeam member is responsible for providing research and evaluation expertise.

*Governance Committees*Accreditation Steering CommitteeInstitutional Effectiveness CommitteePresident’s Advisory Council (guest)President’s Cabinet (guest)

*Academic Senate Governance Committees*Basic Skills Coordinating CommitteeOutcomes CommitteeStudent Success and Support Program Advisory CommitteeStudent Equity CommitteeStudent Preparation and Success Council (guest)

*Operational Committees*Institutional Review Board

*Other Committees/Groups*Arise Steering CommitteeAdministrative Systems Advisory Group (ASAG)Common Assessment Implementation TeamEnrollment Management TaskforceInstruction TeamPathways WorkgroupStudent Services TeamMultiple Hiring Committees


STAFF ACCOMPLISHMENTSRIE Staff members were asked to note in their project tracking system, Smartsheet, anything that made them proud. Some of those thoughts are noted below. Theirthoughts focus on internal/external dissemination, dealing with complex topics, having their research be used, giving the College more access to data, professionaldevelopment, improving processes, creating project tracking systems, tracking students (cohort model), learning from each other, and collaborating with others across thecampus.

***Permanent Employees***Barbara McNeice-Stallard, Director• Project Management: Used and re-formatted Smartsheet for project tracking. Prioritized and re-prioritized over 300 projects.• Funding: Provided and secured funding for many professional development opportunities for staff.• Professional Development: Encouraged more professional development for learning how to use different software to improve displaying complex data. Providedguided conversations with staff for professional development to allow them to learn from each other and create tools, repositories to improve their efficiencies• Staffing: Successfully advocated for more permanent staff based on needs of college.• Data Visualization: Supported and staffed efforts to use MIS data in Power BI for programs within PIE. Provided PIE Training• Provided direction and assistance with integrated planning for Guided Pathways, Strategic Plan, and Local Goal Setting.• Centralized Office: Ran a centralized Office for the whole college. All researchers on the campus are hired by and report directly to the RIE Director. RIE staff workcollaboratively with others in the office to improve their efficiencies. RIE members work closely with the college to use their cohort tracking skills and collaborative inquiry toprovide the best methodology for the project at hand. They work collaboratively with the Information Technology (IT) office to ensure all are appropriately supporting eachother and the projects each Office should efficiently handle.

Doris Torres, Administrative Specialist III• Continued to learn new operations within RIE’s parameters of operations• Well versed in Budget Balancing with the direction of BS, SSSP C&T, Supplies• Learned Smartsheet and supporting Team in its use of it• More familiar with Director and knowing priorities within the office

Maria Tsai, Senior Research Analyst• Worked collectively on SSSPAC and IT on Building AQ Version upgrades and Version 2 by Validating AQ placement by AQ rules, Providing preliminary AQ dataupdates• Supported AACC Pathways project annual data needs• Supported Title III & V Grant eligibility application• Trained/Coached researchers on various research projects• Worked collaboratively with a large team of faculty, classified, and managers to support all aspects of Phase 1 & 2 Assessment Questionnaire implementation, datavalidation, and impact analysis on enrollment, course success, and student equity. Presented at and attended numerous local, state, and national conferences todissemination Mt. SAC project work and to learn more to bring back to the College

Lisa DiDonato, Educational Research Assessment Analyst• Presented at state and national conferences. Presentation of research work conducted through the AANAPISI grant with tips on how other institutions and dolikewise. This presentation also covered how to use data to effect change. Program officer for the United States Department of Education was impressed by our work and


told the group that this is the type of research they should all have for their grants.• Completed Annual Performance Reports for Grants (Arise and Title V)• Completed data Reports to support the Grants (20+ reports for Arise 30+ reports for Title V)• For the Arise program, a hardcopy letter and email are sent to students informing them of their progress on the guided pathways variables of the Arise program.This project has been well received by the college and at National Conferences as a method of keeping students informed about their progress. Other programs have statedthey planned to use this in their work.• Supported creating Student Equity Plan• Supported Connect 4 program on High School Feeder reports

Annel Tagarao, Educational Research Assessment Analyst• Provided data for national, federal cohort project for US Dept. of Education’s Projects• Coordinated CTE Current student Survey with CTE Deans and reported on it• Assisted Associate Vice President of Instruction on Enrollment Management projects and provided direction to professional expert• Supported Outcomes Committee and Institutional Effectiveness Committee in their program review work• Expanded Power BI into PIE, Dashboards. Provided college-wide PIE training.• Conduct student and faculty focus groups, as well as develop and analyze an online student surveys. In the end, integrate the qualitative and quantitative data to inform the campus about the impact of the Assessment Questionnaire on both students and faculty. The process was highly informative with regards to highlighting areas ofstrength with respect to the the new placement process. But we also found areas that were in need of improvement to better impact and support students and faculty.

John Barkman, Educational Research Assessment Analyst• Provided data to researchers to support their work to more easily display the complex Student Equity Data for the College• Provided data for Student Equity Projects (e.g., STEM, Library)• Supported creating Student Equity Plan and data visualizations• Provided data visualizations in PIE for the College.• Provided program-specific data visualizations (e.g., EOPS, CalWorks)• Provided multiple reports regarding usage of the Center and persistence in STEM courses, both overall and disaggregated by demographics. The project createdreporting queries and templates which will allow for easy reporting of persistence in a list of courses, and reporting by demographic of any metric expressed as a rate.• This has been a busy year for the development of state-wide metrics to measure what is happening at the Community Colleges. In order to help understand thevarious different metrics coming from different initiatives, I prepared lists of all the metrics showing their similarities and differences.

P. Marcell Gilmore, Educational Research Assessment Analyst• Increased usage of Venngage infographics• Promoted within the department to work on Guided Pathways• Member of the Pathways Workgroup and supporting researcher on Pathways projects• Presented AACC Pathways complex data to the Pathways Workgroup with the Senior Researcher, M. Tsai• Conduct student and High School Outreach (HSO) focus groups in order to attain their feedback on the new Guided Pathways website. Feedback from the processwas highly helpful in shaping the GPS website to be more functional for students. For the feedback session, this was also an opportunity to have IT in the room as weobtained feedback from students. IT made edits to the web page as suggestions were proposed. It was an excellent example of taking action on what students had to say (i.e., valuing student voices).• Conduct student and faculty focus groups, a well as develop and analyze an online student surveys. In the end, integrate the qualitative and quantitative data to in


form the campus about the impact of the Assessment Questionnaire on both students and faculty. The process was highly informative with regards to highlighting areas ofstrength with respect to the the new placement process. But we also found areas that were in need of improvement to better impact and support students and faculty.

Vera Froman, Educational Research Assessment Analyst• Coordinated the student sexual assault survey from start to finish - Developing survey, working w/committee to edit/reverse, sent out to students (working with ITto get this out), data analysis & report writing• Provided support to BSSOT as a substitute, professional expert, and then a permanent employee to keep the research work flowing and attended a nationalconference and brought back ideas to further support this BSSOT work• Conduct student and faculty focus groups, a well as develop and analyze an online student surveys. In the end, integrate the qualitative and quantitative data to inform the campus about the impact of the Assessment Questionnaire on both students and faculty. The process was highly informative with regards to highlighting areas ofstrength with respect to the the new placement process. But we also found areas that were in need of improvement to better impact and support students and faculty.

Alexsandra Malinowska

• Conduct student and faculty focus groups, a well as develop and analyze an online student surveys. In the end, integrate the qualitative and quantitative data to inform the campus about the impact of the Assessment Questionnaire on both students and faculty. The process was highly informative with regards to highlighting areas ofstrength with respect to the the new placement process. But also found areas that were in need of improvement to better impact and support students and faculty.• Supported creating Student Equity Plan and data visualizations

***Professional Experts ***(project specific, at-will employees)

Cathy Stute• Met with and engaged with Project managers to learn their visions for the students. Felt their passion for the projects, and collaborate with them to develop toolsto extract useful, actionable data and report on their achievements and success• Provided oversight to Basic Skills projects to ensure all researchers were on time with their deliverables• Assisted Associate Vice President of Instruction on Enrollment Management projects and provided direction to professional expert

Katherine Vrooman• Met with and engaged with Project managers to learn their visions for the students. Felt their passion for the projects, and collaborate with them to develop toolsto extract useful, actionable data and report on their achievements and success. Supported Basic Skills, Guided Pathways, and other projects. Provided beginningmethodology for the Guided Pathways project that asks students their perspectives on what they had to do to get to this point in their academic careers; this work alignswith the business process analysis done with employees to understand what has to happen for students to earn a degree or certificate at the college.

Wook Kim• Assisted Associate Vice President of Instruction on Enrollment Management projects and provided direction to professional expert• Supporting all aspects of President's Cabinet project to track students' use of services across the campus (aka Cohort Project).

Jamille Borer


• Provided data visualizations in PIE for the College.• Supported many college-wide surveys (PIE, Safety)• Provided program-specific data visualizations (e.g., EOPS, CalWorks)

Contributors to the Report:

Director, Barbara McNeice-Stallard

Senior Research Analysts: Maria Tsai

Research and Assessment Analysts: Lisa DiDonato, John Barkman, Annel Medina Tagarao, P. Marcell Gilmore, Vera Froman

Administrative Specialist III: Doris Torres

Professional Experts: Cathy Stute, Jamille Borer, Katherine Vrooman, & Wook Kim

Unit Goals Resources Needed Where We Make an Impact: Closing theLoop on Goals and Plans

Status: Active

Reporting Year: 2018-19

Report Directly on Goal: RIE continues to provide manyopportunities for dialogue across campus, withincommittees/groups, and related to specific projects.

Data Coaching and collaborative inquiry are standardpractices within the RIE Team that allows others to takeownership of their data and be part of the process fromstart to finish. Data coaching is the process of "engagingcollege personnel in providing direct support to their peersto build their capacity to access and use data" (perChancellor's Office Guided Pathways document on GetStarted...Data Coaching to Inform Guided PathwaysDevelopment athttps://www.mtsac.edu/research/GPDataCoaching-RPMarch2019.pdf . Collaborative Inquiry is also a processfor reviewing data and research in a manner in which manypeople are at the table to examine the educational practiceand improve upon it.

While the RIE Team is excellent at working with those who

% Completed: 25Report directly on Goal

Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

DIALOGUE - RIE will promote andfacilitate purposeful collegeengagement in inclusive, informed,reflective, and intentional dialogueabout institutional quality andimprovement based on reliableinformation and provide clear inquiry,meaningful interpretation, and broadcommunication.

Date Goal Entered (Optional):06/26/2017


Unit Goals Resources Needed Where We Make an Impact: Closing theLoop on Goals and Plansrequest their services (either via pre-decided fundedprojects (e.g., Student Equity, Grants) or ad hoc researchrequests), there is still a need to advise the broader campusas to what the RIE does and the important findings fromspecific projects. This work will be developed via the webpage re-design, branded communications, and specialized-focus on projects (e.g., Student Equity). (07/15/2019)

Report directly on Goal

Describe Plans & ActivitiesSupported (Justification of Need):We need to understand how todevelop an effectivecommunications plan, develop it,and execute it. We would likeassistance from Marketing andCommunications

What would success look like andhow would you measure it?: RIE'sweb page would be updated andmore useful. The College employeeswould recognize RIE's Brand andknow some facts about our students.Type of Request: MARKETING:Requests for services in the areas ofgraphic design, news, andphotography, posting information,communication and social media.Planning Unit Priority: MediumDocumentation Attached?: YesRelated Documents:Research for InstitutionalEffectiveness Process v2.pdfResearch and InstitutionalEffectiveness 4.pptx

Reporting Year: 2018-19% Completed: 25RIE accomplished many steps toward creating theirCommunications Plan:1. Met with Marketing and Communications to draft aCommunications Plan2. Reviewed RIE web page. Took it down and began a fewweb page that is ADA compliant. Worked with IT to manageprocess and content. Creating new staff profiles via new ITtechnology - including profiles and pictures (optional).Working offline to edit narrative.3. Hired Professional Expert to help align RIE'sCommunications Plan with college, state, federal, andnational initiatives.4. As part of the Guided Pathways Sub-group #1 Work, RIETeam met with Pedro Suarez, IT to create an RIE BusinessProcess Map of RIE's work.

Next Steps:a. create a branding for communicationsb. re-review and finalize web page narrative includingshowcasing all employees' profilesc. finalize and activate Communications Pland. add to the RIE web page the RIE Business Process Map,RIE Organizational Charte. determine outline for project-specific highlightf. secure a section for Data Coaching - (1) what is it, (2) howdo we do it at Mt. SAC, (3) link to potentially new Title VGrant, Guided Pathways, and proposed Informed for ActionDayg. Other... (07/15/2019)

Lead: B. McNeice-Stallard

Request - No Funding Requested -Communications Plan

INSTITUTIONAL EFFECTIVENESS - RIE


https://mtsac.tracdat.com:443/tracdat/viewDocument?y=Wzp9xRLffQhh

https://mtsac.tracdat.com:443/tracdat/viewDocument?y=Wzp9xRLffQhh

https://mtsac.tracdat.com:443/tracdat/viewDocument?y=DGOVXchGCwyt

https://mtsac.tracdat.com:443/tracdat/viewDocument?y=DGOVXchGCwyt


Status: Active


Report Directly on Goal: The College continues to supportRIE Team in its endeavors whether it be funding permanentresearch positions, funding operational resources (supplies,software), or inviting the RIE Team to be at the table formajor events (Multiple Measures and AB 705). RIE Teammembers are considered invaluable members of groups andcommittees. Their input and insights are valued by theCollege and sometimes sought before major decisions aremade.

Future endeavors of the College to support Data Coaching(Title V Grant Submission) and Data Visualization (Power BIDashboards in PIE (TracDat)) are clear indicators that theRIE Team has a pulse on the needs of the College. The RIETeam's Institutional Effectiveness Goal is one that will neverend. (07/15/2019)

% Completed: 50


The needs of the College will be better served in 2019-20and onward because the College supported the conversionof two research positions into two of our highest-levelpositions: Senior Research Analyst. The higher-levelresearchers will be able to lead groups in project discussionsand lead research execution. These two researchers willlead the efforts of Enrollment Management, GuidedPathways, and Student Equity. (06/12/2019)

% Completed: 100


RIE is improving its institutional effectiveness. Therecontinues to be an evolution of the need for higher-levelresearchers who can lead groups in project discussions andlead research execution (aka Senior Research Analysts).Changes in funding streams and deadlines also impact ourability to achieve this goal. (05/22/2018)

% Completed: 25


Reporting Year: 2017-18% Completed: 75

Request - Full Funding Requested -Software

Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

will promote and facilitate timely,integrated, ongoing and systematicresearch, planning, and evaluation ofthe College's Mission, Goals, andinstitutional effectiveness based onreliable information and provide clearinquiry, meaningful interpretation,and broad communication.




Describe Plans & ActivitiesSupported (Justification of Need):Provide superior support to projectsusing updated Software: Remark,GIS, SPSS, Display Fusion, ESRI,Venngage & Maintenance. We useda new software, Venngage,extensively in 2017-18 to displaycomplex data in an info-graphicmanner. It is having a far reachingimpact.

What would success look like andhow would you measure it?: RIETeam will be well versed in the use ofthe software and it will have animpact on the College. For example,we have Venngage and it is beingused extensively to support displayingcomplex data in visually appealingmanners.Type of Request: SUPPLIES ANDMATERIALS: Instructional suppliesand materials are items to be used bystudents, faculty and other personnelin connection with an instructionalprogram, less than $500.Planning Unit Priority: MediumDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 5000

RIE received licensing for a few software packages as well asScanners and tablets. This allows the department toconduct higher-level research. The tablets are allowingmore just-in-time discussions with programs. Theseresources are improving our efficiencies. There is acontinued need for yearly maintenance agreements.(03/21/2018)

: Staff were able to takeadvantage of the software andtraining for new and ongoingresearch projects. (08/03/2017)

Reporting Year: 2016-17% Completed: 100The monies were used to help update the Software neededfor the RIE office's operations. This provided the staff withan opportunity to leverage different software for theirneeds. (07/17/2017)


Describe Plans & ActivitiesSupported (Justification of Need): Ifthe complexities of the basic skillgrant (BSSOT) increase, it isimportant to consider the workloadon the part-time permanent

Reporting Year: 2018-19% Completed: 75The College administration is aware of this issue andbelieves a solution is possible to fund the 47.5% goingfoward. A permanent researcher was hired for the positionand is being funded via BSSOT funds until June 2019.(06/12/2019)


Request - Full Funding Requested -47.5% permanent EducationalResearch Assessment Analyst



research position to be increasedfrom 47.5% to full-time (for the lifeof the grant). This increased numberof hours will provide more in-depthresearch opportunities for thefaculty doing this work.

What would success look like andhow would you measure it?: Part-time researcher moved to full-time.Type of Request: STAFFING: Requestsfor permanent employee positions ortemporary/hourly employees.Planning Unit Priority: MediumDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 39900

% Completed: 0The BSSOT grant will expire in June 2019. It is important toconsider where this position, 47.5% researcher, will be re-assigned in order to meet the current and future needs ofthe College. If the College secures a new Title V grant,perhaps this position could move to this research.(05/22/2018)Lead: Barbara McNeice-Stallard

Describe Plans & ActivitiesSupported (Justification of Need):Project Experts: At-will researcherswith specialized high level researchwho are assigned to completeondemand institutional projects.College-wide projects are timeintensive and can be supported byProfessional Experts (e.g, surveysCCSSE, CIRP, Employee). ProjectExperts are required to lead theinitiative. There is no standingbudget line item in the RIE budgetfor this type of support. Having anongoing Project Expert line itemwould allow RIE to service theunpredictable needs that come fromvarious agencies that the College isrequired to satisfy

What would success look like and

Reporting Year: 2018-19% Completed: 25RIE is moving away from relying on professional experts andmore toward hiring permanent research positions. This isdifficult given that so much funding for the RIE Departmentrelies on non-District funding. The recent hiring (July 2019)of two Senior Research Analysts (replacements) shouldhelp propel the efficiencies of the RIE Department once newprotocols and procedures are set in place.

The IT Department is also working with HR to post the newData Engineer job description. This position will provide alink between IT and RIE as well as the rest of the campus.The link will allow RIE to have standardized data views fordata extractions and will allow for closer watch on dataintegrity of the operational data store. The Data Engineerwill work on campus-wide projects that cross departmentssuch as the Cohort Tracking Project. This Project's outcomeis to track students' use of programs, services, and centersin order to show the value added by these services onstudent success.

The long and short of it is that the filling of the RIE staffLead: B. McNeice-Stallard

Request - Full Funding Requested -Professional Experts



how would you measure it?:Spending less time trying to findmoney for institutional researchitems that must be done.Type of Request: STAFFING: Requestsfor permanent employee positions ortemporary/hourly employees.Planning Unit Priority: HighDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 75000

positions and the new Data Engineer will reduce, to somedegree, the reliance on professional experts - over time.(06/12/2019)Reporting Year: 2017-18% Completed: 0In the past, funds (surveys, professional expert) for thiswork was secured from the Accreditation Budget on-demand (Immediate Needs Request). This could be done onan ongoing basis. From a fiscal perspective it is difficult topredict when the funding will be needed as the project isdone every 2-3-4 years as needed. It is important to alsoconsider how RIE should be moving away from professionalexperts and more toward permanent research positions.(07/15/2018)

Describe Plans & ActivitiesSupported (Justification of Need):Standardization of SE (StudentEquity) reports and presentations tofind:-a clear way at looking at the data, ofclosing the equity gap (project basisand college wide)-a clear comparison for each subpopulation (e.g., Minority Males inMath versus those in specificprogram)-leveraging each other's reports andpresentations (guidelines,templates)-cohort tracking template 5/30/18added

Some professional development maybe needed to learn from others.

What would success look like andhow would you measure it?: More

Reporting Year: 2018-19% Completed: 25For the 2019 Student Equity Plan, J. Barkman, L. DiDonato,M. Tsai, and professional experts worked with members ofthe Student Equity Committee and Student Preparation andSuccess Council to review different data provided by theChancellor's Office. Using their skills learned from "Trainingfrom the Back of the Room" by E. Versace, RIE useddifferent training methods (e.g., different types of candyused to represent sub-groupings) to help the audienceunderstand and engage in the equity data. Heat maps wereused to represent the information with detailed dataprovided. The difficulty occurred as the Chancellor's datawere considered invalid and difficult to impossible toreplicate. The RIE Team did the best they could to supportthe College in using the required invalid data by thedeadlines provided by the Chancellor's Office.

RIE worked with members of the Student Equity Committeeto review different ways to present equity data. Using anew software program, Venngage, RIE presented manyinfographic examples. A Mountain and Fish infographic wasdeemed exceptional by the members. This infographic wasused in fall Flex 2018. Two versions were (1) pictures,numbers, but no groups identified and (2) all informationpresent. This allowed the audience to engage in equity


Request - Full Funding Requested -Standardizing of Student EquityReporting and Data Coaching



conversations about the details of theequity gap will be present incommittee minutes (e.g., SECommittee).Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT(POD): Requests that provideprofessional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 2500

conversations in a more holistic manner. The AssociateDean, Eric Lara, is using these infographics also inconference presentations. Groups are also enjoying thepresentation format. The July 2019 meeting with theDirector of RIE, B. McNeice-Stallard, Associate Dean, EricLara and the new Senior Research Analyst, Jaime Rodriguez,revealed the need to update the document for fall 2019.

Other data infographics and data updates were requestedfor both college-wide and project-specific endeavors.

The newly hired Senior Research Analyst has expertise indashboard system, Power BI. This will allow the College todig into its own MIS Data and local data to see the equitypicture from multiple perspectives and take actionaccordingly. (06/12/2019)

Describe Plans & ActivitiesSupported (Justification of Need): In2018-19, RIE will be leveraging the"lead" efforts of the Senior ResearchAnalysts to lead and coordinateprojects with the campus and otherresearchers. This will allow theSenior Research Analysts: (1) to workwith limited direction on largeresearch projects, (2) to providedirection to improve the quality ofthe RIE Team's research efforts, and(3) have other researchers help theSenior Research Analysts in theircomplex research projects.

What would success look like andhow would you measure it?: The RIESenior Research Analysts would be ina lead role helping to ensure theeffectiveness of the RIE Teammembers project work and be


Request - No Funding Requested -Leverage Senior Research Analysts



effective at leading large-scaleprojects across campus.Type of Request: OTHER OPERATINGEXPENSES AND SERVICES: Requestsfor contracted, legal/ audit, personal/consultant, rent/ leases, repairs/maintenance, and other misc.services. May also include request fortravel and conference that does notrequire the assistance of POD.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 0On-Going Funding Requested (ifapplicable): 0

Describe Plans & ActivitiesSupported (Justification of Need): In2018-19, the Multiple MeasuresResearch Action Plan will beexecuted. The complexities of theMultiple Measures Research ActionPlan will be known in early fall 2018.It is anticipated that the new SeniorResearch Analyst and the newGuided Pathways EducationalResearch Assessment Analyst willwork with the current SeniorResearch Analyst (M. Tsai) tocomplete this work each semester.At this time, it is unclear if furtherfunding is needed for supplies,technology.

What would success look like andhow would you measure it?: 2 Related Documents:

Reporting Year: 2018-19% Completed: 25The Multiple Measures Research Action Plan is progressing.The Senior Research Analysts (M. Tsai) is workingcollaboratively with IT, faculty, and administration tosupport implementations (AQ 1 and AQ 2). Analyzes arebeing conducted. The recent hiring (July 2019) of two SeniorResearch Analysts (replacements) should help propelResearch Action Plan and the efficiencies of the RIEDepartment once new protocols and procedures are set inplace.

The IT Department is also working with HR to post the newData Engineer job description. This position will provide alink between IT and RIE as well as the rest of the campus.The link will allow RIE to have standardized data views fordata extractions and will allow for closer watch on dataintegrity of the operational data store. The Data Engineerwill work on campus-wide projects that cross departmentssuch as the Cohort Tracking Project. This Project's outcomeis to track students' use of programs, services, and centersin order to show the value-added of these services onstudent success. (06/12/2019)

Lead: Barbara McNeice-Stallard, M.Tsai

Request - No Funding Requested -Multiple Measures Research ActionPlan will be executed.



months after grades roll, RIEcompletes the research projects forthat semester and presents it to Mathand English Departments as well asthe Guided Pathways Committee.Type of Request: IT SUPPORT:Requests for projects related to theimplementation, integration,application, delivery, and support ofinformation and instructionaltechnologies.Planning Unit Priority: HighDocumentation Attached?: YesRelated Documents:ResearchPlan-draft-November 8-2018-V3.docx

MMRAgenda-Sept30-2018.pdfResearchPlan-draft-November 8-2018-V3.docx

Describe Plans & ActivitiesSupported (Justification of Need):For Guided Pathways, there is aneed for a researcher to providesupport for the various researchprojects and to liaise with the RIEDepartment to ensure efficiencies ofscale are maintained. This personwill need to understand the need foralignment with the Chancellor'sOffice metrics on Launchboard aswell as build a just-in-timedashboard for Mt. SAC to monitorthese and other metrics of ourmilestones and successes.

Per EZ Salary Projection FY 2018-19(Funded by Guided Pathwaysbudget)


Reporting Year: 2018-19% Completed: 100The College supported hiring of a researcher to specificallyprovide research and evaluation direction to GuidedPathways. The position was filled in fall 2018 by an internalresearcher, P.M. Gilmore, who was supporting StudentEquity research. Thank you, Guided Pathways!(06/12/2019)


Request - Full Funding Requested -Educational Research AssessmentAnalyst (new)


https://mtsac.tracdat.com:443/tracdat/viewDocument?y=qpZV8KmL933X


https://mtsac.tracdat.com:443/tracdat/viewDocument?y=XxD1NCdo7yaf



how would you measure it?: Therewould be research support for thevarious research projects and to liaisewith the RIE Department to ensureefficiencies of scale are maintained.Type of Request: STAFFING: Requestsfor permanent employee positions ortemporary/hourly employees.Planning Unit Priority: HighDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 103515

Describe Plans & ActivitiesSupported (Justification of Need):Student Tracker Premium: ThePresidents Membership fee currentlycovers the basic fee for RIE andA&R's access to transfer data andtranscripts from the NationalStudent Clearinghouse (NSCL).Student Equity funding in 2016-17supported a temporary trial periodfor a Premium Service that allowedRIE staff to significantly reducedtheir time on task to work withcomplex unitary transfer datareceived from NSCL. Every time werun transfer data for a cohort ofstudent we have to go through avery complex process to filter andre-arrange the data for ease ofreporting. The Premium Service is anexceptional option that we believesaves many more hours of RIE's workflow than the yearly fee required.

Reporting Year: 2018-19% Completed: 100Funding was received for the President's Membershipaccount to fund the added costs on an ongoing basis.(06/12/2019)


Request - Full Funding Requested -National Student Clearinghouse:Premium Additional Package(transfer data)



What would success look like andhow would you measure it?: RIEwould save many hours of its workflow time with this product. Thissavings would allow the team tospend more time on actual research.Type of Request: OTHER OPERATINGEXPENSES AND SERVICES: Requestsfor contracted, legal/ audit, personal/consultant, rent/ leases, repairs/maintenance, and other misc.services. May also include request fortravel and conference that does notrequire the assistance of POD.Planning Unit Priority: HighDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 2000

Describe Plans & ActivitiesSupported (Justification of Need):Professional and OrganizationalDevelopment (POD) used to find thefunds for this college-wide, onlinesoftware for all employees andstudents. Now that POD is movingfrom reporting to Instruction toreporting to HR, RIE and LibraryLearning Resource Center (LLRC) aretaking the lead on asking for anongoing allocation to support thiswork. There are 400 surveysongoing. Qualtrics allows others tocollaborate on surveys. It allows theRIE office to work work withstudents to provide feedback ontheir surveys. It also is ADACompliant. The price deal we'vereceived in the past may no longer

Request - Full Funding Requested -Qualtrics (College-wide)



be possible. The estimated costs is$50,000 for a 3-year pricing.

What would success look like andhow would you measure it?: Collegeuses Qualtrics for its only onlinesurvey system. Sharing of surveysbetween departments.Type of Request: LOTTERY:Instructional materials that aredesigned for use by pupils and theirteachers as a learning resource andhelp pupils acquire facts, skills, oropinions or to develop cognitiveprocesses.Planning Unit Priority: HighDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 60000

Lead: Barbara McNeice-Stallard andMeghan Chen

Describe Plans & ActivitiesSupported (Justification of Need):The management of the office iscomprised of two areas:(1) Institutional Effectiveness:providing planning, research, andevaluation support for College-wideprojects on institutionaleffectiveness (e.g., Strategic Plan,Student Equity, Guided Pathways,PIE, Cohort Tracking, InstitutionalReview Board, Chancellor’s OfficeData Lake, Student CenteredFunding formula, etc.) and(2) Research: overseeing theresearchers and their research andtechnical skills and projects.

Request - Full Funding Requested -Assistant Director (Range 16)



Similar research offices across thestate have two managementpositions for these two areas: Deanof Institutional Effectiveness andDirector of Research.

Given the current structure of theRIE Office and that fact that itprovides support for both theDistrict and College-program needs,it is recommended to start with anAssisstant Director of RIE. As timeprogresses, the value added inchanging the jobs to a higherposition can be analyzed. It is alsoimportant to note that in 2019 HR isevaluating the management jobclassifications so this may alsochange the suggested structure. Ez-salary used for Range 16.

What would success look like andhow would you measure it?: TheCollege would have a more effectiveprocess and protocol for supportingresearch efforts.Type of Request: STAFFING: Requestsfor permanent employee positions ortemporary/hourly employees.Planning Unit Priority: MediumDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 185013


MANDATES - RIE will support theCollege by fulfilling data requestedfrom federal and state agencies,accreditation bodies, and grantoffices.



Status: ActiveGoal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21Date Goal Entered (Optional):06/26/2017

Status: Active


Report Directly on Goal: RIE continues to provide support tothe ongoing outcomes assessment process. In 2018-19, thissupport was lower as the key RIE Team member was onleave for a bit.

There continues to be a need for the College to re-examineits Outcomes Assessment process and how it aligns withGuided Pathways, Student Equity, and PIE.

RIE looks forward to providing more support and guidanceto the process as it relates to administrative unit objectives(AUOs) and competency-based outcomes. (07/15/2019)


Describe Plans & ActivitiesSupported (Justification of Need):RIE will provide support foroutcomes assessment from all unitsacross campus; Mt. SAC previouslycalled these measures administrativeunit objectives (AUOs). RIE will alsoprovide support for examination ofcompetency-based outcomesassessment as directed by faculty.

As the scope of this work is not yetclear, funding is not requested. It isassumed that RIE will work closelywith the Outcomes Committee andthe Outcomes Coordinator to helpsupport this work across campus.

Reporting Year: 2018-19% Completed: 0Very little support was needed this year for this goal as itrelates to AUOs and competency-based assessment. TheOutcomes Committee and Outcomes Coordinator bothbelieve that more support is needed for outcomesassessment at the College. It is anticipated that theseefforts will increase in 2019-20. RIE is ready to support andprovide direction as needed to the faculty carrying out thiswork. (07/15/2019)

Lead: B. McNeice-Stallard, A.

Request - No Funding Requested -Support Outcomes Assessment

Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

ASSESSMENT - RIE will support andfacilitate the evolution of InstitutionalLevel Outcomes (ILOs), course leveloutcomes, program level outcomesand competency-based learned. ARIE representative will attendmeetings to offer training andsupport to faculty, staff, andadministrators concerning resources.As a member of the OutcomesCommittee, RIE will provideevaluative feedback to ProgramChairs on a random group of courses.




What would success look like andhow would you measure it?:Units/Departments across campushave the knowledge and tools neededto conduct their own outcomesassessment work with support whenneeded from RIE.Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT(POD): Requests that provideprofessional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: HighDocumentation Attached?: No

Tagarao

Status: Active


Report Directly on Goal: RIE continues to work with IT toensure its data security is on target. RIE also works with ITwhen outside entities are requesting data transfers (e.g.,WestEd) via projects already agreed upon by administrationand already approved by the Director of RIE.

The efficiencies of the RIE team will be improved once morestandardized data views/blocks are approved by the three(3) Senior Research Analysts. The addition, from IT, of aData Engineer will also help to ensure that the DataWarehouse is up-to-date and will also help build moreconsistent data views for RIE. (07/15/2019)


Describe Plans & ActivitiesSupported (Justification of Need):This activity allows the RIE teammembers to collaborate with IT in asystematic process.

Reporting Year: 2018-19% Completed: 100The RITA meetings were merged with the weeklyenrollment management meetings. Each week, the RIEteam members are asked for agenda items for themeetings. These items are passed onto IT the day beforethe meeting so that all may be prepared for the meeting.The meetings are very hands-on. Problems are reviewedand solutions suggested. There is a very positiveenvironment created that allows for in-depth collaborative

Lead: B. McNeice-Stallard & A.Bangloy

Request - No Funding Requested -RITA Meetings (ResearchInformation Technology & Antonio)

Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

DATA INTEGRITY - RIE willcollect/maintain/provide useful,objective, reliable, and valid data inan understandable format to collegestakeholders with extensive measuresin data integrity. RIE will keep thisdata secure using the latest methodsIT suggests (e.g., encryptedharddrives and thumbdrives, sharingof data in an encrypted mode).




What would success look like andhow would you measure it?: RIE willattend meetings on a regular basis.Technology knowledge levels of RIEteam members will increase. ITmembers will become moreknowledgeable about RIE's IT needsand operations.Type of Request: IT SUPPORT:Requests for projects related to theimplementation, integration,application, delivery, and support ofinformation and instructionaltechnologies.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 0On-Going Funding Requested (ifapplicable): 0

inquiry. These meetings are exceptionally important to theefficiencies of the RIE Department. Special thanks to IT forsuggesting them! (07/14/2019)

Describe Plans & ActivitiesSupported (Justification of Need):The Directors of IT and RIE agreethat a new position is highly neededto improve efficiencies with datainto and use of data from the datawarehouse. As of now, IT and RIEspend numerous hours troubleshooting data issues, mining thedata, and creating tools andtechniques for building SQL/Accessqueries. A Data Engineer would helpto streamline this process, maintainthe accuracy of the data warehouse,and help to build and maintainqueries that extract data from thedata warehouse in a reliable andvalid manner.

Reporting Year: 2018-19% Completed: 25In 2017-18 PIE, RIE and IT both asked for funding for thisposition. In fall 2018, IT secure the funding and spend therest of the fiscal year with RIE determining the actual scopeof work and job description. Based on this analyses, ahigher-level position was requested and funding received.IT, RIE, and HR collaborated on the final draft jobdescription.

As of July 2019, IT is waiting for a final decision from HRregarding the job classification, salary range and then thehiring process will begin. In the mean time, IT has, with arecommendation from RIE, a professional expert, W. Kim,working via these funds to lead the data collection (IT,interviews) for the Cohort Tracking project. This projectexamines all ways in which students use the College'sprograms and services and how we record their actions.Based on W. Kim's work, the project evolved into a moreholistic view of students before they come into the College

Request - Full Funding Requested -Data Engineer



Job Summary: The Data Engineer willbe responsible for the scripts andprocesses required to extract,transform, clean and move data andmetadata so they can be loaded intoa data warehouse, data mart oroperational data store. Reads,analyzes and digests what Mt. SACwants to accomplish with its data,and designs the best possibleExtract, Transform and Load (ETL)process around those goals. TheData Engineer will be responsible togather, collect, store, do batch orreal time processing on the data andserve it via an ApplicationProgramming Interfaces (API) foropen and easy access. The DataEngineer will also be responsible forevaluating the many Big Data toolsavailable, incorporating them intothe data platform and educatingothers on how best to use them.Est. Salary: Estimate only based onSenior RA

What would success look like andhow would you measure it?: A DataEngineer would maintain theaccuracy of the data warehouse, andhelp to build and maintain queriesthat extract data from the datawarehouse in a reliable and validmanner.Type of Request: STAFFING: Requestsfor permanent employee positions ortemporary/hourly employees.

such as an examination of their academic health (e.g., fiscal,well-being, etc.). When students are at the College, theproject also examines their academic health and mapswhich aspects of health/well-being each program/service isproviding to them.

Through these many data points, there is a clear need formore discussion and professional development related toBig Data and how the College may use it with its current andfuture IT resources. (07/14/2019)

Lead: B. McNeice-Stallard & A.Bangloy



Planning Unit Priority: MediumDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 120000

Status: Active


Report Directly on Goal: The needs of the RIE team forprofessional development are varied. The support providedthe team is helpful. A more consistent, predictable level ofsupport is needed to improve efficiencies and increaseopportunities. (07/15/2019)


Describe Plans & ActivitiesSupported (Justification of Need):New projects are complex andrequire software knowledge that issometimes beyond our current level.Training is needed to help bridgethat gap. Power BI Online and in-person training and travel toworkshops and other College's tosee demonstration of their work.Data Visualization & Training: RIE isrequired to continually update itsway of presenting data in reports aswell as presenting/training the endusers. Leveraging Smartsheet.

What would success look like andhow would you measure it?: RIETeam is fully versed in at least one ofthe major software programs and theRIE Team created at least 1 differentdata visualization and at least onestellar dashboard.Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT

Reporting Year: 2018-19% Completed: 75Minimal costs for training occurred in many ways for PowerBI (data visualization dashboard for PIE within TracDat),Smartsheet (used for RIE Project Mgt). Training was offeredvia online learning modules, webinars, and one-on-onemeetings with the vendor.

For 2019-20, the Administrative Specialist III, D. Torres willbe attending the national Smartsheet Conference. This willbe first time that RIE and the College will support thisexpensive professional development. Funds are being usedfrom the RIE Manager's allocation and being requestedfrom POD. (07/15/2019)


Request - Full Funding Requested -Software Training

Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

PROFESSIONAL DEVELOPMENT - RIEwill participate as learners in acontinuing education process. RIEwill also provide data coachingopportunities for employees so theymay learn more about data availableto them and learn more about how tointerpret and use data. (updated July2019)




(POD): Requests that provideprofessional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 20000

Describe Plans & ActivitiesSupported (Justification of Need):RIE will continue to improve itsoperations by doing the following:• Improve our guidance onuse of data for decision making• Leverage our ownexpertise (skills & knowledge: datawarehouse, leadership, statistics,SQL, Power BI).• Create a skills sheet so weidentify the experts internally• Share lessons learned fromconferences and workshops• Create a space for RIEteam members to give feedback ondraft reports/surveys (e.g., googledrive).• Increase efficiency withuse of templates (e.g., John’stemplates, Lisa’s Templates, Maria’sdata blocks).

What would success look like andhow would you measure it?:Permanent staff will be able to useother's data blocks.Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT(POD): Requests that provide

Reporting Year: 2018-19% Completed: 25RIE continues to leverage its talents and resources toaddress these operational needs. The RIE Team is morecoherent than ever before in helping each other learn andleverage resources. The hiring of a Data Engineer via IT(2019-20) and two (2) more Senior Research Analysts via RIE(July 2019) will help leverage RIE's data warehouse toolsand techniques. (07/14/2019)


Request - No Funding Requested -RIE Professional Development:Efficiencies



professional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 0On-Going Funding Requested (ifapplicable): 0

Describe Plans & ActivitiesSupported (Justification of Need):The department needs additionalresources to cover conference andtravel costs to attend professionalconferences. The RIE team only hasthe Director's $2,000 forconferences plus POD. There aremany ad hoc workshops and eventsthat staff should be attending thatgo beyond these funds.

What would success look like andhow would you measure it?:Permanent staff would have moreopportunities to attend PD state andnational conferences.Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT(POD): Requests that provideprofessional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: MediumDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 12000

Reporting Year: 2018-19% Completed: 0RIE appreciates that there are discussions underway todetermine if it is feasible to provide predictable conferenceand travel funds for the RIE Department so that betterplanning may occur each year. As of now, the RIEDepartment uses funding from many sources: POD, grants,categorical - these are all unpredictable. Each year, the onlyknown monies is the one-time $2,000 given to the managerfor her discretionary use. These funds are not nearlyenough to cover travel, registration for the 8+ RIE Staff andone manager. (07/15/2019)


Request - Full Funding Requested -Conference and travel researchbudget for department staff

Request - Full Funding Requested -Technology: Scanners, Notebookcomputer, Connectors for Large



Describe Plans & ActivitiesSupported (Justification of Need):As the RIE staffing grows, so doesthe need for technology. There are3-4 new work areas being built forRIE staff. Technology is needed foreach including small scanner, laptopwith docking station, connectors formonitors, etc.

Estimated Costs: Unknown (furnituremay already be available; IT mayprovide resources via new computerreplacement program)

What would success look like andhow would you measure it?: Workareas fully functional.Type of Request: NONINSTRUCTIONAL EQUIPMENT:Tangible property with useful life ofmore than one year, other than landor buildings improvements, equal andover $500 per individual item. Usedfor administrative or non-instructional purposes.Planning Unit Priority: HighDocumentation Attached?: No


Monitor

Describe Plans & ActivitiesSupported (Justification of Need):Informed for Action Day is proposedfor March 13, 2020. It will provide anopportunity for 80-100 employees toattend conference sessions oncampus to engage in the following:(1) research being conducted on

Request - Full Funding Requested -Informed for Action Day & DataCoaching



campus by our employees andothers for their doctorate/master,(2) research being conducted oncampus by our employees toimprove programs and services, and(3) research and evaluation tools(Qualtrics). The sessions will useData Coaching and Training From theBack of the Room techniques toengage attendees in the researchand how to use the findings foraction. Informed for Action Day willbe part of a larger series of events toengage the campus in reflective andactionable research using DataCoaching. Various project areas willbe highlighted: Guided Pathways,SEAP, SCE, AB 705, to mention a few.

What would success look like andhow would you measure it?: DataCoaching would be widely used oncampus with an impact on higherlevel questions and higher levelresearch.Type of Request: PROFESSIONAL &ORGANIZATION DEVELOPMENT(POD): Requests that provideprofessional learning opportunitiesfor Mt. SAC employees.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 35000Related Documents:GPDataCoaching-RPMarch2019.pdf

Lead: B. McNeice-Stallard, L.Greenlee, S. Mestas


Report directly on GoalROLE ON CAMPUS - RIE will continueto strengthen its role on campus


https://mtsac.tracdat.com:443/tracdat/viewDocument?y=ejCSbvaBizIR


Status: Active

RIE continues to have a strong role on campus and be a keymember of many teams and committees. This role will befurther strengthened in 2019-20 via the many otherprojects and processes ongoing and proposed in this RIEPIE. (07/15/2019)


Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

through research, evaluation, andplanning services to a variety ofcampus entities. RIE's role will bepivotal to the College's use ofappropriate methodologies toimprove programs and services. RIE'srole in supporting data coaching willbe key this work.


Status: Active


The RIE Team uses innovation in many ways. We economizeour professional development. We join with IT to leverageeach others' skills for training purposes. We work withfaculty and committees to think through and take action oninnovative projects (e.g., Cohort Tracking).

Innovation is an ongoing, cyclical process for RIE that willalways be a main part of how we do our support to andleadership for the College. (07/15/2019)


Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

INNOVATION - RIE will continuallyseek innovative solutions to meet theresearch needs of the College, as wellas improve office efficiency.


Status: Active


Report Directly on Goal: Overall, the College is doing a greatjob at supporting the RIE with its resource needs. Given thelimited monies available, the RIE Team appreciates that ourhighest needs are being met and/or addressed. The highestneed now is for office space that is functional for the workthat the team members need to accomplish. This requireslarge work surfaces, large computer monitors, large storageareas, functional lighting, secure areas, and quiet locationsto be efficient. (07/15/2019)



Request - Full Funding Requested -Office SuppliesGoal Year(s): 2016-17, 2017-18,

RESOURCE ALLOCATION - This Goal istwo fold: (1) RIE will maintainadequate resources to ensureongoing operational needs andachievement of goals and (2) RIE willcontinue to look for evidence that it'sreports provide teams with data-driven decision making opportunitiesto support their aligned subsequentresource requests. (updated July2017)



Describe Plans & ActivitiesSupported (Justification of Need):There is a need to continue toprovide increased office supplies andrelated items to the growing RIEstaff and its growing base ofdepartments and projects it servicesacross the College. Last year'saugmentation helped with this void.We are requesting a further increasethis year be considered ongoingfrom $953 to $2,000.

What would success look like andhow would you measure it?: RIE willbe able to meet the office supplyneeds of the staff in this area.Type of Request: SUPPLIES ANDMATERIALS: Instructional suppliesand materials are items to be used bystudents, faculty and other personnelin connection with an instructionalprogram, less than $500.Planning Unit Priority: HighDocumentation Attached?: NoOn-Going Funding Requested (ifapplicable): 1157

RIE was fortunate to receive $500 from the New ResourceAllocation funding (one time) and $400 from Student Equity(one time) for its supplies. We also received $421 from thework we did for the Federal Government to track studentsfor a financial aid project. Other monies related to thisproject were also shared with other departments whocontributed their time and effort to this work too.(06/12/2019)Reporting Year: 2017-18% Completed: 50The College provided RIE with $500 one time funds to helpwith this void. RIE also received funding from the GrantsDepartment to help cover the remaining deficit. We will stillask for ongoing funding for the increased research officesize. These funds were pivotal to helping RIE to have theresources necessary to improve its efficiencies.(05/24/2018)Reporting Year: 2016-17% Completed: 100The College provided us with $1,100. We were soappreciative of these extra funds as we were able to:secure more supplies for the staff to use in-house and atmeetings and data collection events across campus.(07/10/2017)


Describe Plans & ActivitiesSupported (Justification of Need):We would like to close off our punchlist from our move a few years agointo 4-2435. We look forward to ourwork with Facilities to do so andhave been in contact with themabout this need.

Est. Cost: Unknown


Reporting Year: 2018-19% Completed: 75Some punch list items remain and some new ones areevident (a few more arm chairs material falling apart).(06/12/2019)Reporting Year: 2017-18% Completed: 75Facilities and IT have made some progress on the full punchlist. There is a meeting being set up in August 2018 to try tofinalize the listing. (07/15/2018)


In Progress - Punch List

2018-19, 2019-20, 2020-21Date Goal Entered (Optional):06/26/2017



how would you measure it?: Punchlist is completedType of Request: FACILITIES: Thissection includes minor buildingimprovement projects and alterationsto specific rooms or operationalareas.Planning Unit Priority: MediumDocumentation Attached?: No

Describe Plans & ActivitiesSupported (Justification of Need):Furniture installer training

What would success look like andhow would you measure it?: Trainingreceived in fall 2017Type of Request: FACILITIES: Thissection includes minor buildingimprovement projects and alterationsto specific rooms or operationalareas.Planning Unit Priority: HighDocumentation Attached?: NoOne-Time Funding Requested (ifapplicable): 0

Reporting Year: 2018-19% Completed: 0Two new employees (J. Rodriguez, D. Berumen) will beasked to have an ergonomic evaluation of theirworkstations in July/August 2019. (07/15/2019)Reporting Year: 2017-18% Completed: 100All staff were asked to participate in an ergonomic review.Each person received professional feedback to improvetheir sitting habits by Risk Management. (07/15/2018)


In Progress - To provide anergonomic assessment of the "sitstand stations" in all workstations.

Describe Plans & ActivitiesSupported (Justification of Need):RIE is expanding its staffing and isout of space in 4-2435. We areworking with Facilities to have theback office meeting space convertedinto one office workstation. We arealso assessing the temporaryworkspace in building 21 and we areworking with Facilities to create an

Reporting Year: 2018-19% Completed: 0URGENT: The RIE Office's expansion means it is nowofficially running out of room (something we predicted foryears via PIE). There is a request, that we believe is funded,to covert the back office space into another workstation($10,000). There is also another opportunity for other officespace nearby that is being explored further and it is funded;this space is merely overflow space. Before members aremoved into that space, there is a need to evaluate thehealth and safety of the space and the furniture and IT thatis available for it.

Request - Full Funding Requested -Office Space



office to house all researchers in thenew Bookstore Building (2nd floor).

Immediate known cost is $10,000 formeeting space conversion in 4-2435.

What would success look like andhow would you measure it?: RIEteam would have sufficient officespaceType of Request: FACILITIES: Thissection includes minor buildingimprovement projects and alterationsto specific rooms or operationalareas.Planning Unit Priority: HighDocumentation Attached?: YesOne-Time Funding Requested (ifapplicable): 10000

On July 8, 2019, two new researchers began. This meansthat we are now extra low on workstation space andcarefully examining our daily calendars to ensure all have aworkstation (e..g, vacations, working from home). Whenanother researcher returns from leave in mid-August wewill be sharing office space - working from home moreregularly. The Director discussed the issue with theAssociate VPI and the VPI over the last few years. There ishope that by October, 2019 this issue should be resolved.

Future, Future: The RIE Team is part of the move to the 2ndfloor of the new Bookstore. Excellent! The configuration ofthis space allows for expansion of the RIE Offices. It is hopedthat the College is able to support many workstationsbeyond our current staffing to allow for on-demandexpansion over the many years we will be in this building.

The Director also asked for another manager's office toallow for expansion of the management of the office. Themanagement of the office is comprised of two areas:(1) Institutional Effectiveness: providing planning, research,and evaluation support for College-wide projects oninstitutional effectiveness (e.g., Strategic Plan, StudentEquity, Guided Pathways, PIE, Cohort Tracking, InstitutionalReview Board, Chancellor’s Office Data Lake, StudentCentered Funding formula, etc.) and(2) Research: overseeing the researchers and their researchand technical skills and projects.

Similar research offices across the state have twomanagement positions for these two areas: Dean ofInstitutional Effectiveness and Director of Research.

Given the current structure of the RIE Office and that factthat it provides support for both the District and College-program needs, it is recommended to start with anAssistant Director of RIE. As time progresses, the valueadded in changing the jobs to a higher position can be




Related Documents:5-9-19 budget Mt Sac Buildng 4 Research.pdfMt-St-Admin-2nd-RESEARCH RECONFIGURE B4 AR (1).pdf

analyzed. It is also important to note that in 2019 HR isevaluating the management job classifications so this mayalso change the suggested structure.

It is important to note that the RIE Office operates as acentralized office under the Instruction Team. Allresearchers on the campus are hired by and report directlyto the RIE Director. The Director assigns researchers to workwith the project teams who funded their salaries or who askfor RIE services. In order to improve efficiencies, the sameresearcher will typically be assigned to the same projectteam. RIE staff work collaboratively with others in the officeto improve their efficiencies and learn from each other.Progress on projects is tracked by the staff and manager.Special effort is made to closely track time and effort onfederal grants. Time is allocated and re-allocated accordingto needs of the projects.

The Director provides services to the College as a whole andendeavors to manage the approximately 300 activities andprojects that are required by the multiple funding sourcesincluding the District, Student Equity and AchievementProgram (SEAP), Guided Pathways, and Grants (federal andstate). The centralized model allows for efficiencies andinstitutional effectiveness.

(06/12/2019)

Describe Plans & ActivitiesSupported (Justification of Need):For the future: The College's PODused to find the funding for thiscollege-wide, online software for allemployees and students. Now thatPOD is changing its reporting fromInstruction Team to Human

Request - No Funding Requested -Qualtrics (College-wide)


https://mtsac.tracdat.com:443/tracdat/viewDocument?y=BGFAIOZR5Sts

https://mtsac.tracdat.com:443/tracdat/viewDocument?y=hL3dIUBbCJmU


Resources Team, RIE and LibraryLearning Resource Center (LLRC) aretaking the lead to ask for an ongoingallocation to support this work.There are 400 surveys ongoing.Qualtrics allows others tocollaborate on surveys. It allows theRIE office to work with students toprovide feedback on their surveys. Italso is ADA Compliant. The price dealwe've received in the past may nolonger be possible. The College'slicense expires in 2021. Thecompany is not able to provide aninvoice this far into the future.

What would success look like andhow would you measure it?:Employees and students would beable to continue leveraging Qualtricsfor their research and evaluationneeds.Type of Request: LOTTERY:Instructional materials that aredesigned for use by pupils and theirteachers as a learning resource andhelp pupils acquire facts, skills, oropinions or to develop cognitiveprocesses.Planning Unit Priority: MediumDocumentation Attached?: No

Lead: B. McNeice-Stallard & L.Greenlee


RIE is on the cutting edge of Cultural Change across theCollege. RIE is at many places, at many levels of discussions.We are a key member of teams providing input to changeprocesses and research/evaluation protocols. As such, we

% Completed: 25Report directly on GoalCULTURAL CHANGE - Through our

activities, RIE will support CulturalChange that improve studentachievement and student supportservices and programs. RIE willcontinue to focus on methods to



Status: Active

provide a cross-sectional lens to Cultural Change and westrongly advocate for and support student equity.

There is continued need to train the RIE team on all mattersrelated to equity. The RIE Retreat and RIE Team meetingswere used for this training. These will also be used in thenext year for this training. (07/15/2019)


Goal Year(s): 2016-17, 2017-18, 2018-19, 2019-20, 2020-21

close the student equity gap in theaforementioned.

