Project Blueprint: Templatecvs.holycrosschurch.org/Resources/Eval plan new2.docx · Web viewALWAYS BEING MADE NEW: THE CAMPAIGN FOR THE ELCA. SYNODICAL RENEWING CONGREGATIONS STRATEGIES

ALWAYS BEING MADE NEW:THE CAMPAIGN FOR THE ELCA

SYNODICAL RENEWING CONGREGATIONS STRATEGIES

Evaluation Plan

1

Congregational Vitality Project

12000 N. Washington St. #385Thornton CO 80241

(303) 523-3144

9 / 1 2 / 2 0 1 6

ForTemplate

Evaluation Plan

BackgroundThe Evangelical Lutheran Church in America awarded funds to this synod for a congregational renewal project. These funds came from the Always Being Made New campaign. The purpose of the funds is to allow synods and congregations to experiment with contextually relevant renewal practices. Lessons learned from these experiments will inform future renewal efforts throughout the church.

ObjectiveThe objective of this evaluation plan is to help congregations, synods and churchwide organization understand the impact of each intervention on the participating congregations. This evaluation uses a Participatory Action Research model. That means information gathered under throughout the process will be used to provide feedback so that it can be improved as it goes along. This feedback process will happen both within and across participating synods.

When data are aggregated within each synod and across all participating synods we hope to learn which kinds of congregations benefit from which kinds of assistance.

Another objective of the evaluation plan is to understand the cumulative impact of campaign dollars on congregations. This information along with key learnings may be shared with donors as a form of accountability.

OutcomesEach grantee agrees to measure the outcomes outlined in the customized Logic Model presented as Attachment 1. Specific ways for measuring each outcome are also listed.

ReportingEach grantee will provide three kinds of reports during the life of the grant.

1. Annual Report – due one year after initial grant and each year after that.2. Mid-term report – due approximately half way through the approved grant

cycle3. Final Report – due two months after the end of the grant cycle.

Forms for each report will be provided by the ELCA and completed by the synod. The final report will be accompanied by an external analysis performed by Linda Bobbitt using the data collected from congregations throughout the process.

2

Evaluation tools for each participating congregations

See the timeline for details about when each of these tools is used.1. Pre-test: Lead Readiness survey (Attachment 2)2. Congregational Vitality Survey: Pre and Post3. Leader Reflection: Optional after final training (Attachment 3)4. Congregation Reflection: After program completion (required), and one year

later (optional) (Attachment 4)5. Logic Model score sheet (completed based on final reflection) (Attachment

6)6. Reflection score sheet (completed by DEM and Linda Bobbitt) (Attachment

5)

Data CollectionOne time Events

1. Ask each participant to complete a short evaluation at the end of the event. This evaluation should ask them to name some things they learned and how they plan to use the information when they return to the congregation.

2. Follow-up three months later with an email asking how they have implemented what they learned.

3. Use score sheet and logic model to rate responses. Send all score sheets to Linda.

Congregational Renewal Cohorts: Done for each congregation at beginning, and end of the project

Pretest- before first gathering1. Complete Congregational Vitality Survey with every congregation participant

on a designated Sunday2. Leadership team complete Lead Readiness Survey for congregation3. Pastor confirm average worship attendance and mission support

Final 1. Complete Congregational Vitality Survey with every member on a designated

Sunday2. Pastor confirm average worship attendance and mission support3. Individual members asked to do the Leader Reflection4. Congregational team asked to do the Congregation reflection together

Synod1. Synod planning team will complete synod reflection at midterm and final2. DEM and Linda will score each congregation’s reflections on Logic Model

Score sheet and Reflection Score sheet. They will then compare notes and come to consensus on scores for each participating congregation.

3

ConfidentialityUnless congregations choose to send information to the synod first, only Linda Bobbitt and other ELCA research staff will have access to any data that would reveal the identity of specific participants and/or their congregations. This information will be aggregated for any reports. Permission will be asked of the participating congregation/individual before information that identifies the participant it is shared with the synod or wider church.

The ELCA will retain access to the data from this research for their own purposes outside of this project, providing they maintain the confidentiality of participants at all times.

TimelineAction Time

FrameEstimated Due

DateOne-time training events varies

Complete evaluations and score

Follow-up survey

At the end of the training3 months after each event

Start Cohort Finalize participating

congregations For Fall 2017

Before first session

Initial Cong. Vitality Survey “ “ Complete Lead Survey “ “ Confirm worship attendance and

giving“ “

Begin Cohort sessionsAnnual report July 1, 2017,

2018, 2019Mid-term report? Half-way

through project

January 1, 2018

Wrap up Cohort After all sessions

??

Final Cong. Vitality Survey Record final worship attendance

& giving Final individual reflections Final Congregational Reflections Continue cycle of cohorts

throughout projectFinal Evaluation 2 months

after September 1,

2019

4

completion

Template Synod:

I hereby agree to the evaluation plan outlined in this document.

________________________ ______________Synod Representative Date

_______________________Title

5

Attachment 1: Logic Model for Congregational Renewal Cohorts

Inputs/Resources Activities/Programs Outputs Short-term

DifferenceLong-term Difference

ELCA Renewal Team trains the trainers (regional renewal teams); the DEM/trainers train the congregations

Grant and synod provides for registration/hotel for synod team and synod team pays transportation to training

Congregations that host events provide hospitality and free will offering taken for meals

Supplies paid for by synod

Renewal activities

X congregations participate in each cohort New cohorts each year

Expected of participating teams:

Establish spiritual practices

Listening Work (internal/external) leads to learning

Experimenting leads to learning

Reflection leads to learning

Deeper cong. connections with God, each other and world

Theological imagination expanded – able to articulate what God is up to and how they join God.

Clarified sense of purpose within God’s mission – articulate mission and next steps

Increased engagement with neighbors

New leaders identified and engaged

Increased mission support

Increased worship attendance

6

Measuring Progress Cohort Rubric

Key Categories of Focus

Measurement Evidence that the Difference is occurring in

a MAJOR way

Evidence that the Difference is

occurring, but in a MINOR way

Evidence that the Difference

has yet to happen

Spiritual practicesReflection Leaders describe that spiritual practices have

become part of congregational life and are now informing their continued work.

Spiritual practices happen but it does not appear to be informing their work.

No regular spiritual practice yet

ListeningReflection Leaders describe listening activities they’ve

done that are now informing their continued work.

Listening activities have happened but are not yet being used to inform work.

No listening activities have taken place yet.

ExperimentingReflection Leaders describe experiments they’ve done

that are now informing their continued work.

Experiments have happened but are not yet used to inform work.

No experimentation have taken place yet.

ReflectingReflection Leaders describe reflections they’ve done

that are now informing their continued work.

Reflections have happened but are not yet used to inform work.

No reflections have taken place yet.

Clear Purpose within God’s mission & path forward

Reflection Congregation can articulate their mission/purpose and has clear next steps.

Vague mission/purpose and/or vague next steps.

Congregation cannot articulate a sense of mission/purpose.

Expanded Missional Imagination

Use reflection score sheet with 1-7 scale (afterward)

Leaders describe shifts in missional imagination. (score of 6-7)

Progress but transformation incomplete. (score 4-5)

No shift in imagination, little or no God language. (score 1-3)

Increased engagement in local context

Reflection Leaders describe concrete ways they now engage with their context in a subject to subject manner.

Some local connections but still subject/object, transactional or purely attractional.

No new activities.

Deeper cong. connections with God, each other, world

Congregational Vitality Survey at beginning of process and the end. Annually after(Measures people in pews)

Vitality scores all over 4 or improved from pre-test. Increases in specific practices or specific scores. (TBD)

Some improvements, but not significant from time 1 to time 2

Do change or decrease

New LeadershipReflection New lay leaders are named and specific roles

they’ve played are identifiedLay leaders are named but haven’t played leadership role yet

No new leaders identified

Worship attendanceAnnual report: Average worship attendance

Worship attendance increases more than 5% and congregations have enough people to participate in ministry.

Congregations are holding steady in attendance.

Worship attendance decreases more than 5%

Generous Giving Annual Report: % shared with synod, income/expenses of congregation

Congregations experience increased contributions from members. Congregations increase percentage given to synod.

Some improvements, but not significant from time 1 to time 2

Congregations’ expenses are higher than income.

7

Measuring Progress for Congregations: Score sheetOne sheet for each congregation after project completion

Place an x in the box that best matches the description in Attachment 1

Congregation name: __________________ City:________________ CongID:___________ Date: ______________


Measurement tool Evidence that the Difference is

occurring in a MAJOR way

Evidence that the Difference is occurring,

but in a MINOR way

Evidence that the Difference has yet to

happen

Spiritual practicesFinal Reflection

ListeningFinal Reflection

Experimenting Final Reflection

Reflecting Final Reflection

Clear Purpose within God’s mission & path forward

Final Reflection

Expanded Missional Imagination

Final Reflection

Increased engagement in local context

Final Reflection

Deeper cong. connections with God, each other, world

Congregational Vitality Survey- Linda will complete

New LeadershipFinal Reflection

Worship attendanceAnnual report: Average worship attendanceLinda will complete

Generous GivingAnnual Report: % shared with synod, income/expenses of congregationLinda will complete

8

Attachment 1b: Logic Model for One-time events



oTraineroHost congregation

Short term workshops

X people from y congregations participate

Did they learn what you taught them?

Do they have a plan to use the information.

2-3 month follow up – what have you done with training

9

Event Project Evaluation Rubric

After each training, ask members to answer two questions: what have you learned, and what are your next steps (or some variation of those questions). Collect the responses and ask someone in the synod to read the answers and mark each response using the rating sheet in Attachment 1. Total the marks in teach box. The total responses in each row should equal the total number of participants.



a MAJOR way




has yet to happen

Leaders articulate what they learned

Answer to 2 open ended questions (immediately after or at the end of training)

Leaders describe what they have learned in clear, specific terms. What they learned corresponds with what was taught.

Leaders have vague ideas of what they learned or describe things not taught in class.

Leaders are not able to describe what they learned.

Leaders have clear ideas of how to implement


Leaders articulate clear next steps Leaders articulate vague next steps Leaders don’t articulate next steps.

Event Project Follow-upTwo months after each training, ask participants to answer the following question: Please describe how you have used _____ training. (give specific examples of what you have done). Collect the responses and ask someone in the synod to read the answers and mark each response using the rating sheet in Attachment 1. Total the marks in teach box. The total responses in each row should equal the total number of people who responded to the follow-up.

Date of follow-up: ______________ Number responding: ___________



a MAJOR way




has yet to happen

Leaders implement next steps

Follow-up survey & reflection 3 months later?

Leaders can name specific steps they have taken

Leaders implemented started planning but have not yet implemented action plan

Leaders did not implement training.

10

Event Project Evaluation Score sheet (one for each event)

After each training, ask members to answer two questions: what have you learned, and what are your next steps (or some variation of those questions). Collect the responses and ask someone in the synod to read the answers and mark each response using the rating sheet in Attachment 1. Total the marks in teach box. The total responses in each row should equal the total number of participants.Date of event: ______________ Number participants:_______ Number responding: ___________



a MAJOR way




has yet to happen

Leaders articulate what they learned




Event Project Follow-upTwo months after each training, ask participants to answer the following question: Please describe how you have used _____ training. (give specific examples of what you have done). Collect the responses and ask someone in the synod to read the answers and mark each response using the rating sheet in Attachment 1. Total the marks in teach box. The total responses in each row should equal the total number of people who responded to the follow-up.

Date of follow-up: ______________ Number responding: ___________



a MAJOR way




has yet to happen


Follow-up survey & reflection 3 months later?

11

Attachment 1c: Logic Model multi-session long term individual training programs (not congregational teams)



Training resourcesHost congregation or

site

Long term leadership training

X people participate from y congregations

They learned what you taught

They have a plan to use what you taught them

They establish practices you taught

Theological imagination expanded – able to articulate what God is up to and how they join God.

12

Measuring Progress for long term training: Rubric



a MAJOR way




has yet to happen

Specific outcome depends on the class:

Leader Reflection (immediately after or at the end of training)

Leaders describe what they have learned in clear, specific terms.

Leaders describe how they have started practicing what they have learned in their congregation.

The things they describe match what you taught in the class.

Leaders meet describe what they learned in vague terms or do a good job describing what they learned, but have not started practicing it yet.

Leaders are not able to describe what they learned.

Leaders have expanded missional imaginations


Leaders describe shifts in missional imagination scale below. (score of 6-7)

Progress but transformation incomplete. (score 4-5)

No shift in imagination, little or no God language. (score 1-3)



Leaders articulate clear next steps Leaders articulate vague next steps Leaders don’t articulate next steps.


Follow-up survey & reflection Leaders can name specific steps they have taken

Leaders implemented some next steps but fell short of initial plan

Leaders did not implement training.

Describe the individual missional imagination overall changeBased on the individual reflection, indicate the one response that best represents the leader’s movement or lack of movement in missional imagination.

____ This leader is moving in the wrong direction in both attitude and direction. (1)____ This leader is stuck in the old/unhealthy model– no appreciable movement in attitudes or behaviors. No willingness or ability to work on these things at this time. It may be

the wrong time for this leader to work on leadership skills. (2)____ This leader is struggling– he or she wants to make progress but feels stuck, conflicted or have substantial barriers that keep them from progressing. Change is needed

(barriers overcome) before he/she will be able to move forward. (3)____ There has been a positive change in attitude and/or knowledge, but it is not yet reflected in behaviors/practices. He/she appears to be moving in the right direction without

enough barriers to derail the process. It is still early in the process. (4)____ There appears to be some positive movement in attitudes and behaviors/practices, but the transformation is incomplete so far. (5)____ This leader has transformed from an old model or unhealthy situation to a predominantly new/healthy model (6)____ This leader was always and continues to be in the new/healthy model (7)

13

Measuring Progress of long term training: Score sheet



a MAJOR way




has yet to happen

Specific outcome depends on the class:


Leaders have expanded missional imaginations





Follow-up survey & reflection .

Describe the individual missional imagination overall changeBased on the individual reflection, indicate the one response that best represents the leader’s movement or lack of movement in missional imagination.

____ This leader is moving in the wrong direction in both attitude and direction. (1)____ This leader is stuck in the old/unhealthy model– no appreciable movement in attitudes or behaviors. No willingness or ability to work on these things at this time. It may be

the wrong time for this leader to work on leadership skills. (2)____ This leader is struggling– he or she wants to make progress but feels stuck, conflicted or have substantial barriers that keep them from progressing. Change is needed

(barriers overcome) before he/she will be able to move forward. (3)____ There has been a positive change in attitude and/or knowledge, but it is not yet reflected in behaviors/practices. He/she appears to be moving in the right direction without

enough barriers to derail the process. It is still early in the process. (4)____ There appears to be some positive movement in attitudes and behaviors/practices, but the transformation is incomplete so far. (5)____ This leader has transformed from an old model or unhealthy situation to a predominantly new/healthy model (6)____ This leader was always and continues to be in the new/healthy model (7)

14

Attachment 2 (For congregational teams)

15

Attachment 3

Leader Individual Reflection

Please use this form to reflect on the ___(name of your synod project)____ you did with the synod some time ago and describe has happened since. Your responses will help synod staff and the larger church make changes in the future. Complete this form by answering information for each line or typing responses in the spaces below each question. Feel free to use as much space as you need – don’t worry about page breaks.

Congregation/Organization_______________________ CongID: ________________________________

City:___________________ Synod: _________________________________

Date: __________________ Role of person that completed this form: ________________________________

Questions about your participation in this process.

1. Describe your particular role in this process.

2. Describe how you participated in following practices and how those practices have informed your work so far: (If you have not engaged these practices please say so.)

a. Spiritual Practices (e.g. Dwelling in the Word, Biblical learning, discernment, prayer, etc.)

b. Intentional Listening (e.g. one to ones, appreciative inquiry interviews, focus groups, etc.)

c. Experimenting (e.g. trying new things)

d. Reflecting (e.g. thinking about what happened and what you learned)

Questions about the outcomes of this process.

3. Identify 2-3 discoveries you personally made about yourself during this process. Talk about why or in what way these were discoveries. (i.e. what information, understandings, experiences, practices, reflections did you gain in because of process?)

4. Describe how you personally informed or connected people in your own congregation/organization to this process.

5. Describe any new relationships have you personally established with the residents of your congregation’s neighborhood?

16

6. Write 3-5 short statements about yourself based on what you learned.

They will start with “I used to….” And end with “now I…..”(e.g. I used to (think, feel, have the habit of, have an attitude of, or do) __________; Now I (think, feel, have the habit of, have an attitude of, or do) __________.)

Moving forward:

7. What did you realize or get reminded of while writing this reflection?

8. In light of this process, what do you think do you think is the best next step for your own development as a leader?

9. In light of this process, what do you think do you think is the best next step for your organization/congregation?

10. Describe anything that might keep you from continuing this journey.

11. Describe what gifts or assets you bring that will help you move forward in God’s mission.

12. How can the synod continue to support you moving forward?

Thank you for reflecting on your experience. Please save and return to: [email protected] responses will be shared verbatim with your synod staff so that they can learn from you. Your responses will also be complied as part of a national research project. Reports from the national project may include descriptions of substance of your comments and some quotes without identifying specific individuals or groups.

17

mailto:[email protected]

Attachment 4

Congregational Reflection

Please use this form to reflect on the ___(name of your synod project)____ you did with the synod some time ago and describe has happened since. Your responses will help synod staff and the larger church make changes in the future. Complete this form by answering information for each line or typing responses in the spaces below each question. Feel free to use as much space as you need – don’t worry about page breaks.

Congregation: ____________________________ CongID: ________________________________

City:___________________ Synod: _________________________________

Date: __________________ Person/Group that completed this form: ________________________________

Questions about your congregation’s participation in this process.

1. Describe how you and/or your congregation participated in following practices and how those practices have informed your work so far: (If you have not engaged these practices please say so.)

a. Spiritual Practices (Dwelling in the Word, Biblical learning, discernment, prayer, etc.)

b. Intentional Listening (e.g. one to ones, appreciative inquiry interviews, focus groups, etc.)

c. Experimenting (e.g. trying new things)

d. Reflecting (e.g. thinking about what happened and what you learned)

2. If coaching was involved in this process, talk about how your team worked with your coach. Describe what (if any) impact the coaching experience had your work?

3. Please write the names of any new lay leadership that has emerged in this process. Describe what role each person has played so far.

Questions about the outcomes of this process.4. In your own words, what is your congregation’s current purpose? (Don’t just cite your mission statement, talk about

what it means. If you don’t have a clear sense of your congregation’s purpose, say “not sure”.)

18

19

5. Identify 2-3 discoveries your leadership team made during this process. Talk about why or in what way these were discoveries. (i.e. what information, understandings, experiences, practices, reflections did you gain in because of process?)

6. In light of this process, share examples of new ways this congregation now actively connects people with…

a. God

b. each other

c. the world (your local context and beyond).

7. Describe any new relationships your congregation established with the residents of your community.

8. Write 3-5 short statements about your congregation based on what you learned.

They will start with “We used to….” And end with “now we…..”(e.g. We used to (think, feel, have the habit of, have an attitude of, or do) __________; Now we (think, feel, have the habit of, have an attitude of, or do) __________.)

Moving forward:

9. What did you learn or get reminded of while doing this reflection?

10. What are your next steps?

11. Describe anything in your congregation that might keep from continuing this journey.

12. Describe what gifts or assets your congregation has that will help you move forward in God’s mission.

13. How can the synod continue to support you moving forward?

Thank you for reflecting on your experience. Please save and return to: [email protected] These responses will be shared verbatim with your synod staff so that they can learn from you. Your responses will also be complied as part of a national research project. Reports from the national project

20

mailto:[email protected]

may include descriptions of substance of your comments and some quotes without identifying specific individuals or groups.

21

Attachment 5: Completed for each congregation reflection by David and Linda, then compared.

Marking Congregational Change in Attitudes and PracticesCongregation: ________________ Cong ID__________ Synod: ____________ Date: ______________

Each synod seeks to engage the missional imagination of congregational leaders. We expect changes in attitudes to eventually result in new behaviors/practices. See the second page for examples of old/new or unhealthy/healthy attitudes and behaviors.

After reading congregational reflections, copy the text or summarize content in the appropriate column below. The result will be a list of examples of the attitudes and practices described in the reflection grouped by whether they represent an old or unhealthy model, new or healthy model or movement toward a new or healthier model.

Look specifically for “God language”. Note when/if God is used as the subject of action verbs (e.g. God does x). Look specifically for attitudes and practices or behaviors relating to mission, motivation, leadership development, relationships among congregation members, and relationships with local community.

AttitudesOld model or unhealthy attitudes

Movement New-model or healthy attitudes

Practices or BehaviorsOld model or unhealthy behaviors

Movement New-model or healthy practices

Describe the congregation’s overall changeConsider the notations above. Indicate the one response that best represents the movement or lack of movement in this congregation.

____ This congregation is moving in the wrong direction in both attitude and direction. (1)

____ This congregation is stuck in the old/unhealthy model– no appreciable movement in attitudes or behaviors. No willingness or ability to work on these things at this time. It may be the wrong time for this congregation to work toward renewal. (2)

____ This congregation is struggling with resistance – some wanting to stay in the old and others trying to move toward the new. Some positive steps have been made, however the struggle has prevented significant, steady growth. Change is needed (resistance overcome) before they will be able to move forward. (3)

____ There has been a positive change in attitude, but it is not yet reflected in behaviors. They appear to be moving in the right direction without enough resistance to derail the process. It is still early in the process. (4)

____ There appears to be some positive movement in attitudes and behaviors/practices, but the transformation is incomplete so far. (5)

____ This congregation has transformed from an old model or unhealthy situation to a predominantly new/healthy model (6)

____ This congregation was always and continues to be in the new/healthy model (7)

22

Examples of Old/New model attitudes & behaviorsOld model or unhealthy attitudes New model or Healthy attitudes

Resistance or no buy for process Enthusiastic about processSurvival- fear based Hopeful - optimisticScarcity Generosity & Abundance Deficit focused (what we don’t have) Asset based (what we do have)Inward focus (country club or social club mentality) Outward focus (Driven into the world by God)God not mentioned or distant/passive God used with action wordsAfraid of God or of failing God Trust God with success, free to failAfraid to act – lack of confidence Confident in God’s promises (ok to fail)Church sends people to do mission Church itself is sent to do mission Vague understanding of why they act as a church and as individual disciples

Intentionality of church and individuals

Ask people to join us and what we do Join people to support what they want to doGod is here and we share God with the community God is here and in community. We partner.Subject to object (us/them) Subject to subject (we are partners in community)Congregation as an island in community Congregation as collaborative partnerFixing problems to get us back to where we were. We are being transformed into something newAfraid of or avoid/resist change Change is part of who we are and what we do Newcomers must learn the right way to do things Newcomers may show us new ways to do things“Plastic people” – we don’t talk about difficult things as individuals and/or congregation.

Authenticity- we talk about what is actually happening in our lives and our church

Consumer mentality Sense of ownership among participantsCompeting with other congregations Collaborating with other congregationsPastor as Chaplin Pastor as coachPeople like us are welcome Diversity is valuedYoung people are the church of the future Young people are the church now. They are full members

whose ideas/skills are sought out, valued and used.Old model or unhealthy behaviors New-model or healthy behaviorsShutting down creativity and action Inspiring people to actionRidged structures limit activities Adaptive structures allow for experimentationReactive decision making and planning Proactive decision making and planningConflict/tension or conflict avoidance Open/respectful discussionTight control over what happens or anything goes permission giving without focus

Creative, purpose focused experimentation with reflection and adaptation

Dysfunctional relationships and behaviors Collaborative, functional relationshipsSecrets, poor or lack of communication Proactive, transparent, helpful communicationLack of intentional plan Clear intentional plan for moving forward even if end goal is

vagueCliques – especially in leadership Rotating leaders, using the gifts of all including young people

and newcomersDisconnect between faith and people’s daily life behaviors

Incarnational activities among people (talking about faith, living out vocational call)

Evangelism efforts non-existent or designed only to get new members

Evangelism = teaching people to share faith and invitation

Neighborhood outreach nonexistent or only designed to attract members

Neighborhood outreach that builds relationships and partners

Vague leadership roles – blurred expectations Clear leadership roles and expectationsPower/ authority used to control or assert dominance Power/authority used to lift up weaker members or community

people (servant leaders) Can’t articulate clear purpose Very clear on purpose – can explain

23

Documents

Project Blueprint: Templatecvs.holycrosschurch.org/Resources/Eval plan new2.docx · Web viewALWAYS BEING MADE NEW: THE CAMPAIGN FOR THE ELCA. SYNODICAL RENEWING CONGREGATIONS STRATEGIES