2020 SPIL Evaluation Findings - NYSILC · Web viewBrad Williams shared a Word document that outlined the project’s timelines for future logistical steps for 2020-21 (noted above)

NYSILC 2020 SPIL Evaluation Report

Consultant’s Report on the Fourth Year of the 2017-2020

NY State Plan for Independent Living (SPIL)

as amended

Submitted by Alan Krieger

Krieger Solutions, LLC

March 27, 2021

https://nysilc.org/

NYSILC SPIL Evaluation Report 2020

Table of Contents

Background....................................................................................................................................1

2020 SPIL Evaluation Findings........................................................................................................3

Summative Analysis – How well were the objectives and targets met?.............................................3

Overview:.................................................................................................................................3

List of Objectives for 2020:.................................................................................................................................5

Summary of the evaluation of each of the objectives and consultant observations.....................6

Objective #1: NYSILC operational effectiveness..........................................................................................6

Objective #2: New Yorkers with disabilities will be actively engaged in promoting disability rights - SSAN............................................................................................................................................................9

Objective # 3: Statewide independent living training conference......................................................13

Objective # 4: Database and interface.........................................................................................................13

Objective # 6: Improve capacity to address priority unserved and underserved populations...20

Overall Summary......................................................................................................................... 20

Recommendations.......................................................................................................................21

NYSILC 2020 SPIL Objectives/Evaluation Summaries...................................................................24

Objective # 1: NYSILC will demonstrate its operational effectiveness and capability to develop, monitor, and evaluate the SPIL......................................................................................................................24

Objective # 2: New Yorkers with disabilities will be actively engaged in promoting disability rights through the support of a Statewide Systems Advocacy Network (SSAN) and a statewide coordinator working with priorities identified in the NYSILC statewide needs assessment.......31

Objective # 3: Increase the attendance and involvement of attendees from the IL network, including affiliated stakeholders, by providing support for a statewide independent living training conference............................................................................................................................................38

Objective # 4: Designate funds to develop and establish a database and interface that will compile, analyze, and interpret data from the statewide network....................................................38

Objective #5: Address priority unserved and underserved populations and issues by providing one (1) $72,000 demonstration grant opportunity that can be evaluated by the council, be held to its own unique set of deliverables (outcomes), and subsequently provide a best practices manual for the benefit of the statewide IL network.............................................................41


Background

The NYS Independent Living Council’s Monitoring and Evaluation (M&E) Committee’s job is to conduct on-going monitoring of all projects supported by the Council and the Statewide Plan, and to conduct the evaluation at the end of the year. An evaluation consultant has been working with the Council to continue to refine the evaluation process and make it more outcome oriented. The consultant also completes an evaluation report each year.

The consultant’s job has been evolving. In the beginning, he took an active role in helping develop the evaluation process, working with a number of Council committees. Once the M&E Committee was formed, he helped train committee members in outcome evaluation and offered suggestions for moving objectives and targets to more of an outcome measure. As the M&E Committee has gained expertise and proficiency, he has pulled back to focus more directly on the final annual evaluation and report, with a focus on assessing outcomes or impacts of Council supported efforts, and less focus on program activity which is monitored by the M&E Committee.

The objectives and measures in the SPIL should relate as much as possible to the outcomes, objectives and measures in the contracts or action plans developed (and vice versa). The SPIL objectives should include the additional layer of the program activities that will produce the desired outcomes, but these activities should not be the focus of the evaluation. The draft of the next SPIL continues the shift to that direction (which became effective October 1st , 2020).

This evaluation covers the activities from October 1, 2019 through September 30, 2020 and is related to the objectives in the NY Statewide Plan for Independent Living (SPIL). This is the fourth year of the plan and one of the six objectives (#6 capacity building grants) was completed in a prior year and not active this year. In addition, objective # 3 related to increasing attendance and involvement at the statewide Independent Living training conference was not active this year as the conference was not held this year. The evaluation therefore is based on the other four objectives.

The committee actively monitored the various projects funded through the SPIL and explored how well they were advancing the SPIL objectives. This past year and the year before they also worked actively to make the objectives for the next SPIL more specific, measurable, and meaningful. They had many detailed discussions, and their proposed objectives are another great step forward in the development of the next 3-year SPIL.

Having a dedicated evaluation committee is a relatively new development for the Council, and it is proving to be an effective strategy. They specialize and focus on monitoring and evaluation overall, leaving the other committees to focus on the policy or operational issues they are addressing. Once the M&E committee completed their work, the next step in the development of the SPIL objectives was to have the relevant “program” committees review the objectives and revise/update as needed. The M&E Committee should then be the ones to do a final review prior to finalizing the SPIL, to ensure that the objectives are specific, measurable,

page 1


and outcome oriented. The current SPIL has been extended for a fourth year, allowing additional time to reshape the objectives. This review and revision process has taken a lot of time and should lay the groundwork for future SPILs as well. The Committee spent a lot of time helping shift the activity measures to outcome measures and worked with some of the currently funded projects on developing tools to help measure outcomes.

NYSILC staff, as always, did a great job pulling together the data and working with the committee to finalize the assessments. This report is based on the data provided by the Council, based on monitoring reports and other documentation. The evaluation consultant did not review any of the original data.

As noted in previous reports, the Council has come a long way with writing a more specific and measurable plan. This SPIL’s objectives and targets are more outcome or impact oriented; vague terms and standards have been defined more clearly to make the evaluation more objective. The monitoring protocol has enabled the Council to take a more active role in assessing success and gaining an early awareness if issues arise, so they can be addressed during the program year rather than after the fact. The work done on the new SPIL’s objectives and indicators will move the Council even further down the path of outcome-based evaluation.

This year’s evaluation consulting activities included:

Working with the staff and the M&E Committee to review the prior year’s results and assess accomplishments

Helping the M&E committee critique and clarify the proposed objectives and measurable indicators for the next SPIL. This included working with the committee to incorporate the new logic model and format that was developed for future SPILs.

The Monitoring and Evaluation Committee continued to follow their new process that relies on the NYSILC staff gathering relevant data from records for projects being conducted by NYSILC and from the Designated State Entity (DSE) for projects that were handled through Requests for Proposals (RFP’s) and or contracts. The committee then reviewed the resulting data and discussed any issues related to reaching the objectives.

NYSILC staff compiled the data in late 2019. It essentially represented a preliminary assessment of the SPIL objectives for 2019 from a narrative section of the Federal Program Performance Report (PPR) for New York. This narrative was transmitted to the SPIL evaluator who then drafted an evaluation report from the preliminary assessment. The full summaries are appended to this report.

page 2


2020 SPIL Evaluation Findings

Summative Analysis – How well were the objectives and targets met?

Overview:

Most of the objectives in the SPIL are stated broadly and the measurable indicators or targets are used to assess impact. The Council has moved to a more outcome-based evaluation and has increased the number of indicators that focus on impact and results. As a statewide group, it is often difficult to measure true impact and that is something the committee wrestled with in their work on the next SPIL. One solution has been to incorporate more surveys of people participating in the events and activities to try to assess the impact of this participation. This is not a technical, scientific assessment which would require pre/post testing, and this is not economically feasible. However, it is a great step forward to get a much better sense of what it means to participate in various educational activities. Some of the indicators still relate to whether or not certain activities took place or how many people participated in the activity, not what the impact was of the activity or how that furthers the Council’s mission, but the addition of more outcome measures is a meaningful shift.

The state plan being evaluated here does a much better job than previous plans of writing more realistic targets that are not too narrowly defined and has shifted more of the targets to an outcome focus. However, in some cases, performance targets can be fully met, i.e., the number of activities/people served were achieved, but there may be no clear impact or outcome of these activities. The reverse is also true, some of the targets that the committee might feel were “not fully met” due to data problems or technicalities were actually met when looking at the intention or desired outcome of the objective. Generally, in this year’s effort, the results line up with the intentions but where this is not the case, this has been noted.

The report is generally based on the technical rating – did the objective and indicators, as written, literally get met. In addition, the consultant added comments where a different rating might be more reflective of the impact.

Four of the six 2017-2020 SPIL objectives were active this year. As noted above, Objective #3 was inactive this year and Objective #6 was completed in 2017. The four active objectives had 35 specific measurable performance targets set for this year, grouped under 24 “outcomes.” The Council has adopted a scale or range for rating each target and objective. The scale includes: Exceeded, Fully Met (100%), Substantially Met (at least 60%), Partially Met, Not Met/No Progress. This reflects the understanding that even when an objective or indicator is not fully met, if there was substantial progress, that has important impact and is worth noting.

Overall, based strictly on numbers, the Council exceeded the measures for one objective, substantially met the measures on two objectives and did not meet the targets of the fourth objective (#4 - Designate funds to develop and establish a statewide database and interface). Overall, the SPIL was substantially met for the past year.

page 3


Of the 35 performance targets, 9 exceeded the target (26%) and 18 met the target (51%), so 27 out of 35 active targets were accomplished, a rate of 77%, which is slightly lower than last year, but still very good. There were four targets that were Not Met. The Council has worked to develop more realistic plans and targets that take into account the delays built into working with a large state operational process. The missed targets were primarily in the area of the development of the database, which has been slow to move forward, and progress made this year was related to prior year targets.

As noted in the introduction, these numbers do not tell the whole story. The reported results often show the extent of the activities that were conducted, but don’t always reflect the intention of the objective or the target in terms of impact or outcomes. And for one objective, the targets were so vaguely written it was difficult to do an objective assessment.

The following table lists the six objectives addressed in the state plan, indicates how well the four active objectives and 35 individual performance targets were met. The table uses the Council’s rating scale that ranges from “exceeded” the target to a low of “not met”.

“Exceeded” means that a target was exceeded “Met” means the target was met exactly “Substantially Met” means greater than 60% and less than 100% of the target

was achieved “Partially Met” means greater than 0 but less than 60% was achieved “Not Met” means nothing was achieved “No Progress” means the project was delayed or not started.

page 4


List of Objectives for 2020:

Exce

eded

Met

Subs

tanti

ally

M

et

Parti

ally

Met

Not

Met

No

Prog

ress

N/A

or O

ther

1: NYSILC will demonstrate its operational effectiveness and capability to develop, monitor and evaluate the SPIL.

Objective overall: X 11 total targets. 3 6 2

2: New Yorkers with disabilities will be actively engaged in promoting disability rights through SSAN and a statewide coordinator working with priorities identified in the NYSILC statewide needs assessment.

Objective overall: X 7 total targets. 6 1

3: Increase the attendance and involvement of attendees by providing support for a statewide independent living training conference.

Objective overall: XObjective is inactive for this reporting year.

4: Designate funds to develop and establish a database and interface that will compile, analyze, and interpret data from the statewide network.


5: Address priority unserved and underserved populations and issues by providing one (1) $72,000 demonstration grant opportunity that can be evaluated by the council, be held to its own unique set of deliverables (outcomes), and subsequently provide a best practices manual for the benefit of the statewide IL network.


6: Improve the capacity of the IL network to address priority unserved and underserved populations identified in the statewide needs assessments.

Objective overall: XThis objective was completed in 2017.

TOTALS:

Total Targets 9 18 4 4Total Objectives 1 2 1 2

page 5


Using the Council’s current system means that to “fully meet” an objective requires meeting every single target. This is a very high bar, as often something happens so one or two targets are not fully met. While the goal is always to meet and exceed where possible, if the objectives are written with some “reach” as many of these were, “substantially meeting” them is a strong outcome. Three of the four active objectives were substantially met or exceeded, which is a very good result.

The fourth one is behind schedule and so had progress mostly on last year’s targets, but little progress on this year’s targets. While it technically shows “no progress,” in reality, progress was made on the project, just not on the targets set for this year. Therefore, overall, the Council “Substantially Met” the objectives for the fourth year of the plan.

Summary of the evaluation of each of the objectives and consultant observations.

(For more detail see the SPIL Objectives/Evaluation Summaries on page 24, which follows the consultant’s report.)

Note that in the discussion below, there is a reference to “outcomes” and “targets”. “Outcomes” are usually statement that identify what the Council or Council-supported entities are trying to accomplish. They are not in most cases truly “outcomes” where an outcome notes a desired, measurable result that changes the current situation. Most of the outcomes are descriptions of actions to be taken, not the results of these actions. For Objective 1, many of these outcomes are a maintenance of the status quo – the continued successful operation of the Council – while others do promote progress on the Council’s mission or goals.

Objective #1: NYSILC will demonstrate its operational effectiveness and capability to develop, monitor, and evaluate the SPIL.

This objective had eleven targets. Three of these targets were Exceeded, six were Met and two were Partially Met. Overall good results, with 9 of the 11 targets being met or exceeded. The objective overall, therefore, was Substantially Met. The strong overall performance shows that the Council is operating effectively and meeting all of its administrative requirements.

Operational Status of the Council:As with prior years, four of these targets measured maintaining effective operational status.

It is helpful to assess and document each year that the Council is fully complying with key administrative responsibilities and receiving appropriate oversight (annual audit and annual evaluation). The following four operational targets were all fully met:

Holding four Council meetings with a quorum Completing the financial statement audit, 990, CHAR500

page 6


Completing the PPR report Completing the annual SPIL evaluation.

While none of these represent a change or connect directly to the SPIL’s or SILC’s overall mission, for the SPIL and SILC to both be successful, the Council needs to continue to operate effectively from an administrative standpoint. It may be clearer to make this its own objective to separate it from those that relate more to the mission than the operation.

Council Fund Development:Two targets relate to fund development, which is an important area for Council growth.

These are more outcome oriented. However, they are still administrative in nature and would fit in the administrative objective recommended above. These include:

1. Percentage increase of resource development efforts above contract (the target was a 4% increase, which unfortunately, due to the COVID-19 pandemic, was not met). Most of the resource development funds are generated by the Hall of Fame each year, and due to the delay of this event due to the pandemic, there was a substantial shortfall in meeting the $19,641 goal. NYSLIC developed $9,479 total; 52% below goal.

2. Comprehensive fund development plan completed by committee and consultant as documented (the target is 100%). A final Fund Development Plan was completed by The Association Development Group (ADG) and NYSLIC in the summer of 2020. In late August 2020, the Fund Development Plan was then presented to the NYSILC Development Committee: https://nysilc.org/images/FINAL_2020_NYSILC_Fund_Development_Plan.pdf. This target was fully met.

The other five targets have the potential to be more outcome oriented and could be revised to relate more directly to the Council’s mission. These targets partially relate to addressing critical issues and to increasing youth engagement. For the most part, they continue to measure activities in service to the mission; not outcomes that clearly move the Council forward in its mission. They also focus on a mix of administrative and mission related policy issues. They would be stronger if the administrative issues were connected to the operational targets and these focused exclusively on mission related policy. This has been discussed by the M & E committee when developing the next SPIL, and the intention is to move more to outcome-based measures.

Three of these targets relate to the work of Council committees. A substantive way to measure that work is to measure the issues the committees address and the results of their work on the issues. All three targets were again greatly exceeded, showing a very active year for committee work.

page 7


As mentioned above, these three targets might be more useful, if divided into administrative issues and mission related issues. Administrative issues support the administrative objective but don’t connect directly to the mission. Combining them into one tends to obscure the mission related work. Administrative issues addressed by the committees include:

Office elections, appointments, and member training Making changes to the statewide online survey instrumentProviding feedback on the 2019 SPIL Evaluation ReportApproving the 2020 spend down plan, as well as the 2019 audited financial statements.

Other committee work addressed issues that more directly impact people with disabilities ability to live independently.

Public Policy Committee supporting Black lives and addressing racism and the Executive committee approving it

Addressing the Native American Representative vacancy Promoting and supporting the Hall of Fame event Agreed upon a new method of scoring future Consumer Satisfaction surveys toward

more accurately and fairly measuring the 85% benchmark Completing an outcome/criteria review for the SPIL objectives based on the logic

model.

The three targets related to committee work include:

a) The number of issues where documented progress was made by NYSILC committees. The target was 12; and the total number of issues addressed where progress made was 29. The target was exceeded. It is a slight decrease from last year (34). Given this strong track record, the Council may want to consider increasing this target in the future or raising the bar for what is considered “progress” or removing administrative issues and focus on policy issues that directly impact people with disabilities.

b) The number of issues that were successfully addressed by committees. The Target was 4; and twenty-one actions were successfully addressed by committees. This target was also exceeded and is one more than last year’s result. Again, given the strong track record, the Council may wish to raise the bar on this as well. The examples cited in the data from the PPR are mostly administrative. This target would have more impact if it focused on policy issues that directly impact people with disabilities.

c) The number of unresolved Issues faced by committees. The target was 4 or less, and NYSILC reported only one unresolved issue by the end of the contract year. The continued vacancy of a Native American Representative. So, this target was met and “exceeded”.

page 8


The two remaining targets focused on youth involvement, a key Council initiative. One measured the number of youths participating in the Council’s youth leadership subcommittee, and the other measured the number of youths receiving scholarships for training. These are important in continuing to build the future leaders in the Independent Living network and movement. The number of youths on the subcommittee met the target set this year, continuing to remain stable with the same core group of individuals. The chair of the committee is currently vacant, therefore additional recruitment will be needed. The number of scholarships fell short of the target. This has been a challenging goal to meet over the years, and this year with the pandemic, many training events were cancelled making it more difficult. The Council continues to evaluate this and adjust the process to expand its reach to more youth leaders and to “right size” the target.

Both targets measure participating in activities and not outcomes. This has been discussed with the M & E committee and the scholarship program has become more focused on follow up and impact and this will be reflected in the targets in the next SPIL. Even though it is not required for the current SPIL, it would be helpful to include any follow up information in the evaluation. It is a successful occurrence when youth leaders are exposed to new ideas and opportunities, and it’s even more impactful when they return home and increase their community engagement as a result. Many of them do, and the scholarship now makes this a requirement, so it would be good to report on that. This will shift the target from measuring “activity” to measuring “outcome” or “impact.”

Similarly, the focus can easily shift to a more outcome orientation regarding youth members on the Council subcommittee. Currently only one of the five is an actual Council member. It would be helpful to note what other roles these youth members play in the work of the Council. Having more youth as full Council members and engaged with other Council committees would show a deeper engagement of youth leadership. It would also be helpful to show outcomes by reporting what results the committee has produced, and what role they play in Council deliberations and activities in addition to the review of the scholarship applications.

Objective #2: New Yorkers with disabilities will be actively engaged in promoting disability rights through the support of a Statewide Systems Advocacy Network (SSAN) and a statewide coordinator working with priorities identified in the NYSILC statewide needs assessment.

This is mostly an outcome objective in that it calls for a specific outcome: people actively engaged in advocacy efforts and that people will change as a result of this engagement. The objective had seven outcomes each with one target. The SSAN exceeded six outcomes and met the remaining one.

The New York State Education Department (ACCES-VR) contracts with the New York Association on Independent Living (NYAIL) to coordinate the Statewide Systems Advocacy Network (SSAN). Fifteen ILCs make up the SSAN. NYAIL provided ongoing communication and technical assistance to the SSAN sites.

page 9


Most of the targets for this objective are activity targets that show information was sent and educational activities were held, but not that anyone was “actively engaged”. To help move from an “activity” to an “outcome” based assessment, a survey was distributed to SSAN volunteers in 2018 to assess the impact systems advocacy has on factors such as levels of personal empowerment and self-determination. The SSAN quarterly reports document the amount of activity that was undertaken related to the specific targets. The survey helped measure the impact of the activities undertaken and is a large undertaking, so doing it once per SPIL cycle seems like a reasonable approach. In addition, where SSAN notes that they’ve built new partnerships, NYSILC now asks for descriptions of what these new partners do, showing that they are active and committed partners, not just partners in name.

NYAIL worked with SSAN advocates to develop the statewide agenda. Individuals participated in events prior to the lockdown. The Assembly also passed legislation important to the disability community, including bills to reinstate the Office for the Advocate for People with Disabilities; creating a visitability tax credit, and establishing a small business tax credit to encourage the hiring of people with disabilities. With a pandemic being declared mid-March, and the closures of the Capitol and the state lockdown, the typically busy quarter for legislature was disrupted.

During the budget process, NYAIL joined with a number of other statewide associations and organizations to publicize the impacts on Medicaid recipients, including people with disabilities, if $2.5 billion would be cut from Medicaid. Organized press conferences and rallies were held, a statement of principles was created and signed by over 130 organizations and the group was active throughout the budget process itself. With the pandemic occurring mid-March, the efforts were forced to be focused on social media using the hash tag #NoMedicaidCuts. The budget was ultimately passed with many of the cuts, but some programs were saved.

SSAN also focused efforts and outreach to identify underserved/underserved groups, such as minorities, LGBTQ, and immigrants with disabilities in the community. SSAN sites continue to try to connect with these groups to work together with them during the last quarter in a NYSILC sponsored initiative. A few of the efforts undertaken included:

AIM collaborated with a local Black Lives Matter group in Corning and Ray of Hope Church, an LGBTQ church in Elmira, on a get-out-the-vote TV ad campaign aired on WENY, the local ABC and CBS affiliate

WILC initiated and implemented a “Get Out & Vote” campaign geared towards persons with disabilities/Minority persons with disabilities

LIFE at RCIL created a video with a person of color and a person with a disability talking about the intersectionality between race and disability

ILCHV cosponsored a Public Event on Marginalized People Leading Movements for Justice, Because We All Have a Role and A Stake in Intersectional Justice Sponsored by ILCHV and NYSILC.

page 10


NYAIL worked with SSAN sites, having monthly calls in additional to the monthly public policy subcommittee calls. Much of the discussion moved to being focused on virtual activities once the second quarter happened. Regular zoom sessions started being offered by SSAN centers with advocates. New partnerships and ways to communicate and educate the public were made, along with new ways to provide public education and community activities during a pandemic.

Training was provided for the SSAN network during the year including the review of the Statewide agenda, a webinar for SSAN advocates and the statewide IL Network on Election Accessibility and Voter rights, a roundtable on Accessible Absentee Voting and GOTV: The Time is Now, a webinar for SSAN advocates and the statewide IL network as well.

Clearly there were important outcomes from the work of the SSAN although as noted earlier, not all of this can be directly attributed to SSAN, as their effort was part of a larger effort.

Below is a description of the seven targets and their outcomes. Six of the targets / outcomes were exceeded and one was met.

1. Number of SSAN significant statewide system changes (Target: 2 - Met)

This is the bottom line of all the advocacy efforts documented in the remaining targets. This is one of the key intended impacts of all the network’s other activities. While the SSAN can’t take full credit for these changes, the network actively supported them and provided educational opportunities that focused on two issues cited below as significant system changes that were realized during the past year:

• ASL interpreter provided during the state daily COVID-19 press briefings• Funding for Olmstead Housing Subsidy and Access to Home preserved.

2. Number of local partnerships and coalitions established by the SSAN network (Target: 75 - exceeded)

Based on information documented in SSAN quarterly reports, the SSAN sites established 138 new partnerships and coalitions, exceeding the goal by 63. Earlier evaluations have noted that simply joining a coalition is not a meaningful change. SSAN now reports on the activities these partners undertake showing that their membership in the coalition is substantive, not in name only. This and the following targets demonstrate the active results produced by this network which help support the important issues impact of the network.

page 11


3. Number of educational alerts disseminated to local volunteers by the SSAN network (Target: 660 - exceeded)

Based on information documented in SSAN quarterly reports, SSAN centers sent a total of 901 alerts to local volunteers. This exceeded the target by 241. This is a measure of activity, not impact, but combined with other measures, this was part of the effort to more actively engage new advocates. This is true for numbers 4-6 below, although all four of these had some effect on number 1 above – issue impact.

4. Number of local public education activities engaged in by the SSAN network (Target: 300 - exceeded)

Based on information documented in SSAN quarterly reports, the SSAN Centers exceeded this goal by engaging in 661 public education activities in the past year. This exceeded the target by 361. (Half-way through the year, these were all switched to virtual events which enabled the network to continue its education efforts.)

5. Number of grassroots organizing activities engaged in by the SSAN network (Target: 90 - exceeded)

Based on information documented in SSAN quarterly reports, SSAN sites reported a total of 212 grassroots organizing activities for the year. This exceeded the target by 122. (Half-way through the year, these were also switched to virtual events which enabled the network to continue its grassroots efforts.)

6. Number of oral or written public testimonies statements or letters provided, in response to a documented request, by the SSAN network (Target: 90 - exceeded)

Based on information documented in SSAN quarterly reports, SSAN sites collectively provided 134 public testimonies during the past year. This exceeded the target by 44.

7. Number of CILs or SCILs without a SSAN contract voluntarily involved in the participation of the network (Target: 2 - exceeded)

Based on information documented in SSAN quarterly reports, four non-SSAN centers continued to participate at various levels in the SSAN.

Overall, this objective and its seven outcomes / targets demonstrated significant progress during the past year. Given that the SSAN typically exceeds its targets, it might be reasonable to increase the targets or increase the measurable results of meeting these targets, as measured through a survey of a small sampling of follow up activities or other tangible changes/actions.

page 12


Objective # 3: Increase the attendance and involvement of attendees from the IL network, including affiliated stakeholders, by providing support for a statewide independent living training conference.

This objective was inactive this year since the conference did not take place.

Objective # 4: Designate funds to develop and establish a database and interface that will compile, analyze, and interpret data from the statewide network.

As noted in prior evaluation reports, the objective here is not really to “designate funds” but to develop and establish a statewide database for the IL network. The objective should focus on the outcome, not the process to get there. This is a multi-year project, that is a restart of a prior effort that was unsuccessful.

There were five outcomes active this year related to this objective, each with one target. One was met, another was partially met, while three were not met. Progress is being made, but the timing of implementation of the database/interface project is off by 1-2 years. This has been a perennial problem with this project, so while it looks like little was accomplished, in fact progress is being made and if the timeline for the targets could have been shifted, a more successful result would have been displayed. Since the SPIL can’t be easily amended, an informal summary showing the shift in timeframes would illustrate the true progress made each year.

The first outcome and target were related to the number of centers testing the prototype of the statewide database/interface product, with a target of 20 centers. During this contract year, six members of the Database Work Group were involved with the project: WNYIL Buffalo, NRCIL Watertown, STIC Binghamton, CDCI Albany, BILS Bronx and TRI Poughkeepsie. It is expected that moving into 2021, the involvement of these centers will increase. Target and outcome were partially met.

The second outcome was related to the status of web application design for the database/interface based on collective center data and feedback. Due to contract delays, progress on the project with web developer ES11 was delayed until January 2020. Work was focused on the development of the web application build. The remainder of the web application took 2-3 more months to complete. In April 2020, NYSILC and STIC had discussions with ES11 regarding the final draft export; the goal being to have a functional export to use as a model for the centers to replicate when outreach is initiated.

Based on needs expressed, an additional goal was to create one database site that the centers could visit to do all their reporting. In May 2020, a small group meeting was held to identify items from the ACCES-VR portal to be included, resulting in the identification of around 12-15 additional data points being incorporated into the database and interface design. In reality, these extra data points only increased the total by about half the number because for many the choices will be optional. This move will streamline the state reporting process.

page 13


Additional meetings occurred with ES11, resulting in a timeline of final steps being identified and the launch of the system in fall of 2022.

Overall, the basic design of the web application was completed, resulting in the outcome and target being met.

The remaining three outcomes and targets were not met due to the timing of the database and interface product being off by 1-2 years.

Objective #5: Address priority unserved and underserved populations and issues by providing one (1) $72,000 demonstration grant opportunity that can be evaluated by the council, be held to its own unique set of deliverables (outcomes), and subsequently provide a best practices manual for the benefit of the statewide IL network.

As stated in prior years, these targets are very vaguely written. There are no numerical targets, so they are very easy to meet. Most of these targets relate to activities that are required in the operation of any effective program. It reads more like an action plan than a set of objectives and targets. Very few relate to any outcome or result and the one that does is still very vague. The purpose of a demonstration project is to develop and test new strategies. The annual evaluation should help answer the following questions – is this effort producing the intended outcomes? Is it successful? Is it worth the investment? Should it be continued and/or replicated? Due to the lack of measurable outcome-related targets, it is not possible to answer these questions.

As indicated by the M&E’s Committee report, there was a lot of activity performed in support of this project, but there are no measurable indicators for what would be an acceptable or successful level of activity. Therefore, every measure was fully met since it was done at least twice and met the standard set by a plural target – e.g., issue press releases.

When creating the last SPIL, it wasn’t known what project would be selected for the demonstration project, and the decision was made to let the local program set its own goals with the DSE (ACCES-VR), objectives and targets. As shown by the result, the Council needs to provide guidance and oversight to ensure measurable, outcome-based targets are set and that they are at a level that would indicate reasonable success.

While the contract was negotiated with these vague targets in it, and it would be difficult to renegotiate the contract, it would have been reasonable for the Council’s M & E committee to meet with the program staff and help them better articulate outcome measures that they hoped to achieve and to put numeric targets into the original broad targets.

The true value of this demonstration project is in the services provided to the unserved/underserved population and in the community connections made. There is probably some meaningful data in survey results and results of people directly served by the program,

page 14


but no survey results are shared in the report and program results are broadly and vaguely stated, making the evaluation of the objective difficult. The M&E Committee’s “lesson learned” is if they have an RFP that allows proposers to design the program and set their own outcomes, they should be provided with guidelines about how to build a rigorous logic model (mission, goals, objectives, and measurable indicators based on impacts) that leads to clear outcome statements. And as part of the review and approval process, the Council should ask to be involved in reviewing the proposed logic model and making recommendations for changes before the contract is finalized. Otherwise, the SPIL should provide some specific impact or outcome targets that are required at a minimum for each project in the future.

The committee identified six outcome areas related to this objective with fourteen total targets. Four outcomes were met, one was partially met, and one was not met. Two targets were not really relevant in the final year of the project and they have not been counted. Of the remaining 12, ten were met, one was partially met, and one was not met. Given the overall results, this objective was substantially met.

Taconic Resources for Independence (TRI) completed the third year of the demonstration project to provide Independent Living services to veterans with disabilities. The project has an additional focus on accessible and affordable housing and transportation in the community. TRI set up their own unique deliverables with the DSE and reports on them quarterly. The content from the quarterly reports serves as the basis for this report.

The Council indicated that the value of this demonstration project will be determined more clearly at the end in its “how to” manual (what worked/what didn’t), which was distributed across the state and posted on the NYSILC website.

Here are the targets for this year as outlined by the agency after being awarded:

A. Outreach:

The three outreach targets as written (see below) are not meaningfully measurable. For example, would one press release, email blast and posting to social media be sufficient? Would one a month be sufficient? How many contacts equal a meaningful collaboration, and how many events need to be attended to be effective?

In addition, the activity of issuing outreach information or meeting with other stakeholders is just that, an activity. The measurable outcome would be how many referrals resulted from these efforts and that should really the be target that is measured.

It could be said that all three targets were met and possibly exceeded, but as noted below, these were counted as 2 met and 1 partially met.

page 15


1. Agency will issue press releases and email blasts which will include printable materials. Will include information on all social media platforms.

TRI’s initial outreach was completed, and they continued to do outreach by posting their various social media platforms as events and information arise. TRI posted community veteran events and functions on Facebook, Twitter, Instagram and Alignable. They also have printed flyers, brochures, and business cards. As a result, this target was technically met.

2. Veteran Specialist will collaborate with the Division of Veteran Services, Mental Health America Living Room Program (homeless program) and Hudson River Housing Veteran Services Program to promote services.

This contract year was the end of the contract and this was completed. TRI’s collaborators have changed. They continued to work on a regular basis with the Division of Veteran Services to help veterans with their compensation claims. The MHA program, Vet2Vet, has expanded again and has a new member who runs VET-TAP, which is designed to help veterans looking for employment. TRI continued to partner with WestCop as well, who assists us with finding housing for veterans. Technically this target was not met since TRI was no longer collaborating with the specified collaborators. This is another example of an overly specific target. A better target might have said, will collaborate with at least three key veterans and housing programs in the region. With that in mind as the intent of the target, this target was met.

3. Veteran Specialist will attend veteran specific events to conduct outreach and to remain current regarding the needs of Dutchess County Veterans. Again, this targets as written is not meaningful. It calls for attending a minimum of two events since “events” is plural. Specific events and outreach were limited due to the pandemic this contract year. Outreach was conducted remotely. As a result, this target was partially met.

B. Program Planning:

Similar comments as above – these targets are not meaningful, are very easy to meet, and don’t measure the quality of the program’s services or impacts.

1. Veteran Specialist will create a dedicated database to document all activities and outcomes and provision of direct services and/or systems advocacy and outcomes.

TRI’s database was previously set up and the veteran drive completed. Input is ongoing and updated regularly. This target was met the last two years and as stated, “will create” means that once it’s created, this target is not relevant and should not be listed again as a target in this year. We have deleted it from the total.

page 16


2. Program Director and Veteran Specialist will join veteran related committees and coalitions.

TRI’s Veteran Specialist is a continuing member of the Dutchess County Veteran Task Force that meets monthly with like-minded agencies to discuss all events and happenings related to veterans in the county. The Veteran Specialist is also a member of the Hudson Valley Task Force that serves five counties, and the Program Director attends meetings as a guest. The Veteran Specialist also attends Bringing Agencies Together (BAT) meetings and meets other non-for-profit agencies in the Dutchess County area. This target is met.

3. The Veteran Specialist will develop program specific intake assessment forms and outcome surveys.

Completed. A more interesting report would be on the results of the participant survey. Simply developing a survey has no value in and of itself. The assessment intake and goals assessment were developed last year. A veteran specific outcome survey was developed and distributed to both past and present participants via Survey Monkey. While this was technically met, again it is of little value as a target unless they summarize the results and issue a report.

4. Veteran Specialist and Program Director will meet weekly to assess program activities and plan best practices manual.

A final draft of the manual was completed and accepted by the Board of Directors and NYSILC. There is a separate target for completion of the manual. This was an initial planning target and therefore is not relevant in the final year of the project, so we won’t count this as a target this year.

C. Project Implementation:

1. The Veteran Specialist will provide comprehensive, quality independent living services to veterans with disabilities to assist them in becoming more stabilized in their homes and communities. The specialist will conduct in depth intake interviews, assessments of needs and define expectations and goals.

This is the end of this grant this contract year. TRI provided services to 1 unduplicated person this year. The total number served was 2. Also, 3 stakeholders (individuals, families, agency personnel) were provided with one-time information and referral that allowed them to self- advocate/resolve issues without the need for a full intake. Based on what has been learned through the past three years from developing the procedure manual, new implementations will continue to be made. This target was technically met since no numeric targets were set.

page 17


2. The Veteran Specialist will communicate with agency partners to provide appropriate linkage to supports and services for veterans with disabilities.

Again, not clear what the expected impact would be of these communications. Seems somewhat redundant with earlier targets of collaborating and attending events.

Agency reported that Vet2Vet is available in multiple counties and the Veteran Specialist works closely with the Dutchess County branch. Their partners include the Dutchess County Division of Veterans Services, Veteran Affairs at various locations, The Social Security Administration, WestCop, and the Legal Services of the Hudson Valley. They are also part of and share outreach and support related information to B.A.T. members, which is comprised of many nonprofits in Dutchess County. TRI will continue to maintain communication with partner agencies as appropriate. As a result, this target was met.

D. Service Provision:

The same comments apply to these two targets.

1. The Veteran Specialist will link veterans to/with supports, services, and benefits as needed.

Completed as this is the end of the grant. TRI linked veterans with disabilities to many services at agencies throughout the county including the Dutchess County Division of Veterans Services, the local VA, the Department of Labor, Social Services, West Cop, Legal Services of Hudson Valley, and the MHA Vet2Vet programs. Target met.

2. The Veteran Specialist will provide independent living skills training based on individual needs and goals.

Most Independent Living skills training reported by TRI involved assisting veterans with disabilities with filling out forms, requesting paperwork and filing for benefits from Veteran Affairs. TRI found many veterans used their in-house Social Security disability benefits counseling services. They assist veterans with applying for compensation and pension with the Veteran Service Office. TRI also helps them apply for VA Healthcare. TRI will continue to maintain this communication with partner agencies as appropriate now that the grant is over.

The NYSILC M & E Committee inquired about this target during one of its quarterly meetings. It noticed that most of the services provided were for information and referral, benefits advisement, and housing assistance. The NYSILC office followed up with TRI and they confirmed that the original intent was to offer IL skills, but the reality was that veterans with disabilities wanted the requested services. Many were homeless or on the verge of homelessness and needed shelter services. As discussed, it would have been better to have more information and an understanding of the services provided, why, and follow up of what happened (impacts) than all these targets. Target met.

page 18


E. Systems Advocacy:

1. The Veteran Specialist will assist veterans with disabilities in organizing a committee for conducting systems advocacy especially as it relates to affordable/acceptable housing. The Veteran Specialist will offer the opportunity to join the systems advocacy group to each veteran during the intake process.

This was offered to consumers on goals form during intake and noted on the assessment sheet. TRI is having difficulty forming the systems advocacy group as veterans are looking for immediate housing options, but they continue to strive to accomplish this goal. They have found that homeless veterans are in crisis and are looking for emergency services and not currently interested in a systems advocacy group.

This target was not met.

F. Best Practices Manual:

1. The Veteran Specialist will research, collect, and store data and document all activities and outcomes to be used to inform a best practices manual. The Veteran Specialist will use data collected from the intake interview and exit surveys to document outcomes or attainment of goals.

Completed. TRI utilized consumer statistics and exit surveys that had been distributed to current and past consumers to supplement work. “How To” best practices manual submitted and approved ahead of time. Archived on NYSILC website: https://nysilc.org/images/TRI_Veterans_Program_Manual.pdf.

This target was met. The M & E Committee greatly valued TRI’s “How To” manual. At the March 2020 committee meeting, it was agreed that Brad should reach out to TRI and request that they update their manual to reflect new lessons learned as a result of the Coronavirus before the end of their project. This was completed just before the project coordinator left the position, with the assistance of NYSILC.

This was the primary objective of this project, to produce a “how to” manual that other centers could use. This is the closest to an outcome measure. Getting input from other centers about how useful and informative the manual was could have been useful in evaluating the impact or outcome of this target.

2. A daily log will be kept tracking activity that will also be used to inform the best practices manual.

This goal was completed, and the target was met.

G. Other:

Highlights:

• TRI continues to talk with the new Dutchess County Veterans Service, Adam Roche regarding establishing a Veterans Court with the District Attorney.

page 19

https://nysilc.org/images/TRI_Veterans_Program_Manual.pdf


• TRI has been working with WestCop/Supportive Services to assist veteran consumes in finding housing and obtain resources.

Challenges:

• The ongoing Coronavirus Pandemic has made TRI’s role more difficult, especially during the second to last quarter, the veteran specialist found another position with Westcop/Supportive Services for Veterans.

• It was a challenge to find a replacement for the veteran specialist position for the last quarter of the grant, going through the interview process and hire a person for three months.

Objective # 6: Improve the capacity of the IL network to address priority unserved and underserved populations identified in the statewide needs assessment by providing ten (10) $25,000 capacity building self-sustaining grant opportunities that can be evaluated by the council, disseminated, and documented for replication for the benefit of the network.

This objective was completed in 2017.

Overall Summary

Overall, the SPIL had good overall results this year, one objective was exceeded, two were substantially met, and a fourth was not met. The objective that was not met was due to the project being delayed and the targets being out of synch with the actual implementation timeline, not because no results were produced.

The M&E Committee continued to add strength to the evaluation process as they monitor results on a quarterly basis and continue to give the evaluation process more attention than had been provided in the past when it was delegated to multiple committees. Terms are better defined, and targets are becoming more realistic and meaningful.

While there is room for improvement, much of the needed changes have been built into the next SPIL, which will take this to an even higher level. One area for continued development which the committee has addressed is when setting targets, it would be helpful to think carefully about the intention of the objective in terms of desired changes, and what are some meaningful and measurable indicators that could show that progress is being made on the objective’s intention. The Council has come a long way from the start in developing outcome-based objectives and indicators. Continued attention to this will create improved plans and evaluations that measure meaningful results and impacts.

Another enhancement the Committee might consider is to revise the targets as

circumstances change. While it may be too difficult to actually amend the SPIL, the committee could indicate that while the target as stated in the SPIL was or was not met, the committee recognized that circumstances had changed and that the project had shifted its focus or

page 20


timeline and as a result, either real progress had been made (as in the case of the data base project) or targets were clarified and perhaps some were not fully met with the new clarification although they technically met the terms of the SPIL (as might have been done for the veteran’s program where more rigorous targets might have been set).

The SPIL is a three-year plan, developed a year before its actual implementation. It is very difficult to foresee all eventualities that far in advance, so it is not unreasonable to make some adjustments in later years to reflect reality on the ground which could provide for a more meaningful evaluation. In the sense that the SPIL is a contract, a technical evaluation can be completed to measure exactly what was in the SPIL, and a more substantive and meaningful evaluation can also be done where targets are adjusted, and projects are evaluated based on a more meaningful set of measures.

Recommendations

The above analysis of the results of each objective focuses on “summative” evaluation, or evaluation strictly of numerical results. “Formative” evaluation looks at using the data to inform and improve the operation of the projects. As noted in prior year evaluation reports, the Council has continued to improve its process, continuing to move from objectives that were more activity based (measuring numbers served, numbers of events) to those that are more impact or outcome based (how the situation overall improved or how people involved changed/learned).

Also, as noted in prior years, some targets in the old plan were overly specific and detailed and were changed in the new plan to focus more on outcomes rather than specific outputs or activities – they indicate how things might change as a result of the plan’s efforts, not just what was done or what was produced. There is more to be done and the Committee has addressed this as they developed the next SPIL.

Specific Recommendations (many of which have been partially addressed in the new SPIL)

1. Continue to move from activity-based measures to outcome measures. It is very challenging for a statewide organization that tries to promote real change at both the state and local level to measure this change. The Council has made great strides in this area by adding in more surveys and other follow up data to try to measure the impact of the Council’s / SPIL’s initiatives. The SSAN and NYAIL conference are good examples of progress made. The veteran’s program also conducted surveys, but this data wasn’t included in the program’s assessment.

2. Focus objectives on outcomes – a change in status. From initially having no objectives, to in an earlier SPIL having 14 objectives, each with multiple performance targets that were mostly activity oriented, to now having a smaller number of objectives that are more outcome focused, is a large step forward. Prioritizing the objectives to a smaller number reflects a better sense of the Council’s role in the SPIL. Continuing to add more

page 21


elements that focus on outcomes will help promote the move to a more outcome-based evaluation. The new SPIL includes language about people being given opportunities and wanting to be involved and gain knowledge and skills. A better outcome measure would be that they do become involved, and they do gain knowledge and skills. The targets and measures in the new SPIL do address some of this. It would be good for the objective to state that as well. Sample outcome objective language could read: By XX (date), XX (target group), will demonstrate XX change. (generally, an increase or decrease from a baseline). While setting a base line may be impossible, showing at least a specific number of people who feel they met the outcome/results would be a more powerful outcome statement. The council continues to demonstrate growth and progress and can focus on new areas of improvement.

3. Make adjustments when needed. Developing a SPIL is a very large and challenging undertaking. It involved forecasting the future and anticipating how contracts will be written that the Council doesn’t directly control. There can be significant shifts from the best guess that the SPIL represents to what really happens. While the SPIL and subsequent contracts can’t be easily amended, the M&E Committee can note where things need to be adjusted and include this in their monitoring and evaluation assessments, so this final report can more accurately represent what happened. Some examples from above are the database project which continues to make progress but is behind schedule so much of the progress reflects earlier year’s targets and the TRI contract which had excessively vague targets in it but may have actually accomplished meaningful results.

4. Take enough time to craft the SPIL evaluation plan. This time around an additional year extension was given so the M&E Committee had enough time to interface with other committees and develop a sound outcome-oriented evaluation plan. In future cycles, this extra year won’t be there, so a more intentional process has to be in place to ensure there is time for this. The SPIL process rightly focuses primarily on how resources should be allocated. The time for writing the actual objectives and performance targets is at the end when there is often little time for reflection and revision.

5. Set realistic targets based on past experiences. In some cases, targets have been consistently exceeded and have not been adjusted. Similarly, some frequently fall short and should be adjusted to a more realistic level.

6. Set more stringent outcome targets. Adding quality criteria to the target that reflect outcomes (e.g., education alerts that are responded to), will increase the cost of measurement, and will bring down the number of “successful” outcomes, so the Council may want to consider reducing numbers while increasing the qualitative aspect of what constitutes success. This will show outcomes that have a measurable impact, not just an activity.

page 22


7. Set targets that help move the Council and the State forward in furtherance of the Council’s mission and broad goals of the plan.

8. Continue to ask “why” as part of evaluation. In the past, the Council’s evaluation process focuses more on assessing “yes” or “no,” was the objective met or not – the summative evaluation. It can be helpful to also look at “why” an objective was met, exceeded, or fell short – the formative evaluation. The M&E committee has begun reviewing this as part of their deliberations, which is another benefit of having a dedicated M & E committee.

9. Evaluate the evaluation process. In the beginning of this process, a number of years ago, the Executive Committee was tasked with evaluating the evaluation process overall and the work of the evaluation consultant. That has not been done for a while. This is now the responsibility of the Monitoring and Evaluation Committee, who reviews and recommends the final draft of the SPIL Evaluation report to the full Council. This is something the Committee should consider in terms of evaluating the evaluation process.

10. NYSILC Objectives should be divided in two. One focusing on “maintaining operational effectiveness” since there have not been corrective actions needed for many years. The other focusing on “achieving the mission” which looks at policy and program initiatives that further the ability of New Yorkers with disabilities to live independently.

11. Deepen the targets for youth involvement to show full engagement and impact of and on the youth engaged.

The full narrative from Section 26 of the NY 2020 ILS Program Performance Report (PPR) follows including detailed evaluations of each objective and performance target.

page 23


NYSILC 2020 SPIL Objectives/Evaluation Summaries

SubPart VI – SPIL Comparison and updates, Other Accomplishments and Challenges of the Reporting Year Section 704(m)(4) of the Act; 34 CFR 76.140

Objective # 1: NYSILC will demonstrate its operational effectiveness and capability to develop, monitor, and evaluate the SPIL.

a) Measurable Indicators/Performance Targets:

i. Number of full council meetings held during the year with a quorum (target 4): 4. (Outcome met)

Preliminary Assessment: NYSILC successfully held all four full council meetings with a quorum during the year. One was held in person at the Albany Marriott on November 15, 2019, and the rest were held via Zoom on March 27, 2020, May 29, 2020, and September 25, 2020 due to the pandemic. Outcome and target met.

Confirmation/Evaluation Rating:Exceeded target/ Met target /Substantially met target (more than 60%), Partially met target (less than 59%), Target not met.N/A or Other

page 24


ii. Progress made by NYSILC Committees (Outcome exceeded)

a) Number of issues where documented progress was addressed by NYSILC committees (target 12): 29.

Preliminary Assessment: Based on the information from the quarterly reports, progress was made on twenty-nine issues by NYSILC committees during the past year. Target exceeded. A few examples included:

The Executive Committee discussed officer elections, member mentoring, diversity training for the council and identified a targeted outreach strategy for filling the Native American Representative vacancy.

The Public Policy Committee made recommended the final solidarity statement supporting Black lives, addressing racism along with the Rochester Black Agenda and also drafted the 2020 NYSILC Post-Election Survey.

The Consumer Satisfaction Survey Subcommittee made changes to the statewide online survey system for use next year.

The Monitoring & Evaluation Committee provided feedback on the 2019 SPIL Evaluation Report.

The Development Committee promoted the nomination process for the 2020 NYS Disability Rights Hall of Fame (HOF) and Honorary Committee

memberships to support the event, which was postponed to 2021 due to COVID-19. Target exceeded.

Confirmation/Evaluation Rating:Exceeded target/Met target/Substantially met target (more than 60%), Partially met target (less than 59%), Target not met.N/A or Other

b) Issues successfully addressed by committees (target 4). 21 actions were successfully addressed. Target exceeded. A few examples included:

Approval of the council’s solidarity statement in support of Black lives and against racism, along with signing onto the Rochester Black Agenda by the Executive Committee.

Approval of the 2020 spend down plan by the Executive Committee. Review and support of the council’s 2021 appointments and

reappointments by the Executive Committee. Executive Committee approved of the Recruitment Committee’s

online recommendation for an interim appointment that was made full by the Regents in June 2020.

Agreed upon a new scoring method for the center surveys toward the

page 25


85% benchmark by the Consumer Satisfaction Survey Subcommittee. Recommendation of the Hall of Fame event until May 6, 2021 by the

Development Committee. Approval of NYSILC’s 2019 Audited Financial Statement by the

Finance Committee. Completion of an outcome/criteria review for the SPIL objectives

based on the logic model by the Monitoring & Evaluation Committee.


c) Unresolved Issues: faced by committees : (target 4) Based on information documented in NYSILC quarterly reports, there was one (1) unresolved issue by the end of the contract year. NYSILC has been attempting to fill its Native American Representative vacancy for almost two years. In June, the council sent letters to one nation and three tribes. After follow-up and no interest in the council, the Executive Committee identified a proactive, targeted outreach strategy to address the issue. Potential candidates were identified, leading to one individual. The Native American Representative vacancy is still an unresolved issue for this reporting year. However, a candidate has been vetted and included in the 2021 appointment packet to fill the Native American Representative vacancy starting in January 2021. Since there was only one unresolved issue, this is a result that exceeds the target of 4 (since it is less). Target exceeded.


iii. Completion and documentation of reporting requirements (Outcome met)

a) Annual financial audit completed “unqualified”, and 990 forms filed fully, accurately as documented: Target: 100%.

Preliminary Assessment: Based on the information from the January-March 2020 NYSILC quarterly report, Brad Williams, Executive Director, responded to Michelle Rosen of Cusick & Company with a set of emails regarding the list of fiscal audit deliverables. Follow up items were reviewed online. The site visit was the morning of January 8,2020. CPA Ken Claflin generated the draft versions of the 2019 Audited Financial Statement (AFS) and 990 form. The Executive Director, bookkeeper and Treasurer reviewed them. There was only one question about the posting of the fundraising event which the CPA

page 26


clarified. The NYSILC Finance Committee met on February 6, 2020 with CPA Ken Claflin providing an overview of the AFS for the group. The committee approved the AFS. The CPA firm initiated the process to generate final copies of the AFS and both filings (990 and CHAR500). Brad Williams facilitated the signing of the two filings identified above. The electronic copy of the 990 was posted https://nysilc.org/resources/key-documents/other/330-2018-990-and-char500/file, while the AFS was emailed out to the full council for review in advance of the March 2020 FCM. Target met.


b) Annual PPR Report completed with DSE to ACL fully, accurately, and on time. Target: 100%.

Preliminary Assessment: NYSILC was responsible for the coordination of the annual 2019 Program Performance Report (PPR) to the Administration for Community Living (ACL). The SILC worked in collaboration with ACCES-VR and Taconic Resources for Independence (TRI) in Poughkeepsie toward the completion of various sections of the PPR. A draft version of the 2019 PPR was sent to the DSE and SILC chair for review and signature. The SILC office input the data and information into the ACL Reporting website and successfully submitted the report by 12/31/19. The report needed to be reviewed by ACL. The council monitored the report’s status.

Staff attended SILC Congress at the end of February 2020 and participated in a session led by ACL where everyone was informed that only 7% of the ILS PPRs were approved. Upon returning, the SILC went online and found that Section 1 (financial) along with the SILC composition section were disapproved. NYSILC responded to an email from ACL related to the two sections and made updates to the PPR as directed. The 2019 NY PPR was approved by ACL. Link to the updated 2019 NY ILS PPR: https://nysilc.org/resources/key-documents/reports/704-reports/358-updated-2019-ny-ppr/file. Target met.


page 27

https://nysilc.org/resources/key-documents/reports/704-reports/358-updated-2019-ny-ppr/file

https://nysilc.org/resources/key-documents/reports/704-reports/358-updated-2019-ny-ppr/file

https://nysilc.org/resources/key-documents/other/330-2018-990-and-char500/file

https://nysilc.org/resources/key-documents/other/330-2018-990-and-char500/file


iv. Annual SPIL evaluation and report completed by evaluator and committees as documented: 100%. (Outcome met)

Preliminary Assessment: As reported earlier in Section D – SILC Duties (B), based on information documented in NYSILC quarterly reports, Executive Director Brad Williams and consultant Alan Krieger of Krieger Solutions, LLC started a process to compile information for the NYSILC 2018 SPIL Evaluation. In January 2020, Brad sent the narrative from Section 26 of the 2018 PPR to Alan. He utilized information from the report for the summative part of the report. Alan then completed a first draft of the 2019 NYSILC SPIL Evaluation report in April 2020. The content was reviewed with Executive Director Brad Williams. Edits and comments were compiled into a second draft. At a 5/14/20 Monitoring and Evaluation Committee meeting, members gave feedback on the second draft 2019 SPIL Evaluation Report. The group’s feedback was included into a final draft of the 2019 NYSILC SPIL Evaluation report, which was presented and accepted at the SILC May full council meeting. The report was posted on the NYSILC website: https://nysilc.org/images/NYSILC_2019_SPIL_Evalution_Final_Report.docx. Target met.


v. Completion of CIL statewide consumer satisfaction survey and report by committee, network, and consultant as documented. (Target 100%). Inactive this year.

Confirmation/Evaluation Rating:Exceeded target/Met target/Substantially met target (more than 60%), Partially met target (less than 59%), Target not met. N/A or Other

vi. Completion of statewide needs assessment and report/distributed and posted to NYSILC website. (Target: 100%). Inactive this year.


vii. Fund Development (Outcome partially met)

a) Comprehensive fund development plan completed by committee and consultant

page 28

https://nysilc.org/images/NYSILC_2019_SPIL_Evalution_Final_Report.docx


as documented. (Target: 100%).

Preliminary Assessment: This outcome was reported substantially completed last year. It has been combined with this target into a fund development outcome for efficiency and to report on its final progress.

The Association Development Group (ADG) presented initial concepts about the Fund Development Plan (FDP) at the November 2019 full council meeting based on interviews and background information (and past knowledge of the council). NYSILC members provided additional feedback to the vendor about the plan. ADG created a second draft of the FDP in June 2020, which was reviewed by NYSILC staff. The document was revised after online and verbal discussions. A final FDP with an action plan was completed by the summer of 2020. The FDP was presented to the NYSILC Development Committee in late August 2020, focusing on the timeline of the action plan over the coming years. Refer to the following link for the NYSILC 2020 FDP: https://nysilc.org/images/FINAL_2020_NYSILC_Fund_Development_Plan.pdf. Target met.


b) Percentage increase of resource development efforts above contract (all other sources). (Target: 4% or $19,641).

Preliminary Assessment: Based on information documented in NYSILC quarterly reports, the council has the capacity to develop resources per authority (B) under the WIOA regulations, as stated in the SPIL. The SILC can use operating funds to ideally support its primary operations and identified programs, such as the Pat Figueroa young adult scholarships, the NYS Disability Rights Hall of Fame, and other opportunities as defined and supported.

During the past year, NYSILC developed (all sources) $9,479. This was 52% below the goal of $19,641. It fell way below last year’s amount of $30,320 which exceeded the target. The result was due to COVID-19 and NYSILC’s decision to postpone the 2020 HOF event to May of 2021. Most of the resource development funds are generated by the HOF event. On a more positive note, the council was able to raise approximately 50% of its HOF goal before delaying the event. NYSILC was on track to meet this target under normal conditions.

page 29

https://nysilc.org/images/FINAL_2020_NYSILC_Fund_Development_Plan.pdf


The following other resources were developed during the year toward the $9,479: HOF Honorary Committee $3,920, HOF Sponsorships $$2,350, HOF Advertisements $750, Donations $1,756, and Interest Income $703. Target partially met.


i. Increase in NYSILC Young Adult Involvement (Outcome partially met)

a. Number of young adults participating in a training sponsorship (target 12): 3. While the SPIL target is identified as 12, recent changes made to the Pat Figueroa Scholarship (PFS) program makes it more realistic to project that 5-6 individuals will benefit from the opportunity each year.

Preliminary Assessment: Based on information documented in NYSILC quarterly reports and the NYSILC 2019-2020 Annual Report, three scholarships were provided to young adults during the year and one application was declined.

During the past year, the Youth Leadership Subcommittee approved two scholarships during the fall of 2019, both to the same young woman, a former Ms. Wheelchair NY from NYC. She attended and presented at a Cerebral Palsy conference in Hartford, CT October 7th on the topic: “Dream Big! Self Confidence through Self Advocacy.” She also attended the NYS Self-Advocacy Conference September 10th at the Albany Marriott. The committee was pleased with her leadership growth.

In the first quarter of 2020, an application was approved for a young adult from Brooklyn to attend the Families Together Annual Conference in Albany in April 2020, which was canceled due to COVID-19. A second application from the same person was received and respectfully declined because it was unrelated to disability and peer models. They encouraged him to connect future training applications with his leadership growth.

During the third and fourth quarter of 2020, due to the impact of COVID-19, all face-to-face training events were postponed or reorganized to virtual events and no applications were received. We will monitor this going forward and make recommendations of how to best support the leadership development of young adults with disabilities in New York. Target partially met.

page 30



b. Number of young adults actively participating in NYSILC’s youth leadership subcommittee (target 5): 5.

Based on information documented in NYSILC quarterly reports, the NYSILC Youth Leadership Subcommittee consists of five active members: Crystal Rivera is the lone council member, along with Lyndsi Wickert, David Dodge, Melanie Hecker, and Marc Rosen as non-council members. The chair of the committee is currently vacant. Brad Williams, NYSILC Executive Director, coordinates email communications for the group. The committee participates in the review and approval of PFS applications. The committee will need to reach out and recruit new members. The group is aware it must monitor the applications to the scholarship opportunity next year due to the impact of COVID-19 and make appropriate decisions. Target met.


Overall, this objective and its six outcomes with ten targets demonstrated very good progress during the past year.

Objective # 2: New Yorkers with disabilities will be actively engaged in promoting disability rights through the support of a Statewide Systems Advocacy Network (SSAN) and a statewide coordinator working with priorities identified in the NYSILC statewide needs assessment.

There were seven outcomes related to this objective with seven targets. Preliminary results indicate that the SSAN exceeded six outcomes and targets and met the remain set.

The New York State Education Department (ACCES-VR) contracts with the New York Association on Independent Living (NYAIL) to coordinate the Statewide Systems Advocacy Network (SSAN). ACCES-VR also contracts with the 15 ILCs that make up the SSAN. The SSAN sites in 2019-2020 were: ARISE Child & Family Services, Syracuse (ARISE); AIM Independent Living Center, Corning (AIM); Access to Independence of Cortland County, Cortland (ATI); Brooklyn Center for Independence of the Disabled, Brooklyn (BCID); BRIDGES, New City; Center for Disability Rights, Rochester (CDR); Center for Independence of the Disabled, New York, Manhattan (CIDNY); Independent Living Center of the Hudson Valley, Troy (ILCHV); Independent Living, Inc., Newburgh (IL,

page 31


Inc.); Northern Regional Center for Independent Living, Watertown (NRCIL); Resource Center for Accessible Living, Kingston (RCAL); Resource Center for Independent Living, Utica (RCIL); Southern Tier Independence Center, Binghamton (STIC); Westchester Disabled on the Move, Inc., Yonkers (MDOMI); and Westchester Independent Living Center, White Plains (WILC).

Coordination Efforts

NYAIL worked with SSAN advocates to develop the statewide agenda. Individuals participated in the educational day in Albany on 2/10/2020 in cooperation with the Consumer Directed Personal Assistance Association of NYS (CDPAANYS). In addition, the Assembly’s Legislative Disabilities Awareness Day was held much earlier than usual this year, on 1/29/2020. Many centers bring advocates to attend the event, which includes a couple of workshops on issues important to people with disabilities. The Assembly also passes a package of legislation important to the disability community. This year, the package included the following bills: to reinstate the Office for the Advocate for People with Disabilities; to create a visitability tax credit; and to establish a small business tax credit to incentivize the hiring of people with disabilities by small businesses.

With a pandemic declared mid-March and the entire state going under lock-down, this quarter was completely unprecedented. NYAIL and the SSAN’s typical busy quarter with the legislature in full swing was completely upended as the Capitol closed to visitors in March and the legislature did not hold regular session.

NYAIL joined with a broad cross-section of other statewide associations and organizations in educating the public and policy makers on how cutting $2.5 billion from Medicaid would impact Medicaid recipients, including people with disabilities. The group organized press conferences and rallies outside of the public hearings and meetings the MRT held in New York City and Albany. The group created a statement of principles which over 130 organizations signed onto. The group was active throughout the budget process and NYAIL organized the SSAN to participate in many of the group’s planned activities. Many advocates in the SSAN attended MRT meetings to speak to the impact cuts to long-term care would have on people with disabilities. Many advocates also participated in rallies outside of the meetings and helped amplify the message of Protect Medicaid! on social media. The legislators were very aware of the group’s efforts and the group also got some media coverage. As the Capitol shut down to the public due to the corona virus, much of the advocacy was centered on social media and the hash tag #NoMedicaidCuts trended two days in a row in New York. Unfortunately, many of the cuts were passed in the final budget, although some programs were saved – one of which is identified in the outcome/target section.

In the last quarter of year, the SSAN discussed the additional support from NYSILC to conduct outreach to identified unserved/underserved groups, such as minorities, LGBTQ, and immigrants with disabilities in their communities. SSAN sites identified ways

page 32


to connect with these groups and find ways to discuss common issues and work together. Based on information documented in SSAN quarterly reports, a survey was distributed to SSAN volunteers in the first quarter to assess the impact systems advocacy has on factors such as levels of personal empowerment and self-determination. The report will appear in Section 29, Additional Information.

Technical Assistance and Training

Technical assistance was provided to a new SSAN advocate. Web materials were provided for review, along with a follow up discussion about the distinction between advocacy and lobbying activities.

NYAIL held monthly calls with the SSAN sites, in addition to the monthly public policy subcommittee calls for issue discussions. These calls provide a forum for the advocates to discuss the activities they had been holding or planning. There was much discussion within the network regarding moving to virtual activities. The advocates shared the new ways they were providing public education activities and doing their activities. Many of the SSAN centers started holding regular Zoom sessions with advocates. New partnerships and new ways to provide public education and community activities were discussed and implemented throughout the quarter. This crisis provided a real opportunity for the SSAN to bring in new volunteers and to reach people virtually they may not otherwise have with the usual focus being on in-person participation.

Following the close of the second and third quarters, NYAIL provided SSAN advocates with their numbers to date for their performance outcomes. The Director of Advocacy provided follow up with sites that were behind on some of their deliverables.

Training and technical assistance opportunities were provided to the SSAN network during the year. They included:

• Review of Statewide Agenda, February 6th via teleconference for SSAN advocates.

• Get Out The Vote: Election Accessibility and Voter Rights, June 20th , via webinar for SSAN advocates and the statewide IL network.

• Roundtable on Accessible Absentee Voting, June 12th, via webinar for SSAN advocates and the statewide IL network.

• GOTV: The Time Is NOW, August 19th, via webinar for SSAN advocates and the statewide IL network.

Performance Outcomes/Targets:

1. Number of SSAN significant statewide system changes (target: 2). 2.

Preliminary Assessment: Based on information documented in SSAN quarterly

page 33


reports, the SSAN was supportive of and provided educational opportunities with individuals and stakeholders where the following systems changes were realized during the past year:

• ASL interpreter provided during daily COVID-19 press briefings! The Governor was holding daily press briefings regarding COVID-19, providing critical information for all New Yorkers. The lack of an on-screen ASL interpreter was preventing many in the deaf community from being able to get this critical information. A variety of deaf, protection and advocacy and IL advocates addressed the issue. Specifically, the concern was raised by two deaf systems advocates, calling for the State to include an on-screen ASL interpreter. Initially, the State responded to our advocacy by providing an interpreter on a live feed available on the Governor’s website. This did not meet the needs of many in the deaf community who rely on the television to access the briefings. Advocates in the network held press conferences to bring attention to the issue and wrote letters and made phone calls. Ultimately the matter was settled in the courts. However, much advocacy was done within the SSAN to raise this issue in the press and to put pressure on the State to provide an on-screen ASL interpreter.

• Funding for Olmstead Housing Subsidy and Access To Home Preserved! In the proposed Executive Budget, the funding allocation for Medicaid Redesign Team (MRT) supportive housing programs would have had its funding decimated. This housing allocation funds both the Olmstead Housing Subsidy (OHS) and Rapid Transition Housing, both of which provide rental subsidies for people with physical disabilities to keep them out of institutions. Access To Home which provides home modifications for people on Medicaid is also under the MRT supportive housing allocation. The SSAN worked to educate the public as NYAIL worked with other providers in a large coalition to raise awareness of the threats to people’s ability to remain out of institutions and homeless shelters. Ultimately, the funding was fully restored in the final budget. Target met.


2. Number of local partnerships and coalitions established by the SSAN network (target: 75): 138.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, the SSAN sites established 138 new partnerships and coalitions, exceeding the goal by 63. A few examples of the partnerships and coalitions include:

page 34


• STIC and the League of Women Voters of Broome and Tioga Counties joined forces in a Get Out the Vote (GOTV) effort to people with disabilities who are living in different living situations to help with voter registration and absentee ballot requests.

• ILCHV organized a public event on, “Marginalized People Leading Movements for Justice Because We All Have a Role and A Stake in Intersectional Justice.”

• BRIDGES partnered with the Rockland County Board of Elections to provide a virtual information session to educate voters about absentee voting.

• ATI joined the Cortland Forward Anti-Racism Taskforce with city council members, the mayor of Cortland, and other intersectional and community members to discuss issues of and solutions to racism in Cortland.

• ARISE Joined the Onondaga County Health Access Task Force—an outshoot of the Statewide Senior Action Council. Target exceeded.


3. Number of educational alerts disseminated to local volunteers by the SSAN network (target: 660): 901.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, SSAN centers sent a total of 901 alerts to local volunteers. This exceeded the target by 241. Educational alerts ranged in topic from Make Sure You Are Registered to Vote, Failures in COVID-19 Response Harms People with Disabilities and Older Adults, and #RacismIsAPublicHealthCrisis Twitterstorm. Target exceeded.

Confirmation/Evaluation Rating:Exceeded target/Met target/Substantially met target (more than 60%), Partially met target (less than 59%), Target not met.N/A or Other.

4. Number of local public education activities engaged in by the SSAN network (target 300): 661.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, the SSAN Centers exceeded this goal by engaging in 661 public education activities in the past year. This exceeded the target by 361. Examples of local public education include:

RCAL conducted Voter Registration and Absentee Ballot outreach and education to consumers and for members of the public.

page 35


ILCHV participated in NYAIL’s 30th anniversary of the ADA, sharing stories of experiences pre-ADA and how far we’ve come.

ATI Cortland’s SSAN advocate presented information about Independent Living to attendees of YOUTH POWER! for their Virtual Youth Leadership Forum.

WILC initiated and implemented a Get Out and Vote campaign geared towards persons with disabilities/minority persons with disabilities.

ARISE held their annual ADA Celebration—30th Anniversary—televised by WCNY TV-24.

LIFE at RCIL created a video with a person of color and a person with a disability talking about the intersectionality between race and disability. Target exceeded.


5. Number of grassroots organizing activities engaged in by the SSAN network (target: 90): 212.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, SSAN sites reported a total of 212 grassroots organizing activities for the year. This exceeds the target by 122. Examples of grassroots organizing activities include:

ATI Cortland’s ADA Celebration & Social Media Campaign — The SSAN advocate recruited local volunteer advocates and organized a virtual celebration/social media campaign surrounding the 30th anniversary of the Americans with Disabilities Act. Community members with disabilities submitted videos to be promoted on the ATI Facebook page throughout the month.

STIC convened a group of education advocates planning workshops for the coming year to address virtual special education and ongoing needs of students with disabilities.

ATI’s SSAN advocate organized and recruited local volunteer advocates to participate in two transportation focus groups in partnership with Cortland County Mobility Management; community members with disabilities shared their experiences using Cortland’s various transit systems and provided feedback to assist with updates to the Cortland County Coordinated Transportation Plan.

AIM ILC collaborated with a local Black Lives Matter group in Corning and Ray of Hope Church, an LGBTQ church in Elmira, on a get out the vote TV ad campaign to be aired on WENY, the local ABC and CBS affiliate. Target exceeded.

page 36



6. Number of oral or written public testimonies statements or letters provided, in response to a documented request, by the SSAN network (target: 90): 134.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, SSAN sites collectively provided 134 public testimonies during the past year. This exceeds the target by 44. Examples of public testimony include:

Several SSAN centers provided oral testimony at one of the joint legislative public hearings on COVID-19 and nursing homes, adult homes, and home care.

CIDNY, ARISE, RCAL, ILI, and NYAIL all spoke at one of two public hearings on the subject.

ILI submitted comments upon request to Disability Rights New York (DRNY) on their priorities for the coming year.

NRCIL provided written public testimony for the COVID-19 and Mental Illness Public Hearing.

Several SSAN centers provided comments to the NYS Department of Health regarding their implementation of cuts to long-term care which was passed in this year’s budget as part of the Medicaid Redesign Team (MRT) process. Target exceeded.


7. Number of CILs or SCILs without a SSAN contract voluntarily involved in the participation of the network (target 2): 4.

Preliminary Assessment: Based on information documented in SSAN quarterly reports, four non-SSAN centers continued to participate at various levels in the SSAN. Bronx Independent Living Services (BILS) is a member on NYAIL’s Housing Committee. Taconic Resources for Independence (TRI) is a member of NYAIL’s Employment and Transportation Committees. Western New York Independent Living (WNYIL) is active on NYAIL’s Housing Committee and participates in sponsored events. Finger Lakes Independence Center (FLIC) has staff participate in sponsored events and is a member of NYAIL’s Advocacy listserv. Target exceeded.

page 37



Overall, this objective and its seven outcomes with seven targets demonstrated significant progress during the past year.

Objective # 3: Increase the attendance and involvement of attendees from the IL network, including affiliated stakeholders, by providing support for a statewide independent living training conference.

This objective was inactive this year.

Objective # 4: Designate funds to develop and establish a database and interface that will compile, analyze, and interpret data from the statewide network.

There were five outcomes active this year related to this objective with five targets. Preliminary results indicate that one outcome and target was met, another set was partially met, while three outcomes and targets were not met. The timing for the implementation of the database/interface project is off by about 1-2 years. While progress is being made on the project, this situation has impacted the completion of outcomes and targets as originally planned. Please note that outcomes/targets not identified below (in numbered sequence) are inactive this year. Measurable Indicators/Performance Targets:

1. Number of centers testing the prototype of the statewide database and interface product, providing feedback related to the process: (target 20). 6.

Six members of the Database Work Group are affiliated with centers: WNYIL Buffalo, NRCIL Watertown, STIC Binghamton, CDCI Albany, BILS Bronx, and TRI Poughkeepsie. The involvement of centers will increase next year. In November 2020, an email was sent out to the network requesting centers identify their point of contact for their data systems and provided a brief survey to gauge the level of center capacity to export data. Moving forward, technical assistance will be provided related to data export/entry, overall training, and beta testing. Outcome and target partially met.

Confirmation/Evaluation Rating:Exceeded target/Met target/Substantially met target (more than 60%), Partially met target (less than 59%), Target not met, N/A or Other

2. Report on status of database and interface design based on collective center data

page 38


and feedback completed as documented: (target 100%). 100%.

Due to new contract delays, the database/interface project with web developer ES11 was delayed implementation until late January 2020. The major focus of the work was on the development of the web application database/interface build. Within a month, the interface user log-in system was completed. The rest of the web application build took 2-3 more months to complete in its initial format. The Database Work Group responded on the Basecamp communication board with comments when necessary.

In April 2020, Brad Williams of NYSILC and Ken Dibble of STIC had discussions with Rocco Semeraro of ES11 related to creating the first draft export. The goal was to test and have a functional export to use as a model for the centers to use when outreach was initiated.

Need was expressed to incorporate elements from the existing ACCES-VR database portal. The goal was to create one database site that the centers could visit to do all their reporting. Centers could upload/input their 6-month and yearend state reports in addition to the value-added statewide summary reports and queries. It would also include other important information i.e., “extra reportable items” (local systems advocacy grid/upload report, institutionalization outcomes, and optional vignettes).

In Mid-May 2020, an urgent small group meeting was held to discuss identified items from the ACCES-VR portal and the agreed upon the advantages of the goal of having one database for the network. Individuals discussed and debated items. Some saw the goal clearly; others focused more on the number of data points increasing and were concerned. ACCES-VR looked into it from their end. Brad Williams followed up after the meeting with ES11 to determine that the number of data points (starting at around 15) had the potential to expand by another 15, but realistically would only grow by another 5-7 because several of the items were choice options. ES11 noted that a database with a total number of data points between 20-22 is minimal. The web developers have been busy creating the import design based on the use of an export, along with a secondary page that can be used to provide manual input as a back-up option.

On July 23rd, Brad Williams had a follow up meeting with the web developers to confirm the requested changes to the web application to accommodate the data points from the ACCES-VR data portal. They agreed to meet again on August 4th to review the final changes. On August 4th, they reviewed an updated export and sample report. A few adjustments were identified. The rest of the meeting was spent identifying potential timelines for future logistical steps related to the final steps of the project, such as: finalizing the export/reaching out to centers and ensuring they had the capacity to upload an export, working on the extra reportable items for the web application, finalizing the reports, creating the training

page 39


manual/providing a webinar to the network, conducting beta testing, and launching the system in the fall of 2021.

On September 2nd, the Database Work Group met to review the web application and export. The group platform was not very functional (provided by the web developers). It was decided that in the future, members will get credentials to access the staging area directly. In a review of the web application, Ken noted some items he felt could best be addressed at a brief follow up meeting. Brad Williams shared a Word document that outlined the project’s timelines for future logistical steps for 2020-21 (noted above). The group supported them.

On September 10th, the NYSILC Executive Director, Ken Dibble, and ES11 met to briefly review the web application and then discuss the export. Ken had a couple of questions about the export, but in the end was satisfied with a strategy of reaching out to centers with the sample export.

Overall, the basic design of the web application was completed. Outcome and target met.


3. Percentage of centers that participate in training related to the product and process: (target: 75%).

Preliminary Assessment: As noted above, the timing of this outcome and target has been off by about 1-2 years due to project delays. It is anticipated in the project’s timeline of final action steps that training should be addressed next year (2021). Outcome and target not met.


4. Successful launch of the statewide database and interface, as documented: (target 100%).

Preliminary Assessment: As noted above, the timing of this outcome and target has been off by about 1-2 years due to project delays. It is anticipated in the project’s timeline of final action steps that the launch of the database/interface should occur in the final quarter of 2021. Outcome and target not met.

Confirmation/Evaluation Rating:

page 40


Exceeded target/Met target/Substantially met target (more than 60%), Partially met target (less than 59%), Target not met, N/A or Other

5. Percentage of centers that successfully are connected to the data interface and upload the first requested data: (target: 90%).

Preliminary Assessment: As noted above, the timing of this outcome and target has been off by about 1-2 years due to project delays. It is anticipated in the project’s timeline of final action steps that this outcome/target will be determined after the launch, which should occur in the final quarter of 2021. Outcome and target not met.


Overall, NYSILC remains optimistic about the progress made on the data interface project during the past year. The project’s timeline of final action steps will help to guide members toward a successful launch and transition to a new system for the statewide network to utilize.

Objective #5: Address priority unserved and underserved populations and issues by providing one (1) $72,000 demonstration grant opportunity that can be evaluated by the council, be held to its own unique set of deliverables (outcomes), and subsequently provide a best practices manual for the benefit of the statewide IL network.

There were six measurable outcome areas related to this objective with fourteen targets. Preliminary results indicate that four outcomes were met, one was partially met, while one was not met. Twelve of the targets were met, one was partially met, while one was not met. The last outcome area was a general “other” category that noted program highlights and challenges.

Taconic Resources for Independence (TRI) completed the third and final year of the demonstration project to provide Independent Living services to veterans with disabilities. The project has an additional focus on accessible and affordable housing and transportation in the community. TRI set up their own unique deliverables with the DSE and reports on them quarterly. The content from the quarterly reports serves as the basis for this report.

Measurable Indicators/Performance Targets:

A. Outreach

1. Agency will issue press releases and email blasts which will include printable materials. Will include information on all social media platforms.

page 41


Completed. This is the end of the grant. We continue to do outreach by posting on our various social media platforms as events and information arise. We post community veteran events and functions on Facebook, Twitter, and Instagram. A new social media platform called Alignable has become available. We have printable flyers, brochures, and business cards. Target met.


2. Veteran Specialist will collaborate with the Division of Veteran Services, Mental Health America Living Room Program (homeless program) and Hudson River Housing Veteran Services Program to promote services.

Completed. This is the end of the grant. As reported earlier our collaborators have changed. We continue to work on a regular basis with the Division of Veteran Services to help veterans with their compensation and pension claims. We continue to work with MHA Vet2Vet and the different programs they feature. Vet2Vet has expanded to include a woman’s veteran representative. We have continued to partner with WestCop as well who assists us with finding emergency and permanent housing for veterans. While the project has ended, TRI will still maintain appropriate contact with these important programs for service coordination. Target met.


3. Veteran Specialist will attend veteran specific events to conduct outreach and to remain current regarding the needs of Dutchess County Veterans.

Specific events and outreach were limited as noted in the Highlights and Challenges section below. All outreach conducted remotely as indicated above in the outreach section. Partially met.


Outcome and targets substantially met.

B. Program Planning

page 42


1. Veteran Specialist will create a dedicated database to document all activities and outcomes and provision of direct services and/or systems advocacy and outcomes.

Completed. This is the end of the grant. The database has been set up and the veteran drive has been completed. Input is ongoing and updated regularly. Taconic Resources for Independence is switching to CIL and contacts will be updated through that program. Target met.


2. Program Director and Veteran Specialist will join veteran related committees and coalitions.

Completed. This is the end of the grant. Veteran Specialist is a continuing member of the Dutchess County Veteran Task Force that meets monthly with like-minded agencies to discuss all events and happenings related to veterans in the county. The Veteran Specialist is also a member of the Hudson Valley Task Force that serves five counties, and the Program Director attends meetings as a guest. Attends Bringing Agencies Together (BAT) meetings to network and meet other non-for-profit agencies in the Dutchess County area. Veteran Specialist presents our veteran program information to agencies attending BAT meetings. While the project has ended, time permitting, TRI will l maintain contact with as many veteran related committees and coalitions as possible. Target met.


3. The Veteran Specialist will develop program specific intake assessment forms and outcome surveys.

Completed. This is the end of the grant. We found that our TRI assessment intake and goals assessment are adequate to use for this program. A veteran specific outcome survey has been developed and distributed to both past and present participants via Survey Monkey. Target met.


4. Veteran Specialist and Program Director will meet weekly to assess program

page 43


activities and plan best practices manual.

Completed. Final draft completed and accepted by Board of Directors and NYSILC. Target met.


Outcome and targets met.

C. Project Implementation:

1. The Veteran Specialist will provide comprehensive, quality independent living services to veterans with disabilities to assist them in becoming more stabilized in their homes and communities. The specialist will conduct in depth intake interviews, assessments of needs and define expectations and goals.

Completed. This is the end of the grant. 1 unduplicated person was served this quarter. The total number served this quarter was 2. Also, 3 stakeholders (individuals, families, agency personnel) were provided with one-time information and referral that allowed them to self-advocate/resolve issues without the need for a full intake. 1Now that the Special Populations/Housing and Transportation Demonstration Project has ended, we will implement what we have learned in the past three years from developing the procedure manual. Especially, the new partner’s agencies we have establish in our community to assist our veteran service population. Target met.


2. The Veteran Specialist will communicate with agency partners to provide appropriate linkage to supports and services for veterans with disabilities.

Completed. This is the end of the grant. Vet2Vet is available in multiple counties and the Veteran Specialist works closely with the Dutchess County branch. Our partners include the Dutchess County Division of Veterans Services, Veteran Affairs at Castle Point and Montrose, The Social Security Administration, WestCop and Legal Services of the Hudson Valley. We are also part of and share outreach and support related information to B.A.T. members, which is comprised of many nonprofits in Dutchess County. TRI will continue to maintain communication with partner agencies as appropriate to provide Independent Living Services, resources, and connections to our veteran service population.

page 44


Target met.



D. Service Provision:

1. The Veteran Specialist will link veterans to/with supports, services, and benefits as needed.

Completed. This is the end of the grant. We have linked veterans with disabilities to many services at many agencies throughout the county including the Dutchess County Division of Veterans Services, the local VA, the Department of Labor, Social Services, West Cop, Legal Services of Hudson Valley, and the MHA Vet2Vet programs. Target met.


2. The Veteran Specialist will provide independent living skills training based on individual needs and goals.

Completed. This is the end of the grant. Most independent living skills training involves assisting veterans with disabilities with filling out forms, requesting paperwork and filing for benefits from Veteran Affairs. We find many veterans use our in-house Social Security disability benefits counseling services. We assist veterans with applying for compensation and pension with the Veteran Service Office. We also help them apply for VA Healthcare. TRI will continue to maintain communication with partner agencies as appropriate to provide Independent Living Services, resources, and connections to our veteran service population. Target met.



E. Systems Advocacy:

page 45


1. The Veteran Specialist will assist veterans with disabilities in organizing a committee for conducting systems advocacy especially as it relates to affordable/acceptable housing. The Veteran Specialist will offer the opportunity to join the systems advocacy group to each veteran during the intake process.

This is the end of the grant. Offered to consumers on goals form during intake. Noted on assessment sheet. We still have difficulty forming the systems advocacy group as veterans are looking for immediate housing options, but we continue to strive to accomplish this goal after the grant. However, we did not meet the goal. We have found that homeless veterans are in crisis and are looking for emergency services and not currently interested in a systems advocacy group at this time. Not met.


Outcome and target not met.

F. Best Practices Manual:

1. The Veteran Specialist will research, collect, and store data and document all activities and outcomes to be used to inform a best practices manual. The Veteran Specialist will use data collected from the intake interview and exit surveys to document outcomes or attainment of goals.

Completed. We utilized our consumer statistics and exit surveys that had been distributed to current and past consumers to inform of our work. The final draft of our best practice’s manual has been approved by NSILC and our Board. Recent additions were made to address the coronavirus pandemic and approved by NYSILC. Archived on NYSILC website: https://nysilc.org/images/TRI_Veterans_Program_Manual.pdf. Target met.


2. A daily log will be kept tracking activity that will also be used to inform the best practices manual.

This goal was completed. Target met.

Confirmation/Evaluation Rating:Exceeded target/Met target/Substantially met target (more than 60%), Partially

page 46

https://nysilc.org/images/TRI_Veterans_Program_Manual.pdf


met target (less than 59%), Target not met, N/A or Other


G. Other:

Highlights:

Ongoing. TRI continuing talks with the new Dutchess County Veterans Service, Adam

Roche, what we started with the former Veterans Service Office about establishing a Veterans Court with the District Attorney. This would help returning individuals recently released from active duty who are suffering from disabilities like Post Traumatic Stress Disorder and Traumatic Brain Injury. This will be a going objective of TRI to work with all parties to set up a Veterans Court system.

TRI has been working with Westcop/Supportive Services to assist our veteran consumers in finding housing and obtain resources. Our former Veteran Specialist is now working at Westcop/Supportive Services, and one of his regions is Dutchess County.

Challenges:

Ongoing.

The ongoing Coronavirus Pandemic has made our role more difficult especially last quarter our veteran specialist found another position with Westcop/Supportive Services for Veteran.

It was a challenge to find a replacement for a veteran specialist for the last quarter of the grant, going through the interview process and hire a person for three months.

.

page 47

Documents

2020 SPIL Evaluation Findings - NYSILC · Web viewBrad Williams shared a Word document that outlined the project’s timelines for future logistical steps for 2020-21 (noted above)