Overview of The Wisconsin PROMISE Program Procedure Manual

Overview of The Wisconsin PROMISE Program

Procedure Manual

February 13, 2014

● Your manual describes the recruitment, enrollment, and random assignment process for Wisconsin PROMISE

● Your manual has been customized for your program specific processes

● Each manual includes materials that might be useful for the recruitment, enrollment, and random assignment processes as well as materials that Wisconsin PROMISE has already developed

Overview of Manual

2

● Study Overview● Research Procedures● Verifying Group Assignment● Data Security Procedures● Resources and Support for Study Enrollment● Other Evaluation Activities

Overview of the Presentation

3

I. Study Overview

● A joint initiative of the U.S. Departments of Education (ED), Health and Human Services, and Labor, and the Social Security Administration (SSA)

● Objectives – Immediate: Improve the provision and coordination of

services for youth ages 14-16 receiving Supplemental Security Income (SSI) and their families

– Long-term: Improve educational attainment and employment outcomes and reduce reliance on SSI

Promoting Readiness of Minors in Supplemental Security Income (PROMISE)

5

● Six PROMISE programs– Funded by ED for five years

● Recruit and enroll at least 2,000 youth and their families over a two-year period– Half will be in the program group that will receive

PROMISE services and half will be in the usual services group

● Mathematica provides technical assistance around recruitment, enrollment, and random assignment

Promoting Readiness of Minors in Supplemental Security Income (PROMISE)

6

II. Research Procedures

Step 4: Program staff notify youth and family of group assignment and eligible services

Step 3: Program staff enter data into RAS and youth is randomly assigned to a study group

Step 2: Youth and family complete consent form in its entirety and program staff review for completeness

Step 1: Program staff conduct recruitment and outreach efforts of eligible youth

Four Key Study Enrollment Steps

8

Research Procedures

Step 1: Conduct Outreach and Recruitment Targeted to Eligible Youth

● 1.1 Wisconsin PROMISE recruitment approach– Direct mailings of recruitment packets will come from the

University of Wisconsin Research Team▪ Recruitment packets include an outreach letter, consent form,

paper intake form, PROMISE Intake Attendant (PIA) contact information, pre-addressed and pre-stamped return envelope.

– Other avenues of outreach:▪ Print advertisements- newspapers and magazines▪ Electronic advertisements- e-mail, social media, websites▪ Out of home advertisements- billboards, flyers▪ Multi-media advertisements- radio, TV, cinema, mobile▪ Outreach from existing programs and schools

Step 1: Conduct Recruitment and Outreach Efforts of Eligible Youth

10


11

● Study rationale ● Sponsors and program objectives● Location of PROMISE programs● Study objectives● Research design● Research activities for Mathematica● Research activities for PROMISE programs● Study schedule● Study contacts

Appendix A: External Project Description

12

● Overview of PROMISE● Role of National Evaluator● Research Questions● Evaluation Design and Activities● Analyses and Reports● Timeframe● Stakeholder Roles

Appendix B:Study Summary PowerPoint Presentation

13

Appendix C:Wisconsin Outreach Letter

Introduces Wisconsin PROMISE as part of a federal program to learn how to help youth with disabilities meet their school and work goals

Explains how to be included in the study

Includes contact information for the PIA.

Provides contact information for the project PI

14

● 1.2 Develop the Means to Track Outreach Efforts– Wisconsin recruitment and outreach activities will

be recorded by Research Specialists from the University of Wisconsin Research Team.

– The eligible youth list will be sorted and prioritized for recruitment purposes based on:

▪ Age (statewide): youth 16.5 to 16.75 years old, statewide, will be prioritized for the first round of recruitment mailings

▪ Geographic (Milwaukee): zip code 53215


15

● 1.2 Develop the Means to Track Outreach Efforts– Recruitment and outreach activities will be tracked

in excel and will track:▪ Date enrollment package was sent to eligible individual▪ Date the 6 and 12 week follow up mailings were sent (if

enrollment materials are not received within 6 weeks of the initial mailing date

▪ Date enrollment materials are received by Research Specialists

▪ Completeness of enrollment materials; follow-up with participants/family via phone may be required.

▪ Record which PROMISE Intake Coordinator (PIC) assisted participant with enrollment materials, if applicable, based on the individualized PIC return address mailing labels.


16


PROMISE Administrative Recruitment & Enrollment Tracking System

Eligible Individual/Participant Information

Last

Name

First

NameAge

Mailing

AddressCity

Zip

Code

Date

Enrollment

Package

Sent

Date 6

Week

Follow-

Up Sent

Date 12

Week

Follow-Up

Sent

Date

Enrollment

Materials

Received

at SVRI

PIC

Name

Date of

Phone

Follow-Up

w/

Participant

T

or

C

Date

Group

Assign.

Letter

Sent

Gift

Card

Date

Participant

Data

Entered in

InfoPath

17

● 1.3 Use the SSI lists to recruit participants– Initial lists will be processed by Mathematica in February 2014

and will be released by SSA to Wisconsin PROMISE once the MOU between SSA and the program has been finalized

– Youth on the SSI lists will have unique study ID numbers, assigned by Mathematica

▪ Social Security numbers (SSNs) will not be provided

– Lists will be transmitted securely from SSA to Wisconsin PROMISE programs


18

Appendix F:SSA Data Flow Diagram

19

Appendix G:Key Fields Included in the SSA Lists

- Study ID- Youth date of birth- Youth sex- Youth name- Youth telephone

number- Spoken language

preference- Disability name- First and last date

age eligible

- Mailing address for the youth

- Name and contact information for the Representative Payee

- Residential address if the youth and the representative payee do not live together

● SSI lists will include the following data fields:

20

● 1.3a Community Outreach, Recruitment, and Enrollment Procedure– PICs and Research Specialists will attend community

outreach events– PICs will identify a private area where enrollment can be

completed▪ Research Specialists will check the Random Assignment

System (RAS) to determine PROMISE eligibility

– Youth may complete the enrollment form:▪ On site with or without assistance from the PIC▪ Independently at a later time


21

● 1.3b Correct undeliverable addresses– SSA lists will be matched with Medicaid records to

provide most up to date addresses– We also recommend:

▪ Communicating with schools to verify SSA addresses▪ Searching records from other participating partners

– Other Resources:▪ People Finder- free searches based on a person’s name,

age, address, and/or phone number▪ Accurint- in-depth searches for specific individuals, batch

searches based on a list of names, historical mailing and phone numbers, deceased indicators, reverse phone searches, and social network searches


22

● 1.4a Using updated SSI lists of eligible youth– Updated lists of PROMISE-eligible youth will be

provided in July 2014, January 2015, and July 2015– The updated lists will reflect changes such as:

▪ Youth entering or exiting the eligible age range▪ Moving into or out of the designated PROMISE

service delivery areas▪ Beginning or ending the receipt of SSI


23

● 1.4a Using Updated SSI Lists (continued)


Status Code Definition Action(s) to Be Taken

1 Eligible, not enrolled Begin or continue recruitment efforts

2 Ineligible, not enrolled Cease or do not begin recruitment efforts

3 Enrolled Continue to deliver services, update contact information if necessary

24

● 1.4a Using Updated SSI Lists (continued)– I will be in touch as you approach and meet your

enrollment target of 2,000 youth– Wisconsin PROMISE will not provide PROMISE services to

additional youth after the enrollment target has been met. ▪ Any remaining PROMISE eligible youth will receive the usually

available services.


25

● 1.4b PROMISE program youth who move during the study period– Move to another Service Delivery Area:

▪ Provide information for program staff in the new service delivery area

▪ Contact your counterpart in the new service delivery area to inform them that a program group youth will soon be entering the area

– Move to a non-service delivery area but can either manage the logistics or participate in services remotely, so they can continue to receive PROMISE services

Step 1: Outreach and Recruitment

26

● Recapping Step 1– Outreach activities include direct mailings and other

strategies such as newsletters and social networking– Tracking outreach efforts through Excel– SSA will be providing lists of eligible PROMISE youth to

Wisconsin PROMISE – Lists will be updated to reflect changes such as youth moving

in or out of the service delivery area, youth aging in or out of the eligible age range, and youth who begin or end SSI receipt

– You should make efforts to continue service delivery to youth who move to another program area and you should determine procedures for dealing with youth who move out of the service delivery area.

Step 1: Outreach and Recruitment

27

Research Procedures

Step 2: Youth and parent/guardian complete consent and intake forms;

program staff review for completeness

● 2.1 Obtain Youth and Parent/Guardian Consent– Consent and intake forms completed at home

▪ Once the consent and intake forms have been completed, youth will mail their forms to the University Research Team. Research Specialists will enter the consent form information into the RAS and begin random assignment

▪ Youth may contact the PROMISE Intake Attendant for additional information

– Consent and intake forms completed in person▪ PICs may administer the form ▪ Youth may decide to take the form home to complete

Step 2: Complete and Review Consent and Intake Form

29

● What: A new federal program to help with school and work goals

● Who: Youth 14-16 who receive SSI and their families● Why: Learn what services are best at helping meet these

goals● $30 incentive and $10 for each phone interview● Study group assignment● Privacy● Risks● Choice● Right to leave

Appendix H: Wisconsin Consent Form

30

31

● Collects additional information on youth and family including:

Appendix I and J: Intake Form and Intake Form Instructions

- Number of people in the household

- The living arrangement- Primary language spoken at

home- Annual household income- Household Public Assistance- Race/ethnicity- Gender- Educational attainment and services

- Employment history and

future prospects- Health status and primary disability

- Parent expectations for youth-Name, Date of birth, and Social Security Number for other household members over 15 years old

32


33

● Addresses questions about:– General aspects of the national evaluation– PROMISE study groups and group assignment

process– Consent form– National evaluation surveys

Appendix K: Questions and Answers about the PROMISE Evaluation

34

● 2.1a Withdrawing from the program– If a youth or parent/guardian consents and later

decides that they no longer want to participate▪ They must submit a written letter the program

director▪ At that time, Mathematica and the PROMISE

program will cease attempts to contact the youth to participate in data collection


35

● 2.1b Refusal to Provide Consent– If a parent/guardian or youth do not consent:

▪ Inform them that they may access only usually available services

▪ If they change their mind, they may complete the state consent form and be assigned to a study group at that time.


36

● 2.1c Outcome Based Payments for Enrollment– PICs will receive a $60 incentive for each youth they

successfully enroll into PROMISE. ▪ They will also receive reimbursement for travel costs if

travel exceeds 100 miles.– PICs will have unique return labels which will be used

to track how many youth each PIC enrolls


37

● 2.2 Review the forms to ensure they are completely and correctly filled out– Obtain any missing information from the youth and

family– Enrollment and random assignment cannot occur if key

information is missing from the consent form


38

● Recapping Step 2– Youth and parental consent is required for PROMISE

participation– Wisconsin has created its own consent and intake forms– Wisconsin has customized a FAQ document to help obtain

consent from youth and their parents/guardians– If youth withdraw from a study, the PROMISE program and

Mathematica will cease contact with the youth– Consent forms should be reviewed for completeness before

advancing to random assignment


39

Research Procedures

Step 3: Enter Information Into the RAS and

Randomly Assign Youth

● The PROMISE Random Assignment System (RAS)– Verifies eligibility for PROMISE– Gathers contact information on youth and their

parents/guardians– Conducts automated validation checks on the data

entered – Checks that youth have not previously been

randomly assigned– Conducts random assignment– Produces reports on recruitment and enrollment

progress

Step 3: Enter Information into RAS and Randomly Assign Youth

41

● System Security:– Secure password requirements– Multifactor authentication – Session timeouts – 20 minutes and two hours– Data is restricted to specific program and

region, as well as the user account level– All data in the development system is test

data


42

● 3.1 Log in to the RAS– Create a username and password after receiving an

email invitation from your RAS program administrator– Access the RAS at : https://www.promise-ras.com


43

● Log in Screen


44

● Reasons for being kicked out of the RAS:– Entering an incorrect password three times in a row

will lock your account– Not completing the new password and multifactor

authentication within 15 minutes of receiving a password reset email will deactivate your account

● Both of these scenarios will require a password reset by your administrator


45

● Welcome Page


46

● Enroll Youth Link


47

● Eligibility Validation Screen


48

● Data Entry Page


49

● 3.2 Enter information from the consent form


Required Fields for Random Assignment Optional FieldsYouth Data Fields

Youth first name Youth middle initialYouth last name Youth email addressYouth signature Youth Facebook usernameDate of youth signature Youth Address line 2At least one telephone number for youth

Youth Social Security NumberYouth date of birthYouth genderYouth address line 1Youth city/state/zipRegion

50

● 3.2 Enter information from the consent form


Required Fields for Random Assignment Optional FieldsParent/Guardian Data Fields

Parent/guardian first name Parent/guardian middle initialParent/guardian last name Parent/guardian email addressParent/guardian signature Parent/guardian Social Security NumberDate of parent/guardian signature Parent/guardian address line 2At least one telephone number for parent/guardianc

Parent/guardian date of birthParent/guardian genderParent/guardian address line 1Parent/guardian city/state/zip

Sibling Data FieldsDoes the youth have a sibling who enrolled in the study at an earlier date (Yes/No)?

[If Yes/No] Sibling’s middle initial

[If Yes] Sibling’s first name [If No] Sibling’s first name[If Yes] Sibling’s last name [If No] Sibling’s last name[If Yes] Sibling’s Social Security Number [If No] Sibling’s Social Security Number[If Yes] Sibling’s date of birth [If No] Sibling’s date of birth

51

● Data Entry Page


52

● Two additional system checks:– Sibling SSN and DOB against previously enrolled

youth (if sibling information is provided)– Parent/guardian SSN and DOB against

parents/guardians of previously enrolled youth (if no sibling information is provided AND parent SSN is included)


53

● 3.3 Random assignment of youth to program or usual services group– To conduct random assignment, click “Continue to

Random Assignment”– To step away from your computer prior to random

assignment, click “Save Draft”


54

● Youth Summary Screen


55

● 3.3 Random assignment of youth to program or usual services group– For assistance, email the RAS Help Desk

▪ [email protected]

– Call 855-776-6443 from 8:00 am to 4:00 pm Eastern Time

– Do NOT include specific youth information in a voice mail message or email message


56

mailto:[email protected]

● 3.4 Updating youth contact information– Youth address and phone number can be updated

in the RAS or state tracking or MIS system▪ If only updating contact information in the state

tracking or MIS system, Mathematica may periodically request updated contact information

– Youth and sibling date of birth and SSN cannot be updated in RAS

▪ Contact RAS Help Desk for assistance if this information needs to be changed during the enrollment period


57

● Manage My User Information


58

LIVE RAS DEMONSTRATION

● Prior to entering information into the RAS, identify “wild card” cases– Five wild card exemptions per PROMISE program– Reserved for cases for which staff present compelling

reasons for providing PROMISE services– Obtain approval from the program director or a

representative for the program director▪ Program director should contact RAS Help Desk to

implement a wild card exemption

– Once random assignment occurs, the result cannot be changed


60

● Recapping Step 3– Security features have been put in place to ensure

proper use of the RAS– The RAS will collect consent form information about

youth, parents/guardians, and siblings ▪ You should confirm that the consent form is completed

in its entirety prior to data entry▪ You should enter as much data as possible

– The RAS will conduct validation checks to:▪ Ensure that the youth was not previously enrolled▪ Assign youth to the same study group as his/her sibling,

if applicable▪ Confirm that all required data is entered correctly


61

● Recapping Step 3 – The RAS will assign youth to one of two study groups:

▪ PROMISE program group ▪ Usual services group

– Information about the youth case status and study group assignment can be found in the RAS

– Youth and parent contact information can be updated through the “Go to youth information link” at any time after enrollment.

– The RAS Help Desk can provide technical assistance with the system

– Wild cards will be identified before random assignment and should be reserved for compelling cases


62

Research Procedures

Step 4: Notify the Youth Participant of His or Her Group Assignment

● 4.1 Print Group Assignment Notification Letter– Click on the “Generate Notification Letter” link

▪ Will create a customized PDF document for the applicant with his or her study group assignment


64

Appendices L and M: RAS Notification Letters

65

● Wisconsin has customized notification letters

● Generic Spanish versions of these letters will be available– We recommend

attaching the generic Spanish version to the customized English letter printed by the RAS

● 4.1a Notify PROMISE program youth– Youth assigned to receive PROMISE services will be

mailed▪ RAS notification letter▪ A gift card▪ Informational brochure on Wisconsin PROMISE

– Youth will be contacted by a DVR Case Coordinator within 1-2 days after random assignment to connect youth to a DVR Case Manager


66

● 4.1b Notify Usual Service group youth– Youth assigned to receive the usually available

services will be mailed▪ RAS notification letter▪ A gift card


67

● Thank the youth for taking part in the study● Remind the youth that assignment was random● Inform the youth of his/her study group● Describe what services the youth may access based

on the group assignment● Explain how to access PROMISE services or usual

services● Remind the youth of the 18-month survey● Thank the youth again

Appendices N and O: Scripts for Informing a Youth Participant

of His or Her Study Group Assignment

68

● 4.2 Handling upset youth or parents/guardians– Use script for speaking with youth who are upset

with assignment to the usual services group (Appendix P)


69

● Draw from your own experience and training for how to talk with upset clients

● Use active listening to affirm that you understand the concerns of the youth and parents/guardians

● Reiterate that the assignment was done randomly● Explain that they can still access the usually available

services and provide examples● Offer to connect them to some of those services● Inform them that they may talk with a supervisor if

they are still unhappy

Appendix P: Script for Speaking WithUpset Youth or Parents/Guardians

70

● Recapping Step 4– Notification letters will be generated by the RAS

▪ Programs may choose to provide an alternative language form in addition to the official notification

– Wisconsin PROMISE youth will be mailed a notification letter and will receive a phone call from DVR Case Coordinator

– Scripts are provided to assist with notifying participants of their group assignment and for handling upset youth or parents/guardians


71


Procedure Manual

Verify Group Assignment

● It is critical that only program group youth receive PROMISE services

● Deviation from this standard would compromise the reliability of the study’s findings


73

● Check for previous random assignment in the RAS– Enter the youth’s SSN and date of birth in the RAS eligibility

validation


74

● Identify study group status in state MIS system – Research Specialists will enter the participation information

from the intake form and the RAS into the InfoPath site. Research Specialists will ensure that group assignment is entered accurately.

– DVR Consumer Case Coordinators will use the InfoPath site to access a list of treatment group participants who will be entered into the IRIS DVR Referral System and will be tagged as PROMISE participants .

– Case Managers and Case Coordinators can check study group assignment in IRIS.


75

● Mathematica will monitor group assignment integrity– Mathematica will compare study group counts in the RAS to

study group counts from the state MIS


76


Procedure Manual

Security Procedures for the Evaluation

● PROMISE management and staff must protect the confidentiality of personally identifiable information (PII)– Examples include: name, date of birth, address,

telephone number, Facebook name, and SSN

● Particular attention should be given to security when collecting youth information at community outreach events

Security Procedures

78

● Storing and mailing consent forms – You may choose whether to store the consent forms at the

PROMISE offices where they are collected or in one centralized location

– Mailing: ▪ Establish a schedule for sending forms to that location▪ Send the forms via Certified or Registered Mail, UPS, or FedEx

with tracking and receipt confirmation requested. – Storing:

▪ Consent forms should be kept locked up in appropriate storage units, such as locking file cabinets.

▪ Only authorized staff should have access to these storage units, as well.

● Consent forms should be able to be retrieved should a youth or parent/guardian have a question about consent.

Security Procedures

79

● Steps to ensure confidentiality: – Store materials with PII out of sight and locked in

appropriate locking storage when not in use– Keep work surfaces and open storage areas clear

of materials containing PII. Never leave paper documents or portable media containing PII unattended

– When displaying PII on your computer, ensure that unauthorized people cannot see the information

– Log out of the RAS and activate a screen saver whenever you leave your computer

Security Procedures

80

● Steps to ensure confidentiality (continued):– Do not email PII or include PII in the subject line or

body of emails. Do not attach documents that contain PII.

– Be sure that unauthorized people cannot overhear PII being discussed in person or over the phone

– Report the loss, disclosure, or misuse of PII to your supervisor immediately

Security Procedures

81

● Recapping Security Procedures– PII is personally identifiable information such as name, date

of birth, and Social Security Number– It is the responsibility of PROMISE staff to ensure

confidentiality of PII– Security procedures should be put in place for storing and

mailing consent forms– Follow the identified suggestions for maintaining PII security

Security Procedures

82


Procedure Manual

Evaluation Support and Other Research Activities

● Your program’s Evaluation Liaison will:– Gather information to customize procedures– Minimize data collection burden on program staff– Develop plans to address issues that might affect

the ability to evaluate the programs

Resources and Support for Study Enrollment

84

● The National Evaluation Team will provide technical assistance on recruitment and enrollment, random assignment, and ongoing data collection via:– Conferences and webinars– Web-based training– In-person training


85

● RAS Help Desk– PROMISE program staff may contact the RAS help

desk with questions and concerns– 8:00 am to 4:00 pm Eastern Time, Monday–Friday– [email protected]– 855-776-6443


86

mailto:[email protected]

● Process Study– Telephone interviews with program directors

▪ Winter 2014

– Site Visits▪ Summer/Fall 2014 and Winter 2016▪ Interviews with state and local staff▪ Observation of program operations▪ Case reviews▪ Focus groups with PROMISE youth and parents

Other Evaluation Activities

87

● Impact Analysis– Youth and Parent/Guardian Surveys

▪ 18 months and 5 years after random assignment– State Administrative Data

▪ State Vocational Rehabilitation and Medicaid data▪ Additional state data for future SSA analyses

include Unemployment Insurance benefits, SNAP/TANF benefits, and education outcomes

– Federal Administrative Data▪ Data from IRS and SSA automated records to

capture annual earnings and monthly disability benefit amounts

Other Evaluation Activities

88

PROMISE Contacts

ED Point of Contact for PROMISE DemonstrationMarlene [email protected] (202) 245-7525

SSA Point of Contact for PROMISE EvaluationJeffrey [email protected] (410) 597-1815

Mathematica Project Director for PROMISE EvaluationThomas [email protected](202) 484-4698

Wisconsin Site LiaisonRebekah [email protected](609)945-6598

89

Wisconsin PROMISE

THANK YOU!

90

Documents

Overview of The Wisconsin PROMISE Program Procedure Manual