SPF SIGState-Level Evaluation

PARTICIPANT LEVEL INSTRUMENT (PLI)

SAMHSA/CSAP cross-site evaluation overview

Purpose of the PLIData collection required for the PLIRoles and responsibilitiesTiming and submission deadlinesResourcesQ & A

WEBINAR AGENDA

All communities funded by SPF SIG are required to submit cross-site evaluation data.

Quick Cross-site evaluation terms in

cross-site evaluation tools:“Grantee” (State of AK who received the SPF SIG)

“Subrecipients” (6 funded communities in Alaska)

SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

CLI: Part 1 –progress through the Strategic Prevention Framework

CLI: Part 2 –information about the specifi c interventions being implemented (adaptations, # reached, cost, demographics)

PLI – participant change in alcohol consumption resulting from individual level interventions

Completed annually throughout the SPF SIG (Steps 1-5)

Completed quarterly for each intervention once implementation begins (Step 4)

Completed by participants of individual level interventions at pre, post, & follow-up (Step 4)

SAMHSA CROSS-SITE EVALUATION INSTRUMENTS

5

PURPOSE OF THE PLI

Why does SAMHSA require this?

Determine the specific types of interventions and strategies that work, for which populations, and under what circumstances.

Determine strategies that are most effective in producing outcomes related to alcohol consumption

One version for adults and another for youth

Environmental strategies: All middle/high school youth in your community All adults in the community

(examples: media campaigns, policy change in schools, limiting hours of operation in bars/liquor stores)

Individual-level strategies: “Captive Audience” – Face to face contact with

individuals is a key component target small sub-groups or individuals in the

community

(examples: Education program for adults who get a DUI, Lifeskills camp for youth)

WHAT ARE INDIVIDUAL-LEVEL STRATEGIES?

Interventions/ Strategies are planned activities intended to change alcohol related behavior (not building general capacity)

REMINDER ABOUT STRATEGIES

The minimum data collection requirement includes:

A Baseline Survey. Baseline data collection must be conducted before the participant has been exposed to any intervention services and can be conducted any time during the 30 days preceding program entry.

An Exit Survey. The exit survey must be administered after the participant has completed the entire program but no later than 10 days after program completion.

A Followup Survey. The follow-up survey must be administered to program participants at least three months after program exit.

Paper and pencil format

WHEN TO ADMINISTER THE PLI

You can add additional evaluation measures specific to the goals of your strategy

Also fulfi lls state requirement of tracking change in alcohol consumption for each strategy

Help you determine the reach of your strategy which is essential for process evaluation

Help to complete the demographic information required for each strategy on the CLI: Part II.

USEFULNESS OF PLI

Every person who participates in an individual-level strategy must have a unique ID

This same ID is used on each PLI survey participants complete (baseline, exit, and followup)

Following up with participants more than 3 months after the program is diffi cult

Issues around parental consent for surveys done in schools (for youth)

SAMHSA may decide to change some of the survey items in the future

CHALLENGES OF THE PLI

We can provide generic examples of the following to use if helpful or needed:

Parental consent formsYouth assent formsConsent to Contact forms for follow-upLetter describing the SPF SIG and the purpose of

the PLI survey to inform others as needed

Your local evaluator likely has forms they have used before

HOW CAN WE HELP YOU?

Required PLI Questions

Gender (Q1)Ethnicity (Q2-3)Age (Q4)Military service (Q5-6)Past 30 day alcohol use (Q7C)Age at fi rst use of alcohol (Q8C)Approve of someone your age having 1-2 drinks a day

(Q9E)Perceived risk of having 5+ alcoholic drinks 1-2x/week

(Q10C)Questions about attitudes/experiences (Q11-14)Days (in past 30 days) with 5+ drinks in couple of hours

(Q17)Average # drinks on days you drank in past 30 days (Q18)

CONTENT OF THE PLI -YOUTH

Optional Alcohol-Related PLI Questions

How many days consumed alcohol during your life? (Q15)

How easy is it for you to get alcohol? (Q19)How many students in your grade drink alcohol? (Q20)How many closest friends been drunk in last 30 days

(Q21)How do you usually get alcohol? -Multiple choice (Q22)Thoughts and feelings about alcohol use (Q23a, b, c)Where and when have you used alcohol in past 30

days? (Q24)

CONTENT OF THE PLI -YOUTH

Required PLI Questions

Gender (Q1)Ethnicity (Q2-3)Age (Q4)Military service (Q5-7)Past 30 day alcohol use (Q8C)Age at fi rst use of alcohol (Q9C)Perceived risk of having 5+ alcoholic drinks

1-2x/week (Q10C)Questions about attitudes/experiences (Q11-13)Days (in past 30 days) with 5+ drinks in couple of

hours (Q16)Average # drinks on days you drank in past 30 days

(Q17)

CONTENT OF THE PLI - ADULTS

Optional Alcohol-Related PLI Questions

How many days consumed alcohol during your life? (Q14)

In past 30 days, how many times did you ride with someone who had been drinking? (Q18)

In past 30 days, how many times did you drive a car or other vehicle when you had been drinking? (Q19)

CONTENT OF THE PLI- ADULTS

Surveys submitted manually online https://www.pmrts.samhsa.gov/pmrts/Project Directors and Evaluators are

responsible for ensuring data is enteredAn Excel fi le can be downloaded at any time

which will include all PLI data you have entered to-date.

Excel fi le can be used for local data analyses and to enter any added survey variables.

Reminders will be sent by state evaluators of upcoming deadlines

ENTERING PLI DATA

Quarterly Survey collection period

Data Entry Due Dates

1.1.13 - 3.30.13 4.15.13

4.1.13 - 6.30.13 7.15.13

7.1.13 - 9.30.13 10.15.13

10.1.13 - 12.31.13 1.15.14

SUBMISSION DEADLINES FOR PLI

…..Continue with same schedule throughout life of grant

Surveys should be entered as soon as possible upon completion

No later than by the end of the quarter

18

One-on-one assistance by state evaluators Submission deadlines schedule All surveys, guides, codebooks, webinars

etc. will be put out on the SPG SIG website by Oct 31

http://alaskaspfsig.org/Click: Tools + Resources tabUnder: Evaluation

Frequently Asked Questions Federal Cross-site evaluation team collects grantee

questions and disseminates FAQ document to grantees regularly.

WHAT OTHER ASSISTANCE IS AVAILABLE?

19

WHAT ASSISTANCE IS AVAILABLE TO COMMUNITIES?

(CONT’D)

On-line Instrument Training Video Interactive videos demonstrating navigation

through the web-based MRT system.http://kitusers.kithost.net/support/diticsupport/MRT/SPFSIGInstrumentsGLICLIPLITrainingVideos/tabid/1114/Default.aspx

Help Line If you experience technical diffi culties in

using the web-based system, contact the DITIC Help Desk by email or phone

1-888-DITIC-4-U (1-888-348-4248) Monday - Friday, 9:00 am - 8:00 pm EST