Instructor: Bess A. Rose What examples of logic models have you found since the last class? Where did you find them?

Instructor: Bess A. Rose

What examples of logic models have you found since the last class?

Where did you find them?

A legislator is calling you….

But first he needs information from program designers, experts, and research.…

He has some money he’d like to earmark for your program….

He wants current web-based sources that are credible, accurate, reasonable, and well-supported.

Do you take the call?

Yes!

Today, we are going to learn how to evaluate Internet sources.

We’ll use the CARS method developed by Robert Harris

He has a great web site: http://www.virtualsalt.com/evalu8it.htm

Given a computer with Internet access, participants will locate and evaluate sources, such as program designers’ web site, program curriculum materials, experts, and research-based materials, so that 100% of resources are accurately evaluated in terms of how credible, accurate, reasonable and supported they are.

Logic Model Template

Inputs – resources (funds, staff, materials), activities, and strategies

as designed by program creator and experts and theorized in research (using credible, accurate, reasonable and supported sources)

based on implementation o using a method for

identifying, recording, maintaining and updating information on all program sites in Maryland

o using a tool for conducting site visits of programs in Maryland and observing and recording program activities in a manner that is objective and reliable

Outcomes – short-term, intermediate, and long-term changes in knowledge, attitudes, skills, and behaviors

as designed by program creator and experts and theorized in research

expected by program implementers (using interviews)

expected by key stakeholders (using surveys)

stated in grant applications

Relationship – a logical explanation of how the inputs lead to the outcomes

is there a direct cause-and-effect relationship?

what assumptions are implied?

will this relationship be true in any context?

what factors might confound this relationship?

Program Name and Information In order to identify inputs and outcomes, we need to start by locating appropriate

sources

In order to identify inputs and outcomes, we need

to start by locating appropriate sources

What word has become synonymous with web searches?

Hint: it is both a company name and a verb

Adj.: The feeling of being overwhelmed, disoriented, or confused as a result of finding millions of web sites in response to one simple little search request

Credibility

Accuracy

Reasonableness

Support

Is the source trustworthy?

Who is the author? (Is it a person or an organization?)

Who is the publisher?

What are the author’s credentials?

Is there evidence of quality control?

Is the source a known or respected authority?

Goal: an authoritative source, a source that supplies some good evidence that allows you to trust it.

Harris, R. (2007). Evaluating Internet Research Sources. http://www.virtualsalt.com/evalu8it.htm

Anonymity

Lack of Quality Control

Negative Metainformation. If all the reviews are critical, be careful.

Bad grammar or misspelled words


Up to date (within last 5 years)

Factual

Detailed, exact, comprehensive

Audience and purpose reflect intentions of completeness and accuracy

Goal: a source that is correct today (not yesterday), a source that gives the whole truth.


No date on the document

Vague or sweeping generalizations

Old date on information known to change rapidly

Very one-sided view that does not acknowledge opposing views or respond to them


Fair, balanced, objective

Reasoned, logical

No conflict of interest

Absence of fallacies or slanted tone

Goal: a source that engages the subject thoughtfully and reasonably, concerned with the truth.


Intemperate tone or language (“stupid jerks”)

Overclaims (“Thousands of children are murdered every day in the United States.”)

Sweeping statements of excessive significance (“This is the most important idea ever conceived!”)

Conflict of Interest (“Welcome to the Old Stogie Tobacco Company Home Page. To read our report, 'Cigarettes Make You Live Longer,' click here.”)


Listed sources

Contact information

Available corroboration

Claims supported

Documentation supplied

Goal: a source that provides convincing evidence for the claims made, a source you can triangulate (find at least two other sources that support it).


Numbers or statistics presented without an identified source for them

Absence of source documentation when the discussion clearly needs such documentation

You cannot find any other sources that present the same information or acknowledge that the same information exists (lack of corroboration)


Evaluating Sources Using the CARS Checklist (Harris, R. (2007). Evaluating Internet Research Sources. Retrieved April 20, 2009, from http://www.virtualsalt.com/evalu8it.htm)

Credibility trustworthy source, author’s credentials, evidence of quality control, known or respected authority, organizational support. Goal: an authoritative source, a source that supplies some good evidence that allows you to trust it.

Accuracy up to date, factual, detailed, exact, comprehensive, audience and purpose reflect intentions of completeness and accuracy. Goal: a source that is correct today (not yesterday), a source that gives the whole truth.

Reasonableness fair, balanced, objective, reasoned, no conflict of interest, absence of fallacies or slanted tone. Goal: a source that engages the subject thoughtfully and reasonably, concerned with the truth.

Support listed sources, contact information, available corroboration, claims supported, documentation supplied. Goal: a source that provides convincing evidence for the claims made, a source you can triangulate (find at least two other sources that support it).

Credibility Accuracy Reasonableness Support

Reference Author's Credentials

Evidence of Qual-ity Con-

trol

Meta-information

Timeli-ness

Comp-rehensiveness

Audience and

Purpose Fairness Objectivity

Moderate-ness

Consistency World View

Source Docu-menta-tion or Bib-

liogra-phy

Corroboration External

Consistency

ERIC provides unlimited access to more than 1.2 million bibliographic records of journal articles and other education-related materials, with hundreds of new records added twice weekly. If available, links to full text are included.

Education Resources Information Center http://www.eric.ed.gov

Sponsored by the Institute of Education Sciences (IES) of USDE

USDE. (2009). Education Resources Information Center (ERIC). http://www.eric.ed.gov

Within the ERIC Collection, you will find records for: journal articles books research syntheses conference papers technical reports policy papers other education-related materials

Relevance to education

Quality guidelines: Completeness, Integrity, Objectivity, Substantive Merit, Utility/Importance

Sponsorship by professional societies and organizations (national or international), and state or federal government agencies

Editorial / Peer-review Processes: Adherence to ethical guidelines, Fidelity to editorial conventions, Methods of selection, Procedure for retractions, Opportunity for comments/opinions, Publication history, Reprint availability, Scholarly review board, Selectivity

USDE. (2009). Education Resources Information Center (ERIC). http://www.eric.ed.gov

Lesson 2 Assessment

Locating Appropriate Resources for Identified Educational Programs

Context: In order to gather information for constructing your logic model, you need to consult

appropriate resources. You have learned how to identify such resources based on the CARS criteria. For

this task, you will apply this knowledge and carry out an actual search.

Task: Conduct a search for credible, accurate, reasonable and supported information about your

program. Include, as applicable, the program designers’ web site, program curriculum materials,

experts, and research-based materials. Use the “Evaluating Sources Using the CARS Checklist” activity

(Harris, 2007) to guide your search and your evaluation of the sources you find. Using the elements of

the CARS checklist, justify your determination that each of the sources is appropriate. Submit a written

justification for each source that demonstrates it is credible, accurate, reasonable and supported. It

may be useful to provide this information in a table format.

Rubric

Aspect Unacceptable Needs Revision Meets Standard Exemplary Written justification

A sufficient justification for all 4 areas (credible, accurate, reasonable and supported) is missing for most or all of the sources, OR the learner does not locate any information.

A sufficient justification for all 4 areas (credible, accurate, reasonable and supported) is missing for some of the sources.

A sufficient justification for all 4 areas (credible, accurate, reasonable and supported) is provided for all of the sources.

A sufficient justification for all 4 areas (credible, accurate, reasonable and supported) is provided for all of the sources, and the sources represent a thorough and wide-ranging variety of materials from program designers, experts, and research.

Complete the Lesson 2 assessment

You’ll apply the CARS evaluation method using the CARS table

You only need to do 3 sources

Email completed assessment to instructor

There is no deadline, but do it soon so you can get timely feedback

Prepare a one-page handout for local school system program coordinators that lists resources, save as a document with links, and post to Kazoo - \\Kazoo\dosss\General_Division Information\Logic Models\Lesson 2 - Resources.

Reflect in your journal about the resources you’ve found and share your reflections in the discussion on Kazoo - \\Kazoo\dosss\General_Division Information\Logic Models\Reflections.

Lesson 3: Discerning Key Information on Inputs from Program Design

Thurs. 7/2/09

1:00 – 2:00 p.m.

Room 839 – Computer Training Room

Documents

Instructor: Bess A. Rose What examples of logic models have you found since the last class? Where did you find them?