Publishing in Measurement Journals Journals as People (Not Just Outlets), Publishing as a Process (Not Just an Event) Presentation for EDMS MSMS Steve

Publishing in Measurement Journals

Journals as People (Not Just Outlets), Publishing as a Process (Not Just an Event)

Presentation for EDMS MSMSSteve Ferrara

American Institutes for ResearchDecember 4, 2006

Measurement Journals 2

Overview for this talk

A story about an article in EMIP Some requested themes

What some editors look for Review process Responding to review feedback Discerning the real meaning of an editorial

decision letter


Educational Measurement: Issues and Practice (EMIP) Aimed at practitioners and users of tests, as well as

professional educators, legislators, school personnel, and interested citizens. Its primary purpose is to promote a better understanding of educational measurement and to encourage reasoned debate on current issues of practical importance to educators and the public. EM:IP also provides one means of communication among NCME members and between NCME members and others concerned with educational measurement issues and practices.

http://www.ncme.org/


Measurement journals and other excellent outlets EMIP Journal of Educational Measurement Applied Measurement in Education Educational Assessment Educational and Psychological Measurement Applied Psychological Measurement Objective Measurement Measurement: Interdisciplinary Research and Perspectives International Journal of Testing Measurement and Evaluation in Counseling and Development Language Testing Educational Leadership (e.g., theme issue on assessment November 2005) Phi Delta Kappan NEA Today Journal of Applied Testing Technology Practical Assessment, Research and Evaluation Content area journals (e.g., JRST, Reading Teacher)


History of this article: late 2003

“Findings…are quite important…study seems to be well designed and executed.”

“However, my position is that the current version of this paper is not suitable for publication in EMIP. In general, the paper…For that reason, I will not send out the paper for review.”

“You may want to consider revising the current paper and resubmitting it to EMIP. If you do, please include a discussion of how you have addressed each of these concerns. I will be pleased to send out a revised paper for review.”


History: early 2004

“The reviewers and I concur that this paper is appropriate for publication in EMIP, once you respond to a small number of issues. These issues focus on providing clarity and explicitness for the wide range of EMIP readers.”

“If you choose to revise and resubmit the current paper, please include a discussion of how you have addressed each of the reviewers’ comments and mine.”


History: summer 2005

Appeared in summer issue of EMIP: 24 (2) An excellent paper


Review and decision process

Decisions by people who follow guidelines and make judgments using explicit and implicit standards

Decide whether to have it reviewed Reviewer comments to author and

recommendation to the editor Weigh the recommendations, make a

decision Write the editorial decision letter


Responding to review feedback

Respond to every single comment that is made Treat every single comment as advice

You’ve already put yourself on the line by submitting Get past defensiveness, outrage at the unfairness of it all,

etc.

Treat the process as an opportunity Your professional development Improve your paper Improve your future work


Discerning the real meaning of an editorial decision letter I have read your paper, * (SF-), submitted for consideration for

publication in EMIP. I also have read and considered comments on the paper from three reviewers. You have written an interesting/thought-provoking/compelling paper, and I appreciated the opportunity to read it.

The reviewers and I concur that the current version of the paper is not appropriate for publication in EMIP.

I offer the following comments for your consideration. I have enclosed for your consideration comments from the three

reviewers. If you choose to revise and resubmit the current paper, please include a discussion of how you have addressed each of their comments and mine.

Or: You may want to consider other outlets for your work. I have chosen not to send your paper out for review.


Discerning the real meaning (cont.)

The editor does not want to discourage submissions (or may want to) EMIP: 25-30 unsolicited submissions a year to fill

~16 slots JEM: 100+ unsolicited submissions a year

Editors want to treat colleagues fairly and not make enemies

Editors want to fill out the journal with good and compelling articles


What editors may look for Want to influence the field

Need to feel confident about decisions Want to be collegial and professional

Need to uphold the standards of the journal and the field Don’t want to be embarrassed or make an error

Care and caution Fairness, quality

Issues, practices, methods to target Hot topics in the field Recent papers New ideas Editor’s publication record Editorial Advisory Board


Feel free to contact me:

Steve Ferrara

Managing Research Director

Educational Assessment Program

American Institutes for Research

202-403-5431

[email protected]

Documents

Publishing in Measurement Journals Journals as People (Not Just Outlets), Publishing as a Process (Not Just an Event) Presentation for EDMS MSMS Steve