Download pdf - Career Progress in Online and Blended Learning Environments

Original Article

Career Progress in Online and BlendedLearning Environments

Melissa DeRosier, Ph.D., Rebecca Kameny, Ph.D., Wendy Holler, M.A.

Naomi Ornstein Davis, Ph.D., Emily Maschauer, M.S.

Objective: The authors examined the career achievement ofearly- and mid-career researchers in social, behavioral, andmental health who participated in a career-developmentconference.

Method: Trainees participated in a career-development conferenceeither through attending a live conference supplemented with anonline version of the conference (Combined: N=46) or through theonline version of the conference alone (Web-Only: N=60). Anobjective measure tracked the trainees’ publications, involvement inresearch projects, honors and grant awards, collaborations, andscientific presentations before and 9 months after participation inthe career-development conference.

Results: Statistical analysis showed that trainees improved foreach category measured, with no significant differences acrossthe Combined and Web-Only groups. The strongest variableaffecting improvement was Time, and the most significant timeeffect was seen in the production of presentations and publications.A significant Gender difference was present, with women showinggreater total career progress than men.

Conclusion: Career-development conferences can supportcareer growth for trainees. Online training provides a cost-effective and time-efficient alternative to in-person methods,while still enhancing key markers of career progress.

Academic Psychiatry 2013; 37:98–103

Training and supporting the pool of mental health re-searchers required for a thriving research community

is difficult in part because of the length of time between ed-ucation and research career independence (1). For example,the average age for receiving a National Institutes of Healthindependent research grant was 36 in 1981, but 42 in 2010(2). During this time, financial and professional pressurescan drive talented individuals out of research and into otherfields (3).Career-development conferences contribute to the reten-

tion of high-quality researchers by providing informationspecific to a research career, such as the grant submissionprocess, as well as information generally applicable to allearly-career professionals, such as finding a work–life bal-ance (4). Conferences can also offer “survival skills” train-ing by providing techniques for and examples of successfulresearch career skills such as writing and negotiation strat-egies (5). Also, programs can facilitate the development ofmentoring skills and relationships in a context where bothsenior and junior scholars can benefit (6, 7).

Career-Development Conferences Online

Although career-development conferences provide anestablished forum for continuing education, the costs ofattending conferences in person can be prohibitive (8–10).Online conference dissemination may be a viable methodfor extending the value of career-development conferences.Conference websites, for example, can reduce registrationand travel costs and increase international exposure forconference materials and ideas (11). Online conferencematerial also allows researchers to fit educational time intoa busy schedule and supports different learning preferences(12).Previous work has shown that conference materials

adapted for online training can yield similar knowledgegains when compared with in-person training (13). These

Received July 20, 2011; revised September 21, 2011, January 26, 2012;accepted February 15, 2012. From the 3-C Institute for Social Develop-ment, Cary, NC. Send correspondence to Dr. Melissa DeRosier; e-mail:[email protected] © 2013 Academic Psychiatry

98 http://ap.psychiatryonline.org Academic Psychiatry, 37:2, March-April 2013

findings are consistent with the “no significant difference”theory, which holds that qualities other than the trainingmedium drive success for online education (14). In otherwords, online and in-person instruction are generally botheffective for conveying knowledge when the same in-struction methods are used (see references 15–18 forreviews).

Career-Advancement Skills

Three objectively measurable research skills are wellrecognized as indicators of scientific development: runningsuccessful research projects, writing publications, and se-curing honors or grant awards. Several of the major career-development institutes explicitly list writing and earninggrants in their mission statements (19–21). Furthermore,grants completed, papers published, and honors received areall used to measure excellence of scientific contributionsduring tenure decisions (22–24).

Scientific collaborations and presentations also providebeneficial experience to early-career researchers. Collabo-rations can be formal or informal and range from brain-storming to results- or expertise-sharing (25). Sharing dataand experiences is particularly useful because it can in-fluence scientific perspectives and challenge assumptions,especially at international levels (26). Collaboration mayalso improve researchers’ chances for publication (27),a critical yet often challenging accomplishment for early-career researchers. Finally, scientific presentations empha-size both the cognitive and the social elements of careerdevelopment (28) by offering professional and networkingbenefits for the individual researcher as well as providinga forum for disseminating scientific findings.

The current study examines online and in-person career-development training and the career advancement ofattendees. Data are presented from 106 trainees who par-ticipated in three different career-development institutes:the 2008 Career Development Institute (CDI) for BipolarDisorder, the 2008 CDI for Psychiatry, and the 2008Leadership Training Institute (LTI). Thework contributes tothe question of how online and in-person research trainingcan positively affect career change.

Method

ParticipantsParticipants in the current study included 106 trainees

who participated in one of three career-development insti-tutes, each of which took place during the spring of 2008:

CDI for Bipolar Disorder (N=41), CDI for Psychiatry(N=41), and LTI (N=24). All participants were either early-career or mid-career researchers. Of the 106 participants,75% percent of the participants were early-career, and 25%were mid-career. “Early-career” was defined as havingobtained a terminal degree, having fewer than 10 peer-reviewed publications, and either having or seeking full-time employment in a research career (e.g., applying fora K award; first-year assistant professor status). Mid-careerwas defined as having obtained a terminal degree, havinga broader range and higher number of peer-reviewed pub-lications than early-career individuals, possibly having re-ceived some research funding but not yet being fullyindependent, and having some experience in a researchcareer (e.g., receiving an R03 award; associate professorstatus).

The sample of trainees included 66% women and 34%men with a self-identified racial/ethnic distribution of 71%White, 15% Asian, and 14% Black, with 14% of the totalsample reporting being of Hispanic descent. Chi-squareanalyses showed no significant differences across thethree conferences by gender, career level, or ethnicity.However, significant differences in race were found (x2[4]=35.30; p ,0.0001) where more Black (50% for the LTIversus 4% for the CDIs) and fewer White trainees (33% forthe LTI versus 81% for the CDIs) participated in the LTIthan in the other two conferences. This difference reflectsthe fact that the LTI was designed specifically for womenand minority researchers.

ProceduresThis study received expedited Institutional ReviewBoard

approval. Conference organizers solicited applications fortheir conference via national researcher list-servs, e-mail noti-fications to academic departments in relevant fields across thecountry, and advertisements in relevant professional organiza-tion publications. Each application required completion of anonline form, submission of a CV, a statement of research ex-perience and/or interests, and at least one letter of recommen-dation.More than 285 researchers applied for the three trainingprograms, and attendees were competitively selected by theconference organizers, with a selection rate of approximately50% of applicants.

Participants for this study were recruited from the list ofconference attendees, aswell as recommendations of conferencefaculty and direct e-mails to early- and mid-career researcherswhosepublicationsmatched the conferences’ respective areas ofspecialty. Study participants filled out an online or hardcopyconsent form before completing pre-conference questionnaires.

Academic Psychiatry, 37:2, March-April 2013 http://ap.psychiatryonline.org 99

DEROSIER ET AL.

Participants had access to the career-development conferencesthrough one of two conditions: 1) combination of in-person andonline training (Combined: N=46), where participants attendedthe institute and also had subsequent access to the conferencematerials online; and 2)Web-Only training (Web-Only: N=60),where trainees had access to the conference materials-onlyonline. Chi-square analysis showed no differences across theCombined and Web-Only conditions by conference, gender,race, Hispanic status, or career-level.To maintain a high level of quality for the online com-

ponent, live conference presentations were videotaped andadapted for online display (12). The online presentationsprovided viewing options that appealed to different learningpreferences (e.g., full video presentation with transcript,slides and transcript only, transcript-alone). The mp3 audiofiles and slides for each presentation were also available fordownload. An online bulletin board, a downloads area,a links page, and faculty bio pages were also provided foreach conference.

MeasuresParticipants completed a Career Progress Survey before

and 9 months after participation in their respective confer-ence. This timeframe represented approximately 1 year afteracceptance into the study and completion of the pre-conference Career Progress Survey. This survey assesseddifferent aspects of research career achievement and expe-riences, including current position, honors and awards,publications, career-development activities (e.g., seminars,training institutes), funding, and collaboration. Detailedinformation was collected to increase accuracy in reportedinformation (e.g., publication citations, specific grant titles,and presentation title information).

Results

Because of time and scheduling issues, 24 traineesdropped out over the course of the study (23% attrition rate).

Chi-square analyses showed no evidence of selective at-trition. Attrition analysis included initial level of careerachievements, conference, study condition, gender, race,Hispanic status, or career level, and there were no significantdifferences between trainees who exited the study versusthose who remained for any of these categories.In the first set of analyses, the five key markers of career

progress were examined: 1) number of publications (in-cluding peer-reviewed journal articles, book chapters, trademagazine articles, and abstracts); 2) number of grants onwhich the participant served as PI or Co-PI; 3) number ofhonors or awards; 4) number of current research collabo-rators; and 5) number of presentations. Each variable wascollected at pre-assessment and again at a 9-month follow-up. A multivariate analysis of variance (MANOVA)across the five variables was run, with study Condition(Combined versus Web-Only) as the between-subjectsvariable and Time (pre-conference versus 9 months later)as the within-subject repeated factor. A main effect forConference was significant at the multivariate level (F[20,134]=2.13; p,0.05); therefore, Conferencewas included asa covariate and controlled for in all subsequent analyses.Significant multivariate main effects for study Condition

were found both at the pre-conference timepoint (F[5,98])=2.91; p ,0.05) and at 9-month follow-up (F[5, 71]=2.67; p,0.05; see Table 1). Univariate analyses revealedthat participants in the Combined condition showed a sig-nificantly larger number of honors at pre-conference thandid Web-Only participants (F[1, 102]=9.94; p ,0.01), andparticipants in the Web-Only condition showed a trend to-ward higher number of collaborators at pre-conference thandid the Combined participants (F[1, 102]=2.93; p,0.1).However, at 9-month follow-up, only number of honorscontinued to show this difference across study conditions(F[1, 75]=10.26; p ,0.1). There were no significant multi-variate interaction effects at either timepoint as a function ofConference, study Condition, Gender, Race, Hispanic sta-tus, or Career level.

TABLE 1. Least-Squares Means by Condition at Pre-Conference and 9-Month Timepoints

Publications PI on Grant Honors Collaborations Presentations

Pre- 9Months Pre- 9Months Pre- 9 Months Pre- 9Months Pre- 9Months

Combined(SD)

4.02(0.73)

5.72(1.03)

1.09(0.23)

1.32(0.32)

1.50A

(0.18)1.98A

(0.30)2.43B

(0.67)3.25(0.95)

1.72(0.22)

2.53(0.43)

Web-Only(SD)

4.99(0.65)

5.82(0.89)

1.17(0.20)

1.65(0.28)

0.75B

(0.16)0.76B

(0.26)3.95A

(0.60)4.90(0.82)

1.95(0.20)

2.52(0.37)

Means with different superscript letters within a column were significantly different from one another across study conditions.PI: principal investigator; SD: standard deviation.


ONLINE AND BLENDED LEARNING

Examination of the repeated-measures MANOVA in-dicated a significant multivariate effect for change over timefrom pre-conference to 9 months (F[9, 675]=18.00; p,0.0001). The Time effect did not vary by Condition.Univariate analyses revealed that this effect held for four ofthe five career-progress variables: number of publications(F[1, 75]=9.36; p,0.01), number of grants (F[1, 75]=6.95;p ,0.01), number of research collaborations (F[1, 75]=3.95; p ,0.05), and number of scientific presentations (F[1, 75]=11.11; p ,0.01). Each marker of career progresssignificantly increased over time. Only the number of hon-ors received did not show significant change over time.Also, there was a significant multivariate interaction effectfor change over time by career level (F[9, 657]=1.90; p,0.05). The change in number of research collaborationsvaried for early-career versus mid-career level researchers(F[1, 73]=9.80; p,0.01). Specifically, least-squares meansshowed greater improvement for early-career individuals,from a total of 1.15 (SD: 0.67) at pre-conference to 4.15(SD: 0.74) at 9 months, whereas mid-career researchersmoved from a total of 3.92 (SD: 1.13) at pre-conference to4.35 (SD: 1.26) at 9 months.

In a second analysis, a composite variable representingTotal Career Progress was created based on demonstratedchange in each career progress area. For each participant,a Total Career Progress score was calculated by allotting 1point for each keymarker that was greater at 9months than itwas at pre-conference, and an additional point was given forhaving a higher salary or a new job at follow-up. Descriptiveanalyses revealed that 57% of participants had an increase insalary, and 15% of Combined and 13% of Web-Onlytrainees had obtained a new position by the end of the study.

An analysis of variance (ANOVA) with study Conditionas the between-subjects factor indicated no significant dif-ference in Total Career Progress by Condition (CombinedM=1.98; SD: 0.26; Web-Only M=2.03; SD: 0.23). How-ever, a significant difference by Gender was found (F[1,100]=8.12; p,0.01) where women showed greater overallcareer progress over time than did men (women: M=2.37,SD: 0.21; men: M=1.33, SD: 0.30). No other main or in-teraction effects were observed.

Discussion

This study examined the career achievement of train-ees before and 9 months after participation in a career-development conference. Trainees in social, behavioral,and mental health fields of research showed significantimprovement in key career progress markers, with the

number of presentations and publications showing thegreatest degree of expansion. For Total Career Progress,female researchers showed the greatest overall careergrowth. Although the number of women graduating withterminal degrees has increased over time, a commensurateincrease in leadership and faculty positions held by womenhas not yet been seen (29). The findings suggest that womenwith access to a career-development conference may showmore proactive career behaviors than their male counter-parts and thereby be in a stronger position for careeradvancement. This study supports the promise of career-development programs, such as the CDIs and LTI, forsupporting the career trajectory of female researchers inparticular.

Similar to previous studies (15–17), no significant dif-ferences in career progress over timewere evident across thetwo study conditions, as measured through either individualmarkers or the Total Career Progress index. Thus, thecombined in-person-plus-web-training group and the web-only training group each showed significant career progressregardless of how trainees participated in the conferences.This finding is congruent with previous work showing “nosignificant difference” in the benefits of online versus in-person training for knowledge acquisition (14) and extendsthis argument to suggest equal benefits with regard to career-enhancing skills and experiences. Although more work isneeded regarding the degree to which career-developmentconferences directly contribute to improvements in thesekey career progress markers, this study suggests that onlineconferences offer equally potent career-enhancing oppor-tunities for researchers. As has been found in past researchcomparing different training modalities, benefit for traineesis most likely determined by the quality of the underlyingtraining materials, and not the particular training modality(14, 15).

Limitations of this study include its small sample sizeand the lack of a control condition that did not receiveany training. Thus, the degree to which trainee career-development can be attributed to conference participationis limited. Questions remain about the ways in which timespent in various conference participation methods and in-dividual initiative help produce successful career practices.Future research is needed with a larger set of trainees inorder to explore how these factors interact with one anotherto determine career progress for researchers over time. Also,this study’s timeframe was limited to approximately 1 yearbetween pre-conference and follow-up assessment, which isa relatively short period in which to see substantial careergrowth. Whereas significant changes in key markers of


DEROSIER ET AL.

career success were evident in this study, it is important thatfuture research examine the longer-term impact of career-development conferences for career achievement overmultiple years. In particular, given that online conferencesoffer fewer, or at least different, opportunities to obtain andengage with mentors, future studies should examinewhether the impact of online educational programs overa longer career span differs from that of in-person programs.In this difficult funding climate, when training programs

face increased demands for justification, the measure ofcareer advancement is particularly significant. Althougha conference cannot be the sole form of support for trainees,this study bolsters the argument that career-developmentinstitutes can support significant positive change in careerprogress markers for trainees. In particular, online career-development conferences can provide affordable, feasibletraining opportunities to support the retention and career-enhancement of researchers on a broad scale. Given thatonline training modalities offer lower participation barriersand greater cost-effectiveness than in-person methods, thisstudy underscores the potential value of offering high-quality, web-based career-development conferences as aviable means to increase access to career-development train-ing opportunities for researchers.

The authors thank the faculty and trainees of the Career Devel-opment Institutes and the Leadership Training Institute, withoutwhom none of this research would have been possible. In particu-lar, Dr. David Kupfer’s guidance and commitment to researchtraining are greatly appreciated.

This research was funded in part by the National Institute of Men-tal Health (contractsHHSN271200774104C,HHSN278200553104C,and HHSN271200774100C).

References

1. National Research Council: Bridges to Independence: Fos-tering the Independence of New Investigators in BiomedicalResearch.Washington, D.C., National Academies Press, 2005

2. Collins F: Scientists need a shorter path to research freedom.Nature 2010; 467:635

3. JohnsonMO,SubakLL,BrownJS, et al:An innovative programto train health sciences researchers to be effective clinical andtranslational research mentors. Acad Med 2010; 85:484–489

4. Halpain MC, Jeste DV, Katz IR, et al: Summer Research In-stitute: enhancing research career-development in geriatricpsychiatry. Acad Psychiatry 2001; 25:48–56

5. Kupfer DJ, Schatzberg AF, Grochocinski VJ, et al: The CareerDevelopment Institute for Psychiatry: an innovative, longitu-dinal program for physician-scientists. Acad Psychiatry 2009;33:313–318

6. Jeste DV, Halpain MC, Trinidad GI, et al: UCSD’s short-termresearch training programs for trainees at different levels ofcareer development. Acad Psychiatry 2007; 31:160–167

7. Bartels SJ, Lebowitz BD, Reynolds CF III, et al: Programs fordeveloping the pipeline of early-career geriatric mental healthresearchers: outcomes and implications for other fields. AcadMed 2010; 85:26–35

8. Green M: Are international medical conferences an outdatedluxury the planet can’t afford? Yes. BMJ 2008; 336:1466

9. Hall E: Alternative future for academic conference: a responseto Bonnett. Area 2007; 39:125–129

10. Mair J, Thompson K: The UK association conference atten-dance decision-making process. Tour Manage 2009; 30:400–409

11. Thatcher A: Building and maintaining an online academicconference series. Int J Ind Ergon 2006; 36:1081–1088

12. Kupfer DJ: Why disseminate via the Web? 2009; retrievedfrom http://www.4researchers.org/articles/4459

13. DeRosier ME, McMillen J, Davis NO, et al: Tools to supportcareer advancement of diverse social, behavioral, and mentalhealth researchers: comparison of in-person and online train-ing delivery modes. J Online Learning Teaching 2011; 7

14. Russell TL: The No Significant Difference Phenomenon.Chapel Hill, NC, Office of Instructional Telecommunication,North Carolina State University Press, 1999

15. Cook DA, Levinson AJ, Garside S, et al: Internet-basedlearning in the health professions: a meta-analysis. JAMA2008; 300:1181–1196

16. Wofford MM, Spickard AW III, Wofford JL: The computer-based lecture. J Gen Intern Med 2001; 16:464–467

17. Chumley-Jones HS, Dobbie A, Alford CL: Web-basedlearning: sound educational method or hype? a review ofthe evaluation literature. Acad Med 2002; 77(Suppl):S86–S93

18. Sitzman T, Kraiger K, Stewart D, et al: The comparative ef-fectiveness of web-based analysis and classroom instruction:a meta-analysis. Pers Psychol 2006; 59:623–664

19. Advanced Research Institute, 2011: ARI overview; retrievedfrom http://www.mentalhealthtrainingnetwork.org/institutes/ari/overview

20. Career Development Institute for Bipolar Disorder, 2011; re-trieved from http://www.cdibipolar.org/

21. Career Development Institute for Psychiatry, 2011; retrievedfrom http://www.cdipsych.org/

22. Austin Z, Gregory PAM: Promotion and tenure: clinical facultyat schools of pharmacy in Canada. Pharm Educ 2006; 6:267–274

23. Balogun JA, Sloan PE, Germain M: Core values and evalua-tion processes associated with academic tenure. Percept MotSkills 2007; 104:1107–1115

24. McCormick CB, Barnes BJ: Getting started in academia:a guide for educational psychologists. Educ Psychol Rev2008; 20:5–18

25. National Institutes of Health, 2010: Report of Trans-NationalInstitutes of Health Research Conducted in Fiscal Year 2010;retrieved from http://dpcpsi.nih.gov/pdf/2010_Report_of_Trans-NIH_Research.pdf


ONLINE AND BLENDED LEARNING

26. Freshwater D, Sherwood G, Drury V: International researchcollaboration: issues, benefits, and challenges of the globalnetwork. J Res Nurs 2006; 11:295–303

27. Lei SA, Chuang N: Undergraduate research assistantship:a comparison of benefits and costs from faculty and students’perspectives. Education 2009; 130:232–240

28. deVries B, Pieters J: Knowledge-sharing at conferences. EducRes Eval 2007; 13:237–247

29. Brown AJ, Swinyard W, Ogle J: Women in academic med-icine: a report of focus groups and questionnaires, withconjoint analysis. J Womens Health (Larchmt) 2003; 12:999–1008

Why I Like Being an Academic Psychiatrist

Philip R. Muskin, M.D.

I decided I wanted to become a psychiatrist when I was 11 years old. At that time, I had no idea whatpsychiatrists really did, but it seemed like a terrific idea, and helping others felt like a good way to spendone’s life. I most certainly had no idea what an academic psychiatrist did.

For many, an academic is someone who does research, publishes research, and may lecture about that re-search. In some institutions, the only people who really are considered “academics” are those who bring ingrant money to do research, and who publish in peer-reviewed journals; education is important, but notnecessarily the task of such academics. I view being an academic psychiatrist as having the responsibility tolearn a variety of different things in order to be able to educate others. This means that just knowing a lot, orjust seeing patients, or just doing research is not sufficient to be an academic psychiatrist. It means that oneneeds to try to become as learned as possible in a variety of different areas in order to help others educatethemselves. The goal of the education is to enable others to provide the highest quality of clinical care topatients. This is a daunting task. It creates a constant struggle to keep learning, and a constant sense of doubtthat I don’t know enough.

Itmakes for a longday.Eachday isfilledwith seeingpatients, for “book-learning”has little value unless oneapplies it actively. It is only in the clinical encounter that one really learns how to modify “facts” in the realworld of people in need of treatment. Each day contains time supervising, be it answering questions, seeingpatients with residents, struggling with the many ethical and legal issues that confront us regularly, and“formal” teaching, such as lectures. Reading andwriting are typically not confined to “work hours,” as thereis not enough time to do everything at work.

Don’t misunderstand me; there are many evenings that I watch television, particularly during baseballseason. There are also many evenings and weekends during which I read and write. A key element of beingan academic psychiatrist, in my view, is that I enjoywhat I do. Thus, “working” in the evening is not painful.I did not plan to have this type of career. I came to medicine to learn how to help others. I chose psychiatrybecause I felt it was a career where the element of therapy was how I could understand the suffering ofanother person, to enable that person to understand how to change, be it via psychotherapy or psychophar-macology. Academic psychiatry allowedme to findmy true self, not by design, but by followingmy naturalinclinations. I did not know that I had an ability to educate or how enjoyable it was to learn in order to sharethat knowledge with other people. I am thankful that I work in a place that still supports such endeavors.

Dr. Muskin is with the Dept. of Consultation–Liaison Psychiatry, Columbia University Medical Center,New York, NY.


DEROSIER ET AL.