24
in this issue we offer the Officers of the Faculty, “MIT Undergraduate Education at a Crossroads,” (page 4); “Naming the MIT Intelligence Quest,” (page 9); “Nuclear Weapons Education Project,” (page 10); our Teach Talk feature, “MIT Students and Deep Learning: Perspectives and Suggestions,” (page 12); and MIT Day of Action (page 20). MIT Faculty Newsletter Vol. XXX No. 4 March/April 2018 http://web.mit.edu/fnl Massachusetts Institute of Technology continued on page 22 The Erosion of Social Norms Guiding the Government- University Relationship Improving the Urgent Care Experience Through Student-Informed Care continued on page 6 THIS PATRIOTS’ DAY MARKS 243 years since farmers, tradesmen, mer- chants, and sailors fought a bloody war to throw off the yoke of King George and the British monarchy. They established a republic in the place of the monarchy, and sought to make the government account- able to the people. In the years since, the people of most of the Earth’s nations have followed suit and rid their nations of monarchical governments. Among the absolute monarchies still in power, the most egre- gious is that of the Saud family of Saudi Arabia. We are appalled that the MIT administration agreed to meet and pre- sumably negotiated programs with the Saudi Crown Prince, Mohamed Bin Salman (MBS). This was done without any consultation with faculty or student organizations. Editorial MIT Should Not Be Supporting the Saud Monarchy continued on page 3 March 23 Protest at MIT Cecilia Stuopis ARE WE WITNESSING A MAJOR shift in the relationship between the federal government and universities? To speed up the industrialization of the nation in the nineteenth century, the U.S. government provided free land and revenue to start land grant colleges. In the twentieth century government-sponsored research rose sharply, especially after Sputnik. Yet now the federal government proposes taxes on university endowment earnings and grad- uate students’ tuition stipends and wants to dictate which international students uni- versities can admit! Is this a fundamental change in the norms which guided the gov- ernment-university relationship in the past, or only a temporary setback created by a particularly anti-intellectual adminis- tration that dislikes academia, distrusts sci- entific research, and sees universities as dominated by a liberal elite? Bish Sanyal LIKE ALL DEPARTMENTS, LABS, and centers, MIT Medical is always looking for ways to evolve and improve. Whether it’s focusing closer on the cus- tomer experience, being more fiscally responsible, or turning our attention toward planning for the future, we are constantly striving to be our best. In that spirit, in 2016, MIT Medical commissioned a patient journey map that used data-driven methodology to give us a holistic view of how our patients experi- ence care at MIT Medical. This involved analyzing more than 1,000 comments from post-visit surveys, interviewing staff, and shadowing patients. In the end, our journey map (see page 22) identifies every touchpoint within an episode of care and highlights touchpoints where we’re exceeding patient expectations as well as opportunities for improvement.

MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

in this issue we offer the Officers of the Faculty, “MIT UndergraduateEducation at a Crossroads,” (page 4); “Naming the MIT Intelligence Quest,” (page 9);“Nuclear Weapons Education Project,” (page 10); our Teach Talk feature, “MITStudents and Deep Learning: Perspectives and Suggestions,” (page 12); and MITDay of Action (page 20).

MITFacultyNewsletter

Vol. XXX No. 4March/April 2018

http://web.mit.edu/fnl

MassachusettsInstitute ofTechnology

continued on page 22

The Erosion of SocialNorms Guiding theGovernment-UniversityRelationship

Improving theUrgent CareExperience ThroughStudent-InformedCare

continued on page 6

TH I S PATR I OTS’ DAY MAR K S 243

years since farmers, tradesmen, mer-chants, and sailors fought a bloody war tothrow off the yoke of King George and theBritish monarchy. They established arepublic in the place of the monarchy, andsought to make the government account-able to the people. In the years since, the people of mostof the Earth’s nations have followed suitand rid their nations of monarchicalgovernments. Among the absolutemonarchies still in power, the most egre-gious is that of the Saud family of SaudiArabia. We are appalled that the MITadministration agreed to meet and pre-sumably negotiated programs with theSaudi Crown Prince, Mohamed BinSalman (MBS). This was done withoutany consultation with faculty or studentorganizations.

EditorialMIT Should Not BeSupporting the SaudMonarchy

continued on page 3

March 23 Protest at MIT

Cecilia Stuopis

A R E W E W I T N E S S I N G A M A J O R

shift in the relationship between the federalgovernment and universities? To speed upthe industrialization of the nation in thenineteenth century, the U.S. governmentprovided free land and revenue to start landgrant colleges. In the twentieth centurygovernment-sponsored research rosesharply, especially after Sputnik. Yet nowthe federal government proposes taxes onuniversity endowment earnings and grad-uate students’ tuition stipends and wants todictate which international students uni-versities can admit! Is this a fundamentalchange in the norms which guided the gov-ernment-university relationship in thepast, or only a temporary setback createdby a particularly anti-intellectual adminis-tration that dislikes academia, distrusts sci-entific research, and sees universities asdominated by a liberal elite?

Bish Sanyal

L I K E A L L D E PA R T M E N T S , L A B S ,

and centers, MIT Medical is alwayslooking for ways to evolve and improve.Whether it’s focusing closer on the cus-tomer experience, being more fiscallyresponsible, or turning our attentiontoward planning for the future, we areconstantly striving to be our best. In that spirit, in 2016, MIT Medicalcommissioned a patient journey map thatused data-driven methodology to give usa holistic view of how our patients experi-ence care at MIT Medical. This involvedanalyzing more than 1,000 commentsfrom post-visit surveys, interviewingstaff, and shadowing patients. In the end,our journey map (see page 22) identifiesevery touchpoint within an episode ofcare and highlights touchpoints wherewe’re exceeding patient expectations aswell as opportunities for improvement.

Page 2: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

2

Vol. XXX No. 4 March/April 2018

Aron BernsteinPhysics

Robert BerwickElectrical Engineering & Computer Science

*Manduhai BuyandelgerAnthropology

Nazli ChoucriPolitical Science

*Christopher CumminsChemistry

Woodie FlowersMechanical Engineering

Ernst G. FrankelMechanical Engineering

*Jonathan King (Chair) Biology

Helen Elaine LeeWriting and Humanistic Studies

Seth LloydMechanical Engineering

Fred MoavenzadehCivil & Environmental Engineering/Engineering Systems

*Ruth Perry (Vice Chair)Literature Section

*Nasser RabbatArchitecture

Patrick Henry WinstonElectrical Engineering & Computer Science

David LewisManaging Editor

*Editorial Subcommittee for this issue

AddressMIT Faculty NewsletterBldg. 10-335Cambridge, MA 02139

Websitehttp://web.mit.edu/fnl

Telephone 617-253-7303Fax 617-253-0458E-mail [email protected]

Subscriptions$15/year on campus$25/year off campus

01 Improving the Urgent Care Experience Through Student-Informed Care Cecilia Stuopis 01 The Erosion of Social Norms Guiding the Government-University Relationship Bish Sanyal

Editorial 01 MIT Should Not Be Supporting the Saud Monarchy

From The 04 MIT Undergraduate EducationFaculty Chair at a Crossroads Susan S. Silbey, Rick Danheiser, W. Craig Carter

09 Naming the MIT Intelligence Quest Kenneth R. Manning

10 Nuclear Weapons Education Project Luisa Kenausis, Aron Bernstein, Robert Redwine, Michael Hynes

Teach Talk 12 MIT Students and Deep Learning: Perspectives and Suggestions Dave Pritchard

19 Higher Ed in the Era of #MeToo: A Symposium for Faculty and Graduate Students Stacey Lantz 20 MIT Day of Action Roger Levy, Sally Haslanger, Ceasar McDowell for the MIT Day of Action Organizing Team

Letters 21 Suicide and Sexual Harassment at MIT Dan Stroock

M.I.T. Numbers 24 MIT Research Expenditures 1940–2017

contentsThe MIT FacultyNewsletterEditorial Board

Photo credit: Page1: David Lewis

Page 3: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

3

Saudi Arabia, in fact, remains anoppressive absolute monarchy, and thesource of great human suffering, mostnotably from their war on the people ofYemen. Bin Salman was on a U.S. tour,and earlier met with President Trump,who had approved the sale of $billionsworth of U.S. missiles and warplanes tothe Saudi government. The weapons areamong those used in the Saudi-led war onYemen that has left thousands of civiliansdead since 2015. The United States is alsoassisting the Saudi monarchy in the coali-tion’s targeting selection for aerial bomb-ings and actively providing midairrefueling for Saudi and United ArabEmirates jets that conduct indiscriminateairstrikes – the leading cause of civiliancasualties. Meanwhile, the Saudi coalitionis starving millions of Yemenis as agrotesque tactic of war. According to theUN, the blockade of Yemeni ports by theSaudi military has resulted in “the largestfamine the world has seen in decades”causing a massive cholera epidemic andleaving 400,000 children malnourished. Bin Salman’s extensive public relationscampaign directed at Americans haspainted him as a “reformer” who is sup-porting, for example, the rights of womento drive, while ignoring the continuinggeneral oppression of women in SaudiArabia, imprisoning hundreds withouttrial, and actively opposing democraticmovements in other Arab states.Cambridge City Councilor and MITalumnus Quinton Zondervan, speaking at

the demonstration against Bin Salman’svisit, stated clearly “We do not need toshow respect to an oppressor and a bullyand a warmonger.” U.S. CongressmenRho Khanna, Marc Pocan, and Walter

Jones (New York Times, October 10, 2017)have criticized our government for “par-ticipating in a military coalition led bySaudi Arabia and the United ArabEmirates in a brutal military campaign inYemen.” On April 2 the Cambridge CityCouncil went on record in opposition tothe oppressive policies of MBS and SaudiArabia, stated its disappointment at themanner in which the visit was hidden byHarvard and MIT, and requested thatcopies of the resolution be delivered to thePresidents of both MIT and Harvard aswell as MBS. The Tech reported on the meeting intheir April 5 issue, and published a cogentand critical editorial. Due to the complete lack of candor ofthe MIT administration, we don’t knowthe precise nature of the business betweenthe Crown Prince and MIT. According toGrif Peterson and Yarden Katz, ofHarvard’s Berkman Klein Center for

Internet & Society, writing in theGuardian (Friday, March 30, 2018) “. . . Bin Salman’s foundation, MiSK(https://misk.org.sa/en), was accepted as a‘member company’ to MIT’s Media Lab in

2017, which requires a minimum annualcontribution of $250,000 (with a three-year commitment) to the lab. In return,MiSK receives access to the lab’s person-nel, technology, and intellectual property.”Regardless of the content, MIT should notbe entering into an agreement with repre-sentatives of the Saudi regime. This recallsthe Shah of Iran’s effort to secure nuclearengineering graduate slots for the Shah’schosen candidates. This was eventuallyrejected due to opposition from thefaculty. At a minimum, President Reif shouldhave reported to the faculty on the visit ofBin Salman. He now needs to ensure thatMIT has not entered into any furtheragreements with the Saudi government orRoyal Family that ignore their record ofoppression, discrimination, and humanrights violations.

Editorial Subcommittee

MIT Should Not Support Saud Monarchycontinued from page 1

Bin Salman’s extensive public relations campaigndirected at Americans has painted him as a “reformer”who is supporting, for example, the rights of women todrive, while ignoring the continuing general oppressionof women in Saudi Arabia, imprisoning hundreds withouttrial, and actively opposing democratic movements inother Arab states.

Page 4: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

4

Susan S. SilbeyRick DanheiserW. Craig Carter

From The Faculty ChairMIT Undergraduate Education at a Crossroads

W H AT E V E R PAT H O N E T R AV E L S ,

Robert Frost taught us, we may come tothink over time that the choice was conse-quential, “made all the difference.” But, alongany selected path, one thing always leads toanother and different routes each haveattractions. As Frost was in the yellow wooda century ago, MIT education is nowunquestionably at a similar crossroads.Changes in technology, in the globaleconomy, in our democracy, in our students,and in vehicles for delivering education chal-lenge us to examine the path we have hewn.We can continue along the same route, or wecan change directions, very slightly or signif-icantly. The choices, however, are ours – staythe course or make adjustments. Althoughthe curriculum is iteratively shaped throughthe decisions we continually make in depart-ments and in faculty committees, the accu-mulated changes beg us to give more thanroutine attention at this moment to theshape and content of the MIT undergradu-ate degrees. If we are to provide responsibleguidance for our students now, and steward-ship for the Institute into the future, weought to consider the curricular paths weoffer and do so through actively engaged,intense deliberation. Your faculty officers are not alone in rec-ognizing the changes in the world and in ourstudents that have brought us to a crossroads.Consider the recent spate of new majors,experiments, and proposals including, forexample, Vice Chancellor Ian Waitz’s DesignClass on the First Year Experience, the NEET(New Engineering Education Transfor-mation) experiment, the First Year ProjectsInitiative, the proliferation of “flexiblemajors,” the review of whether ABET certifi-cation is necessary for engineering degrees,calls for new GIRs in computational thinking,statistics, as well as ethics and public policy.

To initiate a broad and deep, participa-tory and energetic, conversation about theMIT undergraduate program, we arehosting an all-day workshop on June 14.The workshop will explore the path wehave been following and the crossroads weface. Although we are orchestrating theworkshop, we do not have any preconcep-tions about what its outcomes will be:what is desirable in the way of changes andwhat is worth preserving. We are open toanything. We are proposing a conversa-tion, more specifically, a process, not aconclusion. We are encouraged in this effort by thethoughtful exchanges at recent facultymeetings as curricular proposals have beenbrought for open debate. We expect theworkshop to consider the basic compo-nents of the undergraduate academicprogram, including its structure or config-uration (i.e., the number of subjects andhours for a degree); the distribution ofcontent (i.e., subject matters, choices,options and flexibility); and, variations inpedagogy (i.e., vehicles for delivery, formsof student-faculty and student-to-studentinteractions). Although we wish to pre-scribe little beyond a process for delibera-tion, we hope the conversation willgenerate a variety of curricular paths thatwill initiate detailed discussion and explo-ration beyond the workshop itself. Our goal is to examine the currentundergraduate program critically from theperspective of what is best for our studentsin the twenty-first century. Their needs, noless than their desires, are markedly differ-ent from those of students 50 and 60 yearsago. Our current students encounter aworld more unsettled than it has been fordecades, a time of economic, political, andenvironmental uncertainty. Student anxi-

eties about the future are understandablymore intense than the usual angst youthexperience as they move from adolescenceto adulthood. Students voice apprehensionabout the future of the environment, aboutthe prospects of nuclear war, and certainlymost immediately about the prospects formeaningful and lucrative employment.They are worried, but they are also pas-sionate. They seek better advising andmentoring to explore options and seekflexibility to respond to both positive andnegative course experiences. They embraceMIT’s ambition to make a better world andsometimes recognize the limits of whatthey do not understand. They seek ourguidance. Students’ unease may not be expressedin quite the same ways as the faculty’s,although there are important conver-gences, some involving issues concerningthe HASS component of the GIRs.Faculty voice concerns about enrollmentsdriven by markets for jobs rather thaneducational experimentation and curios-ity, radically skewing the distribution ofstudents across subjects. While somefaculty worry about an excess of theoreti-cal knowledge and insufficient hands-onproject learning where innovationthrives, others are also distressed by theinsufficient attention to the consequencesof technological inventions. Students lackknowledge of history and social organi-zation, and so they keep bumping intothese unseen forces with which they aretoo often ill-equipped to recognize ormanage. This is a world of considerablymore knowledge than we had a century,even a half-century ago; employersexpect scientific and technological skillsin just about all workplaces, jobs MITstudents are especially well equipped to

Page 5: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

5

fill. Yet, at the same time, the need for aworkforce knowledgeable about publicpolicy and experienced in ethical deci-sion-making and responsible citizenshiphas never been greater. As the collectiveconsequences of single, perhaps individ-ual decisions propagate in a digitally con-nected world, our students, some facultysay, seem to display a noticeably reducedlack of commitment to the public collec-tive, preoccupied with individual auton-omy. The better world they imagine isoften one where each person should bean agent of entirely unconstrained desire. Turning to the science/math/engineer-ing component of the GIRs, recent discus-sions of a computational thinkingrequirement have triggered a general dis-cussion of the SME GIRs. While there is noconsensus about how to move forward, fewfaculty members disagree that computa-tional thinking is now an essential tool inan educated person’s toolbox (althoughthere are more than a few disagreementson the definition of computational think-ing). But if we, and colleagues elsewhere,agree that this is important, how manyfaculty would agree that the addition ofthat tool is wiser than other possibilities?Across the faculty we have heard com-pelling arguments that probability and sta-tistics, ethics, history, and public policy areequally valid as General InstituteRequirements. Repeatedly, past reviews have identi-fied three aims for the GIR component ofthe MIT undergraduate curriculum.First, the SME GIRs provide our studentswith foundational building blocks – acommon body of knowledge that depart-ments can then assume in teachingadvanced subjects. Second, the GIRsconfer basic literacy in essential fields byproviding substantive knowledge in areaswith which every MIT graduate shouldbe familiar. Finally, the GIRs introducestudents to methods for creative analyticalthinking by equipping students withportable tools and strategies for problemsolving applicable to a variety of differentkinds of knowledge and thought. Are these aims still valid as goals for theGeneral Institute Requirements? How well

do the current offerings in physics, biology,chemistry, math, and the laboratory andREST subjects fulfill these objectives? Arethere other subjects that ought to beincluded as SME requirements? As substi-tutes for existing GIRs, or as alternatives?Are HASS GIRs working as planned? Canthe first-year curriculum be revitalized bythe inclusion of more hands-on and projectexperiences? Should we incorporate moreflexibility in the GIRs, empowering stu-dents to explore more diverse subjects earlyon? These are all questions that we hope toaddress in the June workshop. Our students are graduating into a

world with a society – and its science andtechnology – that would be unrecognizableto those who conceived the current config-uration of the MIT degree. If we agree thatour existing curriculum ought to providethe foundation to be a responsible citizen inthis brave new world, then we should eval-uate its design and effectiveness to assurethat we achieve its goals. Real educationcertainly isn’t easy, and perhaps it shouldn’tbe. Focused mental exercise is the gift wehope to give our students. Facts (or thosebits and bytes of information representedto be facts) are now a cheap commodity.Determining fact from fiction is a skill wehope to teach. Alumni who discover andanalyze facts are what we hope to educate.The inevitable tension between securingthe immediate-benefit of specialization andthe long-term rewards of general-back-ground education are a continuing chal-lenge for higher education. Moreconcretely, while a specific degree may helpa student find a particular job, it may nothelp them progress in that job. From thisviewpoint, MIT’s fundamental education –experienced as the ability to learn newthings – will likely provide the most tangi-ble and rewarding future benefits. If MIT is the world’s premier technolog-ical institution, this status comes with greatresponsibility. Perhaps we should considerour self-reflective inquiry not as just onemore university taking a hard look at itself,and not just an organization’s response toconsumer demands, but as an opportunityto set a new standard for scientific and tech-nological literacy. If we cannot avoid the

significance of this tumultuous historicalmoment when history seems to haveturned a corner, we certainly need to look atour role in its emergence. How did we gethere, nationally and locally, and at MIT? Isit too self-important to think that some-thing about our education has fed, not onlyamong our own graduates but the public atlarge, an addiction to tweeting and anony-mous surveillance, fake news and wide-spread bullying, all in the name ofautonomy and connection? With the June 14 workshop, we hope tomove beyond our familiar and otherwisewell-functioning committee processes. Wewant to engage the faculty’s collectivewisdom and experience to think togetherabout the future of our students and whatwell-educated alumni need from us now.The workshop is not aimed at makingdecisions. It will be about imagining possi-ble futures. Certainly MIT is not going toabandon what we have done well. We doneed, however, to consider carefully whatwe have achieved. Detail will be needed toflesh out schematic possibilities, and thussubsequent or parallel in-depth reviews ofthe GIRs should be undertaken along witha variety of options for the curriculum writlarge. All proposals face practical con-straints: about requirements, aboutoptions, flexible alternatives, and time thelimit of eight semesters. Eventually, we must discuss the cur-riculum, including the GIRs, and howthey relate to what we – the faculty –believe would be best for our futurealumni. After discussion, to begin in Juneand certainly continue next year, it may bethat no change occurs at all. It would be atragedy, however, if there were no discus-sion at all.

Susan S. Silbey is Leon and Anne GoldbergProfessor of Humanities, Professor of Sociologyand Anthropology, and Professor of Behavioraland Policy Sciences, and Chair of the Faculty([email protected]);Rick Danheiser is A C Cope Professor ofChemistry and Associate Chair of the Faculty([email protected]);W. Craig Carter is a Professor in theDepartment of Materials Science andEngineering and Secretary of the Faculty([email protected]).

Page 6: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

6

In a recent article in The New YorkTimes (November 30, 2017) ProfessorsE.J. Levine of the University of NorthCarolina at Greensboro and M.J. Stevensof Stanford University describe thecurrent reality: an economy wheremedian household income has stagnatedwhile university tuition keeps rising. Theyargue that the old social contract wasdamaged beyond repair before the presentadministration was elected: “Whenstudent loan debt eclipsed the totalamount that Americans owed on theircredit cards in 2010, a rebellion againstfancy academics was well underway.Institutions once deemed essential part-ners in nation-building came to seemoverstuffed and defensive – they enjoyedgenerous tax breaks, yet crankily rebuffedcalls for cost containment. This is the his-torical context in which Congress is sum-moning universities back to thebargaining table.” Why has this shift been met with solittle collective opposition from the aca-demic community? Senior academicadministrators of the universities thathave to pay the new tax on endowmentincome must have jointly lobbied againstit; and graduate students, nationwide, didsuccessfully lobby against taxation of theirtuition benefits. But these two specificresponses, one successful and the othernot, against two specific government poli-cies, do not indicate widespread resistanceto government’s intrusion on universities’autonomy to manage their own resources.This is particularly surprising because thegovernment has not managed its ownresources well. In fact, the recent reduc-tion in corporate taxes is likely to furtherincrease fiscal deficits. This thought had occurred to me firstin 2007 when Senator Charles Grassley ofIowa spearheaded a federal initiative toforce universities to spend more of theirincome from endowments. I was Chair ofthe MIT Faculty then and remember adiscussion in Academic Council abouthow to respond. Lobbying by university

presidents might have stopped SenatorGrassley then, but what really hurt thatgovernmental initiative was the sharpdrop in the stock market, which reducedendowment earnings to negativenumbers. Now that the stock market isbullish, the government is back taxinguniversities for saving and investing well.

Levine and Stevens do not explore whyuniversities in general have raised tuition,particularly since the mid-1980s. Publicuniversities faced severe budget cuts fromstate governments, and raising tuition wasone way to handle the sharp decline inresources. Private universities may havehad other reasons for raising tuition, suchas the decline in government support forresearch, the growing cost of managingresearch facilities, and the increased costof the university administration as itsactivities and investments increased incomplexity with rapid globalization. Bythe end of the 1980s, with the collapse ofthe Soviet Union and the emergence ofthe Internet as a major technologicalchange, most universities were underpressure to become more entrepreneurial.They devised new strategies for revenuegeneration, including large-scale invest-ments in real estate, not only to constructup-to-date dormitories, but also to buildinnovation hubs for entrepreneurial activ-ities that promised new streams ofrevenue. In many cases American univer-sities, both public and private, lookedabroad for additional revenues by creatingjoint research centers and universities,even in nations that differ sharply from

the U.S. in their political systems. Therapid progress and spread of informationand communication technologies pre-sented yet another set of possibilities forrevenue generation. Both public and private universitiesalso made significant efforts to raiseendowment funds. In the 1980s, univer-

sity capital campaigns sought to raisehundreds of millions of dollars; by 2000the campaign goals had increased to bil-lions. In this fundraising climate, universi-ties accepted donations from everyone.While courting funds from wealthydonors and wealthy governments, irre-spective of their stance on any socialissues, universities assured their trustees,faculty, and students that no donor couldchip away the universities’ autonomy tofollow their self-established missions. Thisargument was put forward even whenuniversities accepted funding from privatefirms to conduct particular types ofresearch that would benefit those firms. Infact, universities courted close ties withprivate firms – and they could do sobecause of new real estate holdings. Insum: universities celebrated the new era ofentrepreneurship with a wide range ofprograms, and the new information andcommunication technologies enabledthem to reach far beyond their traditionalcatchment area as the entire world wantedto learn from U.S. universities how tobecome entrepreneurial. Why does the government want tochange the norms of its relationship withAmerican universities when they have

The Erosion of Social NormsSanyal, from page 1

Why does the government want to change the norms ofits relationship with American universities when theyhave been so successful as entrepreneurial entities? . . .I want to share my concern with the readers of theFaculty Newsletter: that precisely because of theirentrepreneurial roles, universities have become moremarket actors than non-profit institutions. In an economycharacterized by rising inequality of individuals’ incomeand earnings, universities cannot advocate for equalitywhen the income gap between wealthy universities andtheir relatively poorer counterparts has also steadilywidened over the last 20 years or so.

Page 7: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

7

been so successful as entrepreneurial enti-ties? There could be many possible expla-nations, which are probably betterunderstood by senior academic adminis-trators than by me. Nevertheless, I want toshare my concern with the readers of theFaculty Newsletter: that precisely becauseof their entrepreneurial roles, universitieshave become more market actors thannon-profit institutions. In an economycharacterized by rising inequality of indi-viduals’ income and earnings, universitiescannot advocate for equality when theincome gap between wealthy universitiesand their relatively poorer counterpartshas also steadily widened over the last 20years or so. No wonder then that nounified group of institutions of higherlearning opposed the government’s recentinitiative to tax the endowment income ofthe wealthiest private universities, eventhough this new government tax policysignifies a direct attack on the universities’autonomy. The stratification of universities asinstitutions competing for new revenueand global rankings does not seem muchdifferent from the way people view cor-porate entities. The annual announce-ment of university presidents’ highsalaries in leading newspapers adds tothis perception of the university as a cor-porate entity, not a place primarily foreducation and learning. The recentranking of universities based on a ratio ofincome earned by recent graduates tototal tuition paid for college education isemblematic of the way average citizensview education as yet another commod-ity in the market place. Why weren’t faculty as organized as thegraduate students in opposing the new taxlaws? Political opposition to governmentpolicies is not new to American campuses.From protests against the Vietnam War inthe 1960s to the more recent marchesagainst immigration policies barringinternational students from certainMuslim countries, faculty have vocallyopposed government policies they con-sider unfair or ill-intentioned. Perhapsfaculty do not perceive the tax on endow-ment as a direct attack on their interests.

Or perhaps they have come to view theuniversity as a business enterprise inwhich they are not stockholders. Facultymay be glad to see their universitiesmaking a good return on endowmentinvestments, but rarely do they realize anydirect benefits from such returns in termsof higher salaries. In contrast, when

endowment income declines, as it did in2007-2008, faculty have been asked toaccept a freeze on annual salary increases.What’s intriguing is that faculty rarelyoppose salary freezes, perhaps becausethey are grateful for relatively stableemployment. As a recent study cited in theChronicle of Higher Education (January24, 2018) has shown, faculty also make atradeoff between salary increases and flex-ibility in their work schedule. Whatever may be the reason for theirlack of response, faculty need to be awareof the larger significance of changes in thegovernment-university relationship. Asthe old norms are chipped away, facultymay face a situation where the value oftenure is questioned, retirement rules arereformulated, faculty productivity ismonitored by groups outside the univer-sity, and academic freedom is equatedwith free speech, as Joan W. Scott recentlywarned in the Chronicle of HigherEducation (January 7, 2018). The assaulton the autonomy of the academic com-munity may begin with small steps gearedtowards reducing the privileges of only itswealthiest members, but it may not endthere unless the entire community – andthat includes faculty, students, andadministrators – is vigilant.

As for students, it is worrying that eventhough they so forcefully opposed taxes ontheir stipends a significant number ofgraduate students are now interested informing unions and, thus, seemingly beingtreated as employees in educational enter-prises (The Tech, February 1, 2018, p.1).This is another sign that knowledge pro-

duction is being viewed increasingly as aprofit-making and reputation-buildingactivity whose benefits should be passed onto “the workers.” Should the associationbetween faculty and students be reduced toa contractual relationship guided by laborlaws? Or, are educational institutions placesof learning where relationships betweenmentors and mentees are guided by highermotives not reducible to market forces? That U.S. universities have benefittedin many ways from working closely withmarket institutions cannot be denied, butthe mission of universities is greater thanthe sum of its entrepreneurial activities,whether selling online education orserving as incubators of market savvyinnovations. The issue is: What makes auniversity an autonomous community ofknowledge seekers, and also an independ-ent voice of moral reasoning – particu-larly when it is rare to find such reasoningin public discourse? I realize that universi-ties are not religious institutions that seekto preach morality. But, unlike the stockmarket, universities are not amoral insti-tutions either. They provide a forum forfree deliberation of moral issues. In fact,universities should be in the forefront of

continued on next page

That U.S. universities have benefitted in many ways fromworking closely with market institutions cannot bedenied, but the mission of universities is greater than thesum of its entrepreneurial activities, whether sellingonline education or serving as incubators of marketsavvy innovations. The issue is: What makes a universityan autonomous community of knowledge seekers, andalso an independent voice of moral reasoning –particularly when it is rare to find such reasoning inpublic discourse?

Page 8: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

8

drawing public attention to issues of highsocial significance. Likewise, the govern-ment should not have to dictate to univer-sities about issues such as sexual assault ortell them how to expand access to educa-tion for people of all colors, sexual orien-tations, and religious beliefs. To thecontrary, universities should help govern-ment to see how inequalities in access hurtnational productivity. To what extent uni-versities’ expanding market entangle-ments are affecting its moral role is not anew question but an important one.Similarly, the extent to which the univer-sity system is beginning to take on thenegative characteristics of markets needsto be assessed. MIT is uniquely positioned to play aleadership role in halting the slow erosionof norms that guided the government-university relationship in the past. It has astrong reputation as a place of learningand creativity without the frills usuallyassociated with wealthy private universi-ties. When one reads about the turmoil atMichigan State, or the outrageously highsalaries of presidents of some private uni-versities, it is amazing how MIT hasavoided being singled out for any seriousbreach of the social norms expected ofuniversities. MIT seems like a place stilldevoted mainly to serious scholarlyresearch. Being an institute of “technol-ogy” provides MIT a strong legitimacy,not only in the U.S. but around the world,as its central mission still seems to be sci-entific inquiry and inventions, not justprofit making.

Remember when President CharlesVest announced to the world that withOpenCourseWare (OCW) virtually all ofMIT’s course content would be availableonline at no cost, and the overwhelmingly

positive reactions to that announcementbecause there was no profit motive behindMIT’s noble gesture? Remember whenMIT announced that women faculty inthe sciences had been treated unfairly? Icannot think of a similar reaction noweven as MIT engages in the ambitiouscapital campaign “to make the world abetter place.” Most universities wouldargue that they too are involved in makingthe world a better place, but none has asyet captured the public imagination withan issue of huge moral significance.Through the recent acknowledgement ofits past connection to slavery, MIT – like afew other universities – is coming to termswith this painful history. The governmentdid not ask MIT to address this issue;autonomous social inquiry led MIT to amoral decision, and this will be respectedworldwide as yet another sign of a greatinstitution where concerns of human

dignity override all other concerns. Thedecision to create a task force to study theimpact of automation is also a goodexample of how universities can studymarkets from the perspective of a social

concern and not just as a potential stake-holder in reaping the benefits of markets. If university faculty understand thesignificance of government action at bothfederal and state levels – such as inArizona where legislatures are paying topush conservative studies (New YorkTimes, February 26, 2018, p. A11) –perhaps there will come a time when theywill lead a march on Washington, DC, likethe million-man march or the morerecent women’s march, to demand auton-omy of knowledge production, which iskey to academic excellence. I sincerelyhope that such a march will not be neces-sary, but the growing signs of increasinggovernment control over university affairsworry me – a lot.

The Erosion of Social NormsSanyal, from preceding page

Bish Sanyal is the Ford Professor in theDepartment of Urban Studies and Planning. Hewas Chair of the MIT Faculty from 2007 to2009 ([email protected]).

MIT is uniquely positioned to play a leadership role inhalting the slow erosion of norms that guided thegovernment-university relationship in the past. It has astrong reputation as a place of learning and creativitywithout the frills usually associated with wealthy privateuniversities. . . . Being an institute of “technology”provides MIT a strong legitimacy, not only in the U.S. butaround the world, as its central mission still seems to bescientific inquiry and inventions, not just profit making.

Page 9: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

9

Kenneth R. ManningNaming the MIT Intelligence Quest

WH E N I FI R ST R EAD President Reif ’sletter announcing the new MITIntelligence Quest (MIT/IQ) initiative, Iwas impressed with its vision and poten-tial, but distressed by the possible percep-tion embedded in its logo. I promisedmyself to write a letter to the Presidentand Provost regarding my concern aboutthe logo, and about the need for cautionin approaching the topic of intelligence.Unfortunately, I dropped the ball, perhapsbecause I was apprehensive about raisingthis concern with the project so far along.Now I regret that I did not write the letter. During the discussion announcing the

initiative at the February Institute facultymeeting, my disquiet increased whenProfessors Susan Silbey and AnneMcCants took the podium to comment.So I decided to weigh in then and there,even though the project had gone publicsome weeks earlier. As a graduate studentover 45 years ago, I had organized protestson the use of IQ (Intelligence Quotient)tests to argue inferiority of blacks byRichard Herrnstein, Arthur Jensen, andothers. Since coming to MIT in 1974, Ihave always taught the perils and negativeramifications of shoddy scholarship sur-rounding IQ and its eugenic uses, espe-cially its espousal of racial inferiority. Ijoined my friends and colleagues StephenJay Gould, then at Harvard, and StephanChorover, here at MIT, to challenge thisso-called scholarship. Gould’s classicwork, The Mismeasure of Man, should berequired reading for colleagues as contextfor all research on intelligence. For us at MIT, it is worth rememberingtoo that Nobel Laureate William Shockleyearned his doctorate here. After winning

the Nobel, he went on to pursue spurious,ill-informed scholarship on IQ, outsidehis field of expertise (semiconductors).Even though our new initiative is future-oriented, our community must approachit with full knowledge and appreciation ofpast (and some present) abuses of schol-arship surrounding human intelligence. The unfortunate choice of the logo IQ

was perhaps not simply a language slip. Itsuggests that the initiative would benefitfrom a greater diversity of faculty input, tohelp sensitize the community to quan-daries that they have either internalized orof which they are entirely unaware. Someat MIT, especially in my two departments– STS (Science, Technology, and Society)and CMS/W (Comparative MediaStudies/Writing) – are well suited for anactive role in adding nuance and contextto this important initiative. Having saidthat, I wish to caution that social, political,and ethical issues should be woven intothe fabric of the project and not “out-sourced” to any one group or person, say,the School of Humanities, Arts, and SocialSciences or its dean, as seemed to be thethrust of the faculty meeting. These issuesare the responsibility of colleagues in theSchool of Engineering, equally if notmore so. Indeed, they are the responsibil-ity of everyone. We should take this as an opportunity

to reflect on the history underlying thecomplicated subject of human and artifi-cial intelligence, particularly the ways inwhich it has been used for both positiveand negative purposes. We will then be ina better position to show how ourapproach at MIT is different, and how wecan achieve the goals that we set. We will

also be able to explain how the checkerednotion of Intelligence Quotient differsfundamentally from our initiative knownas Intelligence Quest. The proverb “If wishes were horses,

then beggars would ride,” comes to mindin response to the hope expressed inFebruary’s meeting for faculty to getinvolved voluntarily. We should not takethat chance. This initiative is so innova-tive, with such remarkable potential, thatproactive grassroots efforts are required toground it in responsible social values. The Provost, the Chair of the Faculty,and a representative from the Office ofthe Dean of Engineering respondedswiftly to concerns raised at the Februaryfaculty meeting, and to the call to action,through further discussion among them-selves and with other faculty members.Many now share these concerns and wantto change the logo so as to disassociate itfrom historical abuses tied to theacronym IQ. The challenge, as I under-stand it, is how to make the change withleast disruption. Alternatives are beingfloated for consideration. Though thelogo naming is likely to be solved withpersistent follow-through, we must con-tinue to frame and articulate a responsi-ble vision for this pioneering initiative,given the regressive social, cultural, andpolitical agendas that have emerged – andcontinue to emerge – around suchresearch. In our probe for intelligence, wemust exercise vigilance to preserve scien-tific integrity throughout.

Kenneth R. Manning is the Thomas MeloyProfessor-Rhetoric/History of Science in theDepartment of Comparative MediaStudies/Writing ([email protected]).

Page 10: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

10

Luisa KenausisAron BernsteinRobert RedwineMichael Hynes

Nuclear Weapons Education Project

N UCLEAR WEAPON S POS E AN exis-tential threat to all of humanity by theirvery nature, and the recent tensionsbetween the United States and NorthKorea have brought that threat back intothe public eye. It is essential for youngpeople, particularly those who grew upafter the end of the Cold War, to be edu-cated on what nuclear weapons are andon their potential effects on the world.The Nuclear Weapons Education Projectat MIT aims to support this goal byhelping professors and lecturers in variousdisciplines prepare a lecture or two, orselected course materials, involvingnuclear weapons topics for their introduc-tory-level courses, thus reaching largenumbers of students. We believe that evena limited introduction to the issues sur-rounding nuclear weapons could helpengage student interest in a topic that isunfortunately increasingly relevant to theyoung people of today. The long-term objective of the NuclearWeapons Education Project is to teach ourstudents, who will become future policy-makers, scientists, journalists, lawyers, andvoting citizens, about the nature andimportance of nuclear weapons. Nuclearweapons play a massive and complex rolein national and global security. It is essen-tial that colleges and universities take stepsto prepare the next generations of policy-makers and citizens for the tremendoustask of protecting the United States – andthe world – from ever enduring theunthinkable devastation of a nuclear war.One or two lectures on nuclear weapons isnot enough to dive into the nuance anddetail that these issues demand from apolicy perspective, but we believe it is

enough to plant the seeds of interest andurgency in the minds of students who areinterested in facing challenging globalproblems. Further, we believe this wouldprovide an opportunity to equip students

with the foundational knowledge andinformational resources to continuelearning about these issues on their own. With increasing effort over the last fewyears, the members of the NuclearWeapons Education Project have beenworking to advance this initiative at MITin several ways. We are working with theinstructors of introductory physics classesto include materials relevant to nuclearweapons issues in problem sets and otherassignments. Also, during IAP 2016, Prof.Bernstein co-taught a course on nuclearweapons with Prof. Jim Walsh of MIT’sSecurity Studies Program (Department ofPolitical Science). The course offered aconcise overview of the history of nuclearweapons, some technical discussion oftheir physics and physical effects, and anintroduction to the political issues andconsequences of nuclear weapons to aclass of about 12 students. Luisa Kenausis,then a junior, was a student in the class.

This past IAP, the Nuclear WeaponsEducation Project offered a series of threelectures on nuclear weapons topics,taught by Ms. Kenausis, Prof. Bernstein,and Dr. Michael Hynes, of the

Department of Nuclear Science andEngineering. Lastly, we have made significantprogress developing the Nuclear WeaponsEducation Project’s Website (nuclear-weaponsedproj.mit.edu) into an informa-tive resource for educators filled withbrief, reliable summaries of informationon a variety of nuclear weapons topics.Work remains to be done on that initia-tive, and we plan to complete most of theremaining tasks with the help of studentresearchers. Going forward, four undergraduatestudents will be working through theUROP program on broad-ranging,summary-level research on nuclearweapons for use on the Nuclear WeaponsEducation Project Website. The studentswill be coordinating with one another aswell as with their faculty advisors – Prof.Bernstein, Prof. Redwine, and Dr. Hynes –on the topics they choose to research and

The long-term objective of the Nuclear WeaponsEducation Project is to teach our students, who willbecome future policymakers, scientists, journalists,lawyers, and voting citizens, about the nature andimportance of nuclear weapons. . . . It is essential thatcolleges and universities take steps to prepare the nextgenerations of policymakers and citizens for thetremendous task of protecting the United States – andthe world – from ever enduring the unthinkabledevastation of a nuclear war.

Page 11: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

11

summarize. Since the content being pro-duced for the Website is intended to be ahigh-level summary of the most impor-tant information that is publicly availableonline, we believe that this UROP willoffer student participants an opportunityto educate themselves on nuclear-relatedtopics that are of interest to them whilerequiring minimal background knowl-edge or research experience. In keeping with the spirit of theNuclear Weapons Education Project, wehave sought to make the process ofresearching and self-educating on thesetopics accessible to our UROP students,who are mostly freshmen. We hope thatthe relative newness of these students tothe field of nuclear weapons will prove tobe a benefit in their work. Since theintended reader of the Website is an edu-cator, likely without a background innuclear weapons, students who do notspecialize in nuclear issues may be partic-ularly well-equipped to identify the con-cepts that will be most foreign andchallenging or most interesting to a non-nuclear audience. The Nuclear Weapons Education Projectis also continuing to pursue opportunities topromote education on nuclear weaponsissues outside of MIT. Prof. Bernstein hasalso been coordinating with physicists atover a dozen universities who are working toadvance this goal in the courses they teach ordevelop new courses or seminars at theiruniversity. We have made particularprogress with Prof. Jim Napolitano andProf. Bernd Surrow, the chair and vice chair,respectively, of the Department of Physics atTemple University. Meanwhile, Ms. Kenausis departedfrom MIT at the end of January to begin aHerbert Scoville Jr. Peace Fellowship at theCenter for Arms Control and Non-Proliferation in Washington, D.C., whereshe continues to work on the NuclearWeapons Education Project in a part-timecapacity. In Washington, D.C., Ms.Kenausis hopes to enlist the support ofother individuals interested in promotingnuclear weapons education via researchand organizational efforts. More con-cretely, Ms. Kenausis is also planning to

give talks on nuclear weapons issues atpublic high schools in the D.C. area overthe coming months with Dr. SaraKutchesfahani, her colleague at the Center. The current members of the NuclearWeapons Education Project steadfastlybelieve that the education of future gener-ations of policymakers and citizens will bea vital step towards the development ofsafe and effective nuclear weapons policy.Further, we believe that a substantiveintroduction or exposure to nuclearweapons issues can be enough to trigger astudent’s interest in these issues, even ifthat introduction is brief. This belief is notfounded solely on optimism: Ms.Kenausis’s academic and career path werepowerfully shaped by her first exposure tonuclear weapons topics in the classroom,and that experience underlies her com-mitment to this project. For his part, Prof. Bernstein has beenparticularly engaged with nuclearweapons issues since the Cuban MissileCrisis. He sat through that event with aRussian colleague, Oleg Chubinsky, whohad recently come to work in thecyclotron that Prof. Bernstein wasrunning. Prof. Bernstein’s involvementwith nuclear arms control was furtherincreased by his interactions with some ofthe Manhattan Project’s alumni, includingMIT’s Victor Weisskopf and PhillipMorrison, and Henry Linschitz of theBrandeis Chemistry Department. For those of us who are old enough tohave lived through the Cold War and whorecall all too well air raid drills and discus-sions of possible paths to survival in caseof nuclear war, it is sobering that we mustinitiate related discussions at this time.But the world has probably not beenpaying enough attention to this ongoingthreat in the past few decades, and recentevents have only emphasized this reality. With the recent and unexpected newsof a possible summit between PresidentDonald Trump and North Korean leaderKim Jong-un in the near future, the workof the Nuclear Weapons EducationProject only stands to become more rele-vant. Accordingly, we are actively seekingthe help and support of others to move

this project forward and expand ourreach. There are two primary ways inwhich members of the MIT communitycan support the Nuclear WeaponsEducation Project:

1. Contact the authors of this article toget involved with the Education Project,by joining our mailing list network ofeducators or perhaps in ways that suityour specific interests.

2. Spread the word about our initiativeby sharing this article, and the NuclearWeapons Education Project Website, withfriends and colleagues at other universi-ties. We welcome contact from educatorsand students at other schools interested injoining our network and/or seekingadvice about promoting nuclear weaponseducation on campus.

In closing, the Nuclear WeaponsEducation Project does not advocate forany particular political goal or promote acertain solution to the challenges posed bynuclear weapons. We do not wish toindoctrinate students with our personalbeliefs or ideas about nuclear weapons orvisions for nuclear policy. Instead, we aimto promote the use of carefully-researched, factual, concise informationabout nuclear weapons to give studentsthe intellectual tools to critically engagewith the challenging issues of nuclearweapons. As today’s students become thepolicymakers and citizens of tomorrow,even a small investment in our students’knowledge of nuclear weapons issues willhelp set the stage for safe, informednuclear weapons policy in the future. Wewelcome feedback at [email protected].

Luisa Kenausis is a Scoville Fellow, Center forArms Control and Non-Proliferation; MIT Classof 2017 ([email protected]);Aron Bernstein is a Professor Emeritus ofPhysics ([email protected]);Robert Redwine is a Professor of Physics andDirector of the Bates Linear Accelerator([email protected]);Michael Hynes is a Senior Lecturer,Department of Nuclear Science andEngineering ([email protected]).

Page 12: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

12

Dave PritchardTeach TalkMIT Students and Deep Learning:Perspectives and Suggestions

AbstractIn the September/October 2017 MITFaculty Newsletter, Freeman et. al. (“HowDeeply Are Our Students Learning?”)exhibit a number of apparently straight-forward (to professors) problems thatbefuddle the vast majority of studentswho are doing well in the course thatteaches the skills necessary to do thatproblem. This is taken as evidence thatour teaching imparts the ability “to runthe model” without imparting “deeplearning.” My research group(RELATE.mit.edu) has found some addi-tional troubling student difficultiesincluding failing to check “solutions” forobvious flaws, starting problems withouteither a conceptual analysis or a coherentplan, and worsening attitudes towardslearning science after taking 8.01 (Physics I).The main thrust of this article is that theeducation research literature providesmany important insights into these – andthat improving our educational outcomesforces us to bring this research (andresearch-developed instruments) to bearon these problems. In particular, literatureon expert-novice differences, modeling-based pedagogy, and students’ epistemo-logical approaches. Additionally, we urgedepartments to adopt standard MIT engi-neering design to reforming subjects inwhich learning outcomes and methods ofassessing them are agreed upon first, andrelevant contemporary knowledge isapplied to reach the desired outcomes.

IntroductionThe September/October 2017 FNL articlecontains a strong indictment of the resultsof our instructional process by a group of

MIT teachers who are widely respected fortheir thoughtful devotion to educatingMIT students: Dennis Freeman, SanjoyMahajan, Warren Hoburg, DavidDarmofal, Woodie Flowers, GeraldSussman, and Sanjay Sarma (hereafterFreeman et. al.). They supply clear evi-dence that our students (and those at othertop universities) have gaping holes in their“deep understanding.” They cannot “useNewton’s laws . . . to model the world,” andare unskilled at “making the model andinterpreting the results.” What they aregood at is “manipulation without under-standing” and “running the model.”

Additional Troubling DeficienciesFreeman et. al. is a description of multipleobserved student deficiencies, but doesnot offer perspectives (e.g., from the edu-cation literature) that might illuminatethe diagnosed difficulties, nor does itsuggest how we might change our peda-gogy so that students obtain deeper learn-ing of skills and habits of mind that wouldenable them to overcome the aforemen-tioned difficulties. I take these omissionsas an invitation to respond to their article.Before sharing some perspectives on thesestudent difficulties, let me first add someadditional difficulties discovered by mygroup’s (RELATE.MIT.edu) research onteaching introductory Newtonianmechanics: 8.01, 8.011, 8.01L, and a three-week Mechanics ReView taughtover IAP to 8.01 students receiving a Dthat offers an opportunity to move on to8.02 rather than repeat all of 8.01.

Student Attitudes About Learning Science

It is important to realize that students

learning 8.01, 18.01, 6.002, etc. are youngadults who have already developed bothknowledge about many GIR subjects andpersonal beliefs about what constituteslearning them. Several Physics EducationResearch (PER) groups have addressedthe problem of determining students’ per-sonal epistemologies by constructingsurveys that “measure student beliefsabout physics and about learning physics”– quoting from the abstract of the latestsuch survey, the Colorado LearningAttitudes about Learning Science Survey[CLASS, (Adams et. al., 2006)]. TheCLASS probes beliefs about personalenjoyment of the subject, perceptions ofrelevance of the subject to the world, andespecially the conceptual thinking andself-confidence of students with respect tosolving problems. Students are deemedmore expert-like if they agree with state-ments similar to these:

• Physics comes from a few principles. • When studying physics, I relate theimportant information to what I alreadyknow rather than just memorizing it theway it is presented. • I am not satisfied until I understandwhy something works the way it does. • I enjoy solving physics problems.

And disagree with these:

• After I study a topic in physics andfeel that I understand it, I have difficultysolving problems on the same topic. • If I don’t remember a particularequation needed to solve a problem on anexam, there’s nothing much I can do(legally!) to come up with it.

Page 13: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

13

• A significant problem in learningphysics is being able to memorize all theinformation I need to know.

The last time (2004) the CLASS wasadministered in 8.01 TEAL it revealed a ~ 7% decrease in learning expertise frompre- to post-survey that extended across alleight categories that CLASS probes. Faintcomfort might arise because typical intro-ductory physics courses in the U.S. lowerthe expertness of learning attitudes ~ 10%across these categories. Of particularconcern is our additional finding for the Dand F students in this class: their problem-solving self-confidence after taking thiscourse was down ~ 30% (in all other cate-gories their responses were indistinguish-able from the ABC students). I find this personally distressing:Newtonian Mechanics was the firstscience where mathematics was used tomodel the world based on a few funda-mental assumptions – the prototypicalmodel of most science and engineeringthat we teach at MIT; it is also the founda-tional science for several MIT majors. Toteach our students that it is a pile of for-mulae unrelated to their world, theirinterest, and their future is a pedagogicaltravesty.

Algebra First vs. Planning a Solution

MIT professors want students to solveproblems like an expert in that domain;often this means by starting with a solu-tion plan based largely on conceptualreasoning. Yet 8.01 students typicallystart by writing an equation in which thegiven variables are already plugged into afundamental equation (e.g., momentumconservation) – often remembered froma previous similar problem. To encour-age this, part a) of several problems onthe 8.01 final demanded written problemplans. My research group found that justover half of our students couldn’t make acoherent plan (i.e., unambiguous, even ifincorrect) – irrespective of whether theirgrade was A- or C (Barrantes & Pritchard,2012). In individual discussions with ~ 20 of our students, all but one said theystarted those problems by seeking rele-

vant formulae and solved algebraically,then went back and wrote the verbal plan(which was part a). They neither made aplan based on physical concepts, nor didthey adduce physical reasons to buttressthe applicability of the equations theystarted from. These findings demon-strate that neither high school nor MITtaught their students to make coherentplans for solving problems, nor even pro-vided a learning environment where the“good” students learn such planning forthemselves. This serious divergence from facultyexpectations stems from student vs.faculty differences in what they think theobjective of answering the problem is:roughly “get the answer” vs. “understandhow you got the solution” and what epis-temological approaches should be used toreach this objective. Students’ tools werestudied by closely observing small groupsof students solving problems (Tuminaro& Redish, 2007), then cataloging the“epistemic games” they brought to bearon the problem. Their University ofMaryland students used primarily six,including:

• Mapping Meaning to Mathematics:map the problem story (circumstancesand givens) to mathematics. • Mapping Mathematics to Meaning:Identify target variables, find equationrelating target to given information(similar to “plug and chug”). • Physical Mechanism Game: constructstory about equations – often in terms of“phenomenological primitives” (DiSessa,1988) such as a bigger object requiresmore force, more force → more velocity,gravity wins out in the end, etc.

The expert approach of “understandproblem and plan solution startingfrom physical principles” was notobserved. For example, when MappingMeaning to Mathematics, “studentsoften rely on their own conceptualunderstanding to generate this [mathe-matical representation] – not on funda-mental physics principles” (Tuminaro &Redish, 2007).

We have made several efforts to teachstudents to write problem plans, includ-ing introducing “Tweet Sheets” thatprovide a framework for problem plans;space for ~ three lines of text and a smallgraphic and instruction on what consti-tutes a plan. Students can fill these in afterdoing group problems in class, then canbring them to the weekly quizzes. Afterthe first few weeks, only about 15% of thestudents brought them to the quiz. Whenqueried about this, most students said “Ididn’t know what to write.” (About 10%of the students said “After I review theproblem and write the tweet sheet, it is soeasy to remember that I don’t need thesheet.”) My research group is still experi-menting to find ways to help studentslearn to plan solutions. Tentatively we’veconcluded that they should have deliber-ate practice (Ericsson, 2009) of twoimportant skills: first, determining whichphysical principles apply and why, andsecond, learning to decompose problemsinto sub-problems within which a uniquesubset of principles apply.

Perspectives For UnderstandingThese DifficultiesThe findings of Freeman et. al. togetherwith our additions are all indications thatour students lack “deep expert-like under-standing”– in spite of their ability to scorewell on our final examinations. Thesefindings include:

1. Overreliance on the mathematicalmanipulations of the models they aretrying to apply, 2. Inability to create these models or todiscover them, 3. Inability to plan a solution, or evento state one retrospectively after answer-ing correctly, 4. Inability to make sense of, verify, andinterpret the solution once it has beenworked out, and 5. Loss of self-confidence in problem-solving and decrease of perception thatNewtonian mechanics is either relevant totheir lives or intrinsically interesting.

continued on next page

Page 14: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

14

Expert-Novice Studies

The novice-expert literature provides thebest framework for understanding thesedifficulties, essentially providing aresearch-discovered list of “novice charac-teristics” that contain many items on thelist above. The watershed paper in thisfield is Categorization and Representationof Physics Problems by Experts and Novices(Chi, Feltovich, & Glaser, 1981). Thispaper shows that novices categorize prob-lems using surface features (pulley, blockon inclined plane, collision, . . .) ratherthan according to fundamental domainprinciples as do experts. Thus students,even when asked to classify problems onthe basis of similarity of solution, classify ablock sliding down a plane without fric-tion as similar to a block on a plane withfriction, rather than as similar to a masson a pendulum (since both exemplifyenergy conservation – i.e., gain kineticenergy mgh after descending a height h). Students classifying problems based onsurface features need a huge mentallibrary of solved problems to find onethat’s sufficiently similar superficially to agiven exam problem that the same solu-tion principles also apply, as evidenced byour students’ increasing (on the post-test )agreement with the novice-like questionon the CLASS “If I want to apply amethod used for solving one physicsproblem to another problem, the prob-lems must involve very similar situations.” Many of the 6000+ references to Chi’spaper expand the list of specific expert-novice differences: experts consider theproblem conceptually and from differentrepresentations to plan their approachbefore starting to write equations, classifyproblems according to deep principles ofthe domain, have much more intercon-nected domain knowledge, and are able toretrieve and apply domain-specificmodels appropriately to unfamiliar prob-lems (this is called strategic knowledge).When solving a problem, experts checkthat their solution process makes sense asthey proceed, and importantly “make

sense of ” their answer using limitingcases, dimensional analysis, comparisonwith common sense, similarity to pastproblems, etc. Novices assume thatmanipulating the mathematical proce-dure for obtaining the answer will give aresult that is correct without checking theresult. I would emphatically add “makingsense of the answer” to the list of our stu-dents’ deficiencies. When I ask MIT fresh-men how they’ll check their answer to amechanics problem, they’ll most fre-quently say “I’d check the algebra.” I knowthat most physics faculty will acknowl-edge this deficiency: in our (Pritchard et.al., 2009) study What Else (besides the syl-labus) Should Students Learn inIntroductory Physics? sense-making wasthe top choice of the professors, and thesecond-lowest preference of the students.That’s understandable: students don’t getinto good colleges by pausing on question4 on a high stakes exam to consider waysto check the answer. Many of my departmental colleaguesexpress a strong desire to help undergrad-uate students “think like a physicist.”When pressed to be more specific aboutwhat this means, they typically say “checkthat what they do ‘makes sense’,” “organizetheir knowledge coherently,” “approachproblems using concepts rather thanalgebra.” I suggest that “think like anexpert” is a good description of theoutcome that most MIT faculty reallywant for their students. The list of expert qualities accords wellwith what we demand of students on ourPhD candidacy exam. I find that classify-ing just one or two student responses tonovel questions as novice vs. expert-likeenables me to reliably predict whether thecommittee will pass or fail a student. TheNAS study How Students Learn (Chi,Bassok, Lewis, Reimann, & Glaser,1989)(Bransford, Brown, & Cocking,2000) summarizes novice-expert differ-ences and is a good place to start learningthis valuable perspective.

Modeling

David Hestenes (Hestenes, 1987) has con-

vincingly argued that models form theeveryday mental tools that most STEMprofessionals use. The word model appears12 times in Freeman et. al. and I find itvaluable in my professional and teachinglife. Models and modeling are central toMIT and explicit in many upper divisionsubjects, especially engineering: we usemathematics to make models of the struc-ture and behavior of some well-specifiedsystem. Yet in most GIRs, we fail to impartknowledge about models explicitly:

1. None of the Science GIR subjectsmention “model” – with the exception of8.02. 2. We recommend textbooks that don’tgive a modeling perspective, e.g., the best-selling Young and Freedman (11th editionused in 8.01 & 8.02) mentions “model”briefly in the introduction and not againin the subsequent 1714 pages. 3. We often provide “formula sheets”that list formulae without indicatingwhat role each formula plays in whichmodel (e.g., law of force, constraint, lawby which state variables change, . . .),thereby encouraging the belief that“subject X is a pile of formulae” ratherthan imparting the view that the formu-lae are only one part of a particularmodel of reality in that subject.

The result is students who, at the endof introductory subjects, can manipulatethe equations, i.e., “run the model” inFreeman et. al. – often without being ableto name the model they are applying orknowing its limitations. For example, in8.01 the equations F=ma and F = μfriction* m*g are used in the samesolution without knowledge of the verylimited applicability of the secondformula (fails for static friction, assumesthat normal force is mg). Expert scientistsand engineers are careful to check theapplicability of the model as part of theirsolution process – the types of systemsand circumstances under which themodel containing these formulae applies,the model’s limitations, and whether itspredictions make sense (NAS13).

MIT Students and Deep LearningPritchard, from preceding page

Page 15: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

15

Hestenes and collaborators have devel-oped a pedagogy called “modeling instruc-tion” for physics that is explicitly designedto teach the ideas and procedures of mod-eling reality. In modeling instruction, stu-dents are guided to discover the basicmodels in laboratory, and to apply them toproblems – a process called “modeling.”This pedagogy leads to very large improve-ments of students’ scores on both conceptinventories and more problem-orientedtests; high school students start lower butfinish higher than students taking tradi-tional introductory courses at selective col-leges (Hestenes, Wells, & Swackhamer,1992). A recent review paper of ~ 50 differ-ent introductory physics courses (Madsen,McKagan, & Sayre, 2015) documentsanother benefit of modeling instruction: ituniformly improves the expertness of stu-dents’ learning attitudes as measured bythe Colorado Learning Attitudes aboutScience Survey (CLASS ) – typically by ~ 11%. Hestenes’ group started the AmericanModeling Teachers Association (modelin-ginstruction.org) that runs two-weekworkshops on modeling instructionwhich upwards of 10% of all U.S. highschool physics teachers have attended, andthis pedagogy is widely known amongphysics high school teachers. (Thissummer there will be a total of ~ 62 work-shops for biology, chemistry, physicalscience, and physics teachers.) This isknown by only a small percentage ofcollege or university faculty – unfortunatebecause modeling pedagogy would givemany MIT professors a valuable perspec-tive on reforming their subjects. As an example, in developing MIT’s three-week Mechanics ReView, my grouphas designed a modeling-based approachto categorizing domain knowledge andproblem solving, and found that in addi-tion to improving grades on the final examby ~ 1.5 grades it improved their expert-thinking by ~ 11% as measured by CLASS.Importantly, these students also showedan improvement in their subsequent 8.02performance relative to their peers whoeither did not take the ReView or who tookanother full semester of traditionally

taught 8.01 (A. Pawl, Barrantes, &Pritchard, 2009; Rayyan, Pawl, Barrantes,Teodorescu, & Pritchard, 2010).

Cognitive

The MIT course catalog is based on a listof topics that are taught in each subject. Acomplementary perspective is to specifythe cognitive skills that the student is sup-posed to learn. Indeed, a cognitive per-spective is highly germane tounderstanding the student difficultiesmentioned above. The figure below showsa cognitive hierarchy, and gives theteacher’s relationship with tasks corre-sponding to questions at each cognitivelevel (below).

The figure divides cognitive knowledgeinto four categories, shown as overlappingovals with examples from Newtonianmechanics at the top. These start on theleft with a foundation of facts, definitions,and simple concepts like how to defineand measure acceleration. Built on thisfactual foundation (expressed by its ovaloverlapping) is knowledge of proceduresand operations – these are the models.Confronted with an unfamiliar problem,an expert applies strategic knowledge tosort through the known procedures(models) to determine which might berelevant or helpful, then solves the prob-lems using the relevant model(s). At thevery right is Adaptive Expertise, theknowledge/ability to create something

new. These categories are closely paral-leled by cognitive taxonomies like those ofBloom and Marzano (Marzano &Kendall, 2007; Teodorescu, Bennhold,Feldman, & Medsker, 2013). This perspective illuminates typicaltasks assigned by teachers; these are pre-sented (below the ovals) in different colorsdepending on whether the problem/taskhas a known answer, and whether theteacher who poses it intends the student toanswer it in a particular way. Let’s catego-rize our current instructional approach(e.g., in 8.01) through this lens: list thetopics in the syllabus, teach this material(mostly concepts and procedures). Suchinstruction doesn’t improve students’

strategic knowledge because they don’tneed to learn how to determine whethermomentum is a key to this problem whenmomentum is this week’s topic. The facts & procedures-based instruc-tional approach allows little opportunityfor helping the students obtain strategicknowledge spanning the whole range oftopics in that subject – the ability toorganize their knowledge of different factsand procedures so that it can be fluentlyaccessed when confronting an unfamiliarproblem. Indeed, in my bookshelf ofintroductory physics textbooks, not oneattempts explicitly to instill strategic knowledge, for example with a chapter

continued on next page

Page 16: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

16

whose title is something like “How toanalyze a new problem to determinewhich of the previous 12 chapters canhelp you solve it.” It is not surprising thatunfamiliar final exam problems involvingseveral of the studied procedures are con-sidered to be very difficult by our stu-dents. Many of the problems in Freemanet. al. don’t have a clear similarity to any ofthe weekly homework problems in thesubject; hence they expose the students’lack of strategic knowledge.

Other Helpful Perspectives

Several other useful (to me) perspectivesare Kahneman’s Type 1 and Type 2 think-ing (quick and reactive vs. thoughtful andlogical) and its relevance to short conceptquestions vs. traditional long-form prob-lems, the importance of quick associationamong relevant domain vocabulary as ameasure of knowledge interconnectedness(Gerace, 2001), and defining “understanda concept” as “fluency with, and interrelat-ing of, the representations commonly usedwith that concept.” (e.g., Motion with con-stant acceleration might be representedwith a table of position vs. time, a formulafor velocity vs. time, a strobe picture, or agraph of velocity vs. position.)

Addressing These DeficienciesHaving broadened the list of troublingstudent deficiencies and offered someperspectives, I now turn to how individ-ual departments can reform our sub-jects to help students overcome thesedifficulties.

Outcomes and Goals

Freeman et. al. and I are distressed that,having done well in our subjects, our stu-dents are not reaching the learning out-comes involving skills, habits, andattitudes that many faculty stronglybelieve are important. I put the blame onour system (shared by other colleges) thatdefines a subject as a syllabus of topics andsubtopics that will be taught by an expertin that subject. This teacher-centered

description lacks any specification of whatis expected of students in terms of skills,habits, or abilities. Thus addressing thesedeficiencies starts with:

#1 Departments must specify subjects interms of outcomes expected of students –both learning outcomes with respect to spe-cific topics, and general skills.

Thus, where the current coursedescription lists a topic, e.g., “momen-tum,” there would be a learning objective“identify when momentum is/is not con-served.” This has the advantage that aprofessor can write a problem that mostother department members would agreeassesses a particular learning objective.Importantly, learning objectives canaddress more general learning outcomesthan topics – for example “learn to checktheir solutions using dimensions andlimiting cases.” Specifying learningobjectives would enable us to emphasizegeneral skills and habits that are gener-ally considered important in the twenty-first century such as the 4 C’s –collaboration, communication, criticalthinking and problem solving, and cre-ativity (p21.org, (NGSS Lead States,2013)). I also recommend that we move ourinstructional goals toward Strategic andCreative cognitive levels because smartphones and Internet search enginesprovide instant access to facts and inte-grated collections of procedures (likeWolfram alpha, the computationalpackage r, Mathematica, etc.). Forexample, we can remove “challenging”

algebra-intensive problems the first timestudents are exposed to a topic, and addreview problems later that explicitlyrequire students to say which previouslystudied topics apply in a given physicalsituation and why – or give problemswith multiple choice answers where thedistractors can be eliminated by dimen-sional analysis or special cases. Given that

freshmen will forget over half of whatthey learn but don’t use regularly bygraduation (Ebbinghaus, Ruger, &Bussenius, 1913; Andrew Pawl,Barrantes, Pritchard, & Mitchell, 2010),but will improve on skills that they reuse,answer checking, dimensional analysis,and determining what’s conservedshould receive more emphasis than theydo now – especially when reforming sub-jects like GIRs where most students willmajor in another course.

Assessment

Given that the department has set thelearning objectives and goals, it is impor-tant to realize that assessment sets thestandard for student learning (and hasstrong influence on the instruction). So,

#2 Departments must adopt assessmentinstruments and develop calibrated ques-tion pools that accurately assess their learn-ing goals.

This process should strongly considerincorporating some of the research-devel-oped instruments. Examples mightinclude instruments that evaluate stu-dents’ ability to reason from fundamental

MIT Students and Deep LearningPritchard, from preceding page

The main thrust of this article is that the educationresearch and cognitive science provide many importantinsights and remedies that address the serious studentdeficiencies identified by Freeman et. al. and in thisarticle – and that improving the outcomes of oursubjects forces us to bring this research (and research-developed assessment instruments) to bear on ourefforts to improve our courses.

Page 17: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

17

physical principles (Andrew Pawl et. al.,2012), assessments of general scientificreasoning (Lawson, 1978), and widelyused instruments whose typical results areknown for different institutions, e.g., theCLASS, Test of Understanding of Graphs(TUG), and discipline-based instrumentslike the venerable Force ConceptInventory that has transformed teachers’views on the importance of conceptualreasoning. We should also considermaking some instruments of our own – agood place to start would be to collectquestions like those in Freeman et. al. Thisprocess will result in stable year-to-yearassessments of student knowledge andlearning. As a side benefit, these assess-ments can complement student evalua-tions of learning in assessing teacherperformance.

DBER – Discipline-Based EducationResearchThe main thrust of this article is that theeducation research and cognitive scienceliterature provides many importantinsights and remedies that address theserious student deficiencies identified byFreeman et. al. and in this article – andthat improving the outcomes of our sub-jects forces us to bring this research (andresearch-developed assessment instru-ments) to bear on our efforts to improveour courses. This is a tremendous chal-lenge due to the immensity of the possiblyrelevant literature. To put this in perspec-tive, a typical faculty member is wellacquainted with literature in a specialtylike Atomic Physics Research for which aGoogle search will have ~ 0.1M hits, incomparison Physics Research will yield ~ 2.7M, and Education Research ~ 12.4M,a count that probably excludes much edu-cation-relevant research from fields likecognitive science, behavioral psychology,etc. Hence it is unrealistic to expect evenour most dedicated professors to knowthe literature relevant to education – eventhe dedicated teacher-authors in Freemanet. al. believe that “researchers in STEMeducation [have not] . . . identified theseproblems and shown their solution.” Ibelieve that the only realistic route to fil-

tering through the voluminous educationresearch to get beyond the “educationaltechnology fix of the day” and find whatwill truly help us improve our subjects isthat:

#3 We must incorporate Discipline-BasedEducation Researchers into processes #1 and#2 above, as suggested in the recentNational Research Study, (Singer, Nielsen,& Schweingruber, 2012).

Summary

The above three recommendations – setgoals, agree on how to assess them, andincorporate relevant educational research– are consistently recommended by edu-cation reformers, and have been success-fully implemented at two selectiveuniversities by Nobel prizewinner CarlWieman (Wieman, 2017). Grant Wigginshas advocated them, calling the process“backward design” to contrast it with thetopics first/assessment last approach thatis typical in universities (Mctighe &Wiggins, 2012). It should be easy for MIT faculty toadopt backward design because it is reallythe “forward design” that we practice inour professional lives: select goals, deter-mine how we’ll measure them, build onrelevant literature, experiment/fail/recycleuntil the goals are met then publish orpatent. Hopefully, we can adopt this famil-iar practice to systematically and scientifi-cally improve MIT undergraduateeducation. I acknowledge many helpful and rele-vant comments by Lori Breslow, SanjoyMahajan, Leigh Royden, and GeraldSussman. I welcome comments on this([email protected]) and encouragefurther Faculty Newsletter articles onimproving MIT education.

ReferencesAdams, W., Perkins, K., Podolefsky, N.,Dubson, M., Finkelstein, N., & Wieman,C. (2006). New instrument for measuringstudent beliefs about physics and learning

physics: The Colorado Learning Attitudesabout Science Survey. Physical ReviewSpecial Topics – Physics EducationResearch, 2(1), 10101. http://doi.org/10.1103/PhysRevSTPER.2.010101

Barrantes, A., & Pritchard, D. (2012). PartialCredit Grading Rewards Partial Under-standing. (unpublished – ask me for a copy)

Bransford, J. D., Brown, A. L., & Cocking,R. R. (2000). How people learn: Brain,mind, experience, and school: Expandededition. Washington, DC: National ….Washington, D.C.: National AcademyPress. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=EJ652656

Chi, M.T.H., Bassok, M., Lewis, M. W.,Reimann, P., & Glaser, R. (1989). HowStudents Study and Use Examples inLearning to Solve Problems, CognitiveScience, 182, 145–182.

Chi, M. T. H., Feltovich, P. J., & Glaser, R.(1981). Categorization and representationof physics problems by experts andnovices. Cognitive Science, 5, 121–152.Retrieved from http://www.sciencedirect.com/science/ article/pii/S0364021381800298

DiSessa, A. (1988). diS88 diSessaPhenomenological Primitives “knowledgein pieces”.pdf. In P. Forman, G. Pufall(Ed.), (p. 49).

Ebbinghaus, H., Ruger, H. A., &Bussenius, C. E. (1913). Memory: A con-tribution to experimental psychology.,1913. http://doi.org/10.1037/10011-000

Ericsson, K. A. (2009). Discovering delib-erate practice activities that overcomeplateaus and limits on improvement ofperformance. In A. Willamon, S. Pretty, &R. Buck (Eds.), Proceedings of theInternational Symposium on PerformanceScience (pp. 11–21). Utrecht, TheNetherlands: Association Europienne desConservatoires Academies de Musique etMusikhochschulen (AEC).

continued on next page

Dave Pritchard is Cecil and Ida GreenProfessor of Physics ([email protected]).

Page 18: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

18

Gerace, W. J. (2001). Problem Solving andConceptual Understanding. Proceedings ofthe 2001 Physics Education ResearchConference, 2–5.

Hestenes, D. (1987). Toward a modelingtheory of physics instruction. AmericanJournal of Physics, 55(5), 440. Retrievedfrom http://scitation.aip.org/content/aapt/journal/ajp/55/5/10.1119/1.15129

Hestenes, D., Wells, M., & Swackhamer, G.(1992). Force concept inventory. ThePhysics Teacher, 30(3), 141.http://doi.org/10.1119/1.2343497

Lawson, A. E. (1978). The developmentand validation of a classroom test offormal reasoning. Journal of Research inScience Teaching, 15(1), 11–24.http://doi.org/10.1002/tea.3660150103

Madsen, A., McKagan, S. B., & Sayre, E. C.(2015). How physics instruction impactsstudents’ beliefs about learning physics: Ameta-analysis of 24 studies. PhysicalReview Special Topics - Physics EducationResearch, 11(1), 1–19. http://doi.org/10.1103/PhysRevSTPER.11.010115

Marzano, R., & Kendall, J. (2007). The newtaxonomy of educational objectives (2nded.). Thousand Oaks, CA: Corwin Press.

NGSS Lead States. (2013). NextGeneration Science Standards: For States,By States. Washington, D.C.: The NationalAcademies Press.

Pawl, A., Barrantes, A., Cardamone, C.,Rayyan, S., Pritchard, D. E., Rebello, N. S.,. . . Singh, C. (2012). Development of amechanics reasoning inventory. In 2011Physics Education Research Conference(Vol. 2, pp. 287–290). http://doi.org/10.1063/1.3680051

Pawl, A., Barrantes, A., & Pritchard, D. E.(2009). Modeling applied to problemsolving. In AIP Conference Proceedings(Vol. 1179). http://doi.org/10.1063/1.3266752

Pawl, A., Barrantes, A., Pritchard, D. E., &Mitchell, R. (2010). What do SeniorsRemember from Freshman Physics ?

Pritchard, D. E., Barrantes, A., Belland, B.R., Sabella, M., Henderson, C., & Singh, C.(2009). What Else (Besides the Syllabus)Should Students Learn in IntroductoryPhysics? In 2009 Physics EducationResearch Conference (pp. 43–46).http://doi.org/10.1063/1.3266749

Rayyan, S., Pawl, A., Barrantes, A.,Teodorescu, R., & Pritchard, D. E. (2010).Improved student performance in elec-tricity and magnetism following priorMAPS instruction in mechanics. In AIP

Conference Proceedings (Vol. 1289).http://doi.org/10.1063/1.3515221

Singer, S. R., Nielsen, N. R., &Schweingruber, H. A. (2012). Discipline-Based Education Research: Understandingand Improving Learning in UndergraduateScience and Engineering. (Committee onthe Status, Contributions, and FutureDirections of Discipline-Based EducationResearch; Board on Science Education;Division of Behavioral and SocialSciences, and Education, Eds.). TheNational Academies Press. Retrieved fromhttp://www.nap.edu/openbook.php?record_id=13362

Teodorescu, R. E., Bennhold, C., Feldman,G., & Medsker, L. (2013). New approachto analyzing physics problems: ATaxonomy of Introductory PhysicsProblems. Physical Review Special Topics –Physics Education Research, 9(1), 10103.http://doi.org/10.1103/PhysRevSTPER.9.010103

Tuminaro, J., & Redish, E. F. (2007).Elements of a cognitive model of physicsproblem solving: Epistemic games,(January), 1–22. http://doi.org/10.1103/PhysRevSTPER.3.020101

Wieman, C. E. (2017). Improving HowUniversities Teach Science.

MIT Students and Deep LearningPritchard, from preceding page

Page 19: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

19

Stacey LantzHigher Ed in the Era of #MeToo: A Symposium for Faculty and Graduate Students

TH E G RAD UATE CON SORTI U M FOR Graduate Studies inGender, Culture, Women, and Sexuality (GCWS) at MIT ishosting a free half-day conference on how the recent #MeToomovement has impacted higher education, specifically at thegraduate and doctoral levels. The #MeToo movement, started by Tarana Burke in 2006, hashighlighted the prevalence of sexual violence within all aspects ofour society – from Hollywood to college campuses. More sur-vivors are sharing their experiences and demanding change fromtheir peers and institutions. It is critical for us to have more cre-ative ways to support, prevent, innovate, and collaborate on thisissue. Many marginalized communities, both inside and outsideof academia, have led the fight to end sexual violence and wemust elevate these voices and communities. During the conference we will feature work by academics,researchers, activists, legal experts, and many in other fields as welook at this issue through an interdisciplinary lens. Panels willfocus on how sexual harassment and assault impact graduateand doctoral students differently than they do undergraduates,and how we can specifically address that difference through acritical analysis of Title IX and the gaps that are created bycurrent legislation, and a focus on the impact of media on the#MeToo movement. Workshops will establish the space fordeeper conversations on creating safer environments, focusingon the specific needs of people with marginalized identities, andspecific action steps that people, regardless of their role, can take. The conference will be followed by a cocktail reception toshare what was learned and continue the discussion!

Stacey Lantz is Program Manager in the School of Humanities, Arts, andSocial Sciences ([email protected]).

Page 20: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

20

Roger LevySally HaslangerCeasar McDowellfor the MIT Day ofAction Organizing Team

MIT Day of Action

WE ARE DELIGHTED TO ANNOUNCE

MIT’s second annual Day of Action, to beheld on the ground floor of the StataCenter on Tuesday, April 17, from 10 am

to 8 pm. The Day of Action is a large-scalegrass-roots civic engagement and actionevent devoted to the political, economic,and social challenges facing us today.

Please join us as we act together tofulfill MIT’s mission “to bring knowledgeto bear on the world’s great challenges,”seeking open-minded dialogue with peersand colleagues of diverse backgrounds andviews. All of us, regardless of political affil-iation, can contribute to identifying andseeking out the roots of the greatest chal-lenges facing our society, and to planningfor actions addressing these challenges inthe present day and in times to come. TheDay of Action is open to all, representingthe full diversity of our society. We aremade stronger by open, respectful dia-logue and the exchange of ideas from thewidest variety of intellectual, religious,class, cultural, and political perspectives.We invite you to join us, to share your con-cerns and questions, your hopes and ideas,and your knowledge and skills. Last year’s Day of Action drew an esti-mated 1,000+ participants from MIT andthe broader local communities; we hopeto see a strong turnout again this year.Sessions provide opportunities to learnfrom experts, build skills, and connectwith community. This year’s sessions,continually updated at https://dayofaction.mit.edu/events, include:

Learn from ExpertsPerils for Democracy; with DaronAcemo lu and Daniel ZiblattConfidence at the Ballot Box: Russian Bots,Midterm Elections and Cybersecurity; bySuzanne Mello-StarkLanguage, Bias, and Power; by JustinKhoo and Roger LevyBlack Panther: Call for Liberation orRepressive Desublimation?; with MichelDeGraff, Grégory Pierrot, and M. Amah Edoh

Learn from experts.Build your skills.Connect with the community.

•prisonandincarceration•scienceadvocacy•transformingeducationthroughopenaccess•foodandwatersecurity•immigrationandvulnerableidentities•andmore

All sessions are FREE and open to the public.

Takeactionthroughpanels,workshops, lms,andlecturesabout:

Page 21: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

21

Conflict on the Korean PeninsulaToday: History, Technology, Politics; byArissa Oh and Vipin NarangComing Out Faithful: A Multi-FaithConversation with LGBTQ ReligiousLeaders; by The Lutheran EpiscopalMinistry at MIT How Not To Be Evil: Tech WorkersAgainst Racism, Sexism, and Surveillance;by Moira Weigel and Ben TarnoffRemote Activism in Electoral PoliticsWorkshop with Sister District Project &Indivisible Somerville; by Graham Jonesand Patrick JoyceExperiences from prison: Incarceration,education, and rehabilitation; by JoliSparkman-Bayron, Reentry Coordinatorat Bethany HouseThe Baltimore Leadership School forYoung Women: a screening of the docu-mentary STEP, with Q&A by VisitingProfessor Kimberly Juanita BrownWho Are We Talking To? TheHumanities, the Public, and the Streets;by Darien Pollock and Myisha CherryTransforming Education Across theGlobe: How MIT Open Learning isImproving Access and Quality ofEducation; by Brandon Muramatsu,

Anna Schrimpf, Sarah Hansen, Robert Fadel,Eryn Heying, and Vijay Kumar (moderator)Demystifying the MIT Corporation; byPaul KominersAsmarina: A film screening and discussionabout Race, Immigration, Citizenship andthe Habesha Community in Milan; byMedhin Paolos (director) and LorgiaGarcia PeñaKorea: The Forgotten War, a documen-tary screening and discussionIf You Care About Truth, Fight for Justice:Universities and the Communities TheyInteract With; by Naomi Scheman, CeasarMcDowell, and Libby HsuThe History (and Present State) ofStudent Activism at MIT; by Steven Pennand Tanya Llanas

Build your SkillsOrganizing for Radical Inclusion:Stories From Hackathons; by ElaineHarris, Fahad Punjwani, Steven MaxPatterson, and Hildreth EnglandOpen access: How you can help make theworld’s scholarly works available to all;by Katharine Dunn, Herng Yi Cheng,Ellen Finnie, Roger Levy, Nick Lindsay,and Jeff Rosenberg

Science Advocacy 101: From interest toaction; by Ortal Ullman of the Union ofConcerned ScientistsCorrupting the Public: Philosophy forthe People; by Brianna Toole and AlexHargroderReimagine Education Through DesignThinking; by Ela Ben-Ur, Jared Cosulich,and Julian SerraoEveryday action: Solidarity investment;by Ryan HarrisonConnect with Community by meetingand discussing with local representativesof the Union of Concerned Scientists, theCenter for Science and Democracy, MassPeace Action, and the Green CenturyFund. The Day of Action is free and open tothe general public; RSVP is appreciatedbut not required. There are also opportu-nities to volunteer to help with day-of-event logistics. Please visit https://dayofaction.mit.edu to learn more – wehope to see you there!

To The Faculty Newsletter:

M IT STU D E NTS HAVE A significantlyhigher suicide rate than students at otheruniversities. According to ”Reappropriate,”an Asian American feminist Website (reappropriate.co/2015/05/asian-american-student-suicide-rate-at-mit-is-quadruple-the-national-average), the rate of suicideamong Asian students at MIT is quadru-ple that of the national average. In 2015,two MIT students committed suicide inone week, and already this year an MITmathematics graduate student took herown life. These facts would make onethink that suicide should be the major

concern that the MIT administration hasabout the well-being of their students. Thus it is both strange and disturbingthat sexual harassment, and not suicide, isthe issue that President Reif and ProvostSchmidt think is paramount. The onlyexplanation is that they believe thatunwelcome sexual experiences are worsethan death and that their belief justifiestheir threat to remove any faculty memberor staff who does not take the sexualharassment course that they have chosento inflict on us. As those who have taken the course arewell aware, its content violates every prin-ciple for which an MIT education stands.

MIT courses are designed to stimulateoriginal thinking; this course is designedto discourage it. When a student or col-league approaches you about a problemthat is related to sex, the course mandatesthat you parrot the words that it hastaught you to say. To make a humanresponse based on one’s own experience isforbidden. It is sad that MIT has chosen toadopt this form of thought control, but itis consistent with other decisions made byPresident Reif about the direction inwhich he wants MIT to go.

Dan StroockProfessor, Department of Mathematics

lettersSuicide and Sexual Harassment at MIT

Roger Levy is an Associate Professor in theDepartment of Brain and Cognitive Sciences([email protected]);Sally Haslanger is a Professor in theDepartment of Linguistics and Philosophy([email protected]);Ceasar McDowell is Professor of the Practiceof Community Development in the Departmentof Urban Studies and Planning ([email protected]).

Page 22: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

22

The map tracks the experiences of afictional patient, “Josh Junior,” who sus-tains an injury while playing flag football.Our use of a student persona for thismapping exercise was deliberate. After all,students are at the core of the Institute’smission and are, by extension, the veryreason for MIT Medical’s existence. Josh’sjourney begins with his impressions ofMIT Medical before becoming a patientand follows him through his initial visit toUrgent Care and subsequent need for spe-cialty and primary care. From the start, we knew that a student-focused map would put a particular spot-light on our Urgent Care service. For mostMIT students, Urgent Care is their firstexposure to MIT Medical; for many, it is theonly interaction they have with MITMedical during their time in Cambridge.Our 18,000 Urgent Care visits each yearaccount for approximately 15 percent ofclinical encounters across all of MITMedical. Not surprisingly, students accountfor 58 percent of Urgent Care visits.

Gathering More DataFollowing our internal operational analy-sis, we were fortunate to work with a tal-

ented team of four students enrolled inSloan’s Health Systems InnovationHealthcare Lab (H-Lab) course. As part ofthe requirements for this course, students

undertake a four-month-long actionlearning project in host healthcare institu-tions. These projects typically involvelooking at complex problems in Internettechnologies, operations management,strategic marketing, and other areas.Similarly to the journey mapping process,our H-Lab team collected and analyzeddata, interviewed stakeholders, and syn-thesized a set of concerns:

1. Reputation: We have a reputationproblem with students. Our wait times arenot transparent, and they can be long,sometimes up to four hours. Students alsohave concerns about privacy duringcheck-in and triage.

2. Systems and operations: Our systemsand operating processes are unaligned,inefficient, and sometimes fall short ofmeeting patient expectations. Thisincludes everything from staffing and thecheck-in process, to our hours of opera-tion and the ways clinical staff communi-cate with each other.

3. Identity: Urgent Care has an identitycrisis. Is it an emergency room? (Answer:

No) Is it primary care? (Answer: No) Is itconvenient care? (Answer: Well, the loca-tion is convenient, but the long wait timesaren’t.) To further confuse the issue, unlike

many local urgent care facilities, oursdoesn’t offer ancillary services like lab orradiology after hours or on weekends.

The result of all of these issues is a dis-jointed patient experience that dispropor-tionally affects students.

Putting Patients FirstAs part of this process, we have alsoturned to our internal experts for theiradvice. For example, within the field ofmental health, there is a well-known best-practice model known as trauma-informed care. This means building apractice that is a safe space for trauma sur-vivors – where they feel safe, understood,and listened to. Trauma-informed carecan take many forms, from redesigningintake forms to replacing bench-likeseating in waiting areas with individualseats that don’t force trauma survivors tosit in close proximity to strangers. But atits heart, trauma-informed care meansplacing the care of one’s most vulnerablepatients at the core of an organization’smission and vision. As mental health practices began toadopt this care model, they discovered

Improving the Urgent Care ExperienceStuopis, from page 1

MIT Medical Patient Journey Map

Page 23: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterMarch/April 2018

23

something interesting – that the changesthey made to become more sensitive totrauma survivors were also welcomed bypatients who were not trauma survivors.That when you design a system toprovide the best possible care to the mostvulnerable patients, it improves care foreveryone. Everyday examples of this phenome-non can be found in the environmentaround us. Closed captioning, originallyintended to benefit deaf and hard ofhearing individuals, now allows everyoneto access television programing in noisybars, gyms, and airports. Similarly, accessramps and sidewalk curb cuts are utilizednot only by wheelchair users, but bycountless others – parents pushingstrollers, delivery people with handcarts,travelers with rolling luggage, and manymore. With this model in mind, Associate

Medical Director and Chief of MentalHealth and Counseling Karen Singletonand Chief of Student Health ShawnFerullo have suggested that the organizingprinciple for MIT Medical should be whatthey are calling student-informed care. While MIT students are exceptionallyintelligent and capable individuals, aspatients, they remain our most vulnerablegroup. Many are novices at seekinghealthcare independently, and few haveestablished clinical relationships at MITMedical. By restructuring our practice toprovide the best possible care for stu-dents – our least savvy healthcare con-sumers and our most vulnerable patientpopulation – we believe we can elevate thecare we provide to all our patients.

Urgent Care ReimaginedAs our Urgent Care Service evolves toprovide more student-informed care, let’simagine how different Josh Junior’s nextexperience will be:

Early one Saturday morning, around 2 am,Josh starts to feel ill. He goes online tomedical.mit.edu to see what to do. Thewebsite has information about common ail-

ments and suggests that Josh’s symptoms aresevere enough that he should call for advice.When Josh calls, he reaches a friendly nurse,who listens, is sympathetic, and instructshim to try to get some rest. She tells him thatif he feels worse or the same in the morning,he should come in after 8 am, when UrgentCare opens.

Josh wakes up and still feels unwell. Butthough Josh is sick, his ailment is hardlyurgent. He reaches for his phone and visitsthe MIT Medical website, where he sees thatthe current wait time in Urgent Care is 45minutes. He doesn’t want to wait that long,and since it’s a Sunday with no classes, heuses the website to make an appointmentfor a few hours later, sets his alarm, and goesback to sleep.

Later, Josh makes his way to MIT Medical.By now, Urgent Care has most likely beenrenamed to better reflect the type of servicewe provide – MIT Medical ConvenientCare, Same-day Care, or Walk-in Clinic.When Josh arrives, the signage is clear, and afriendly person at the front desk guides himto a check-in kiosk. After scanning his IDand answering some questions on a touchscreen, he’s checked in – no more clipboards,paper forms, and pens, or having to discusswhat is bothering him in a somewhat publicsetting. The system alerts us that he hasarrived and tells us why he is here.

Josh takes a seat in the comfortable waitingarea, where he finds snacks, a coffeemachine, and charging station for hisphone. Since he made an appointmentearlier, a medical assistant greets himpromptly. He is discreetly called to an examroom, where the medical assistant takes hisvital signs and communicates that informa-tion to the doctor or nurse practitioner whowill treat him. When the clinician comesinto the room, he or she can begin treatingJosh immediately. There will be no repeti-tive “What brings you in here today?” ques-tions, because Josh has already providedthat information and talked with themedical assistant.

Josh and his clinician look at his medicalrecord together on the large flat-screen TVon one exam-room wall. The clinician givesJosh a prescription and instructs him to goto the lab to get some blood work done.Before he leaves, Josh is handed an iPad tofill out a quick survey about his experience.

Then he heads to the lab, which, along withthe pharmacy, is conveniently open duringthe same hours as Urgent Care – even onweekends. He doesn’t have to leave campusto get his blood test or fill his prescription, sohe can begin taking his medication rightaway. In under an hour, he’s back at hisdorm, and when he feels better, he tells hisfriends about the great experience he had atMIT Medical.

Over the coming year, our team at MITMedical will be working to make thisvision a reality. Our new electronicmedical record, patient portal, and prac-tice-management system implementationin June 2018 will be a big step toward real-izing these aspirations. After reviewingour data and conducting several pilotslooking at our hours, we also anticipatechanging the operating hours for UrgentCare to 8 am to 8 pm 7 days per week,likely with the next academic year. Thischange enables us to open lab, radiology,and pharmacy for expanded hours, thusimproving our ability to care for our stu-dents and other patients during the timeswe are open. Additional work is alsounderway to evaluate the benefits of a newname and brand, and to improve staffingpatterns, patient flow, and facilities. Through data collection, listening, andour transformation to a student-informedcare model, MIT Medical’s Urgent CareService will deliver the best possible careto the MIT community. By focusing onstudent needs, we will elevate the experi-ence of every patient who walks throughour doors.

Cecilia Stuopis is Medical Director and Head,MIT Medical ([email protected]).

Page 24: MIT Faculty Newsletter, Vol. XXX No. 4, March/April 2018web.mit.edu/fnl/volume/304/fnl304.pdfMIT Faculty Newsletter March/April 2018 3 Saudi Arabia, in fact, remains an oppressive

MIT Faculty NewsletterVol. XXX No. 4

M.I.T. NumbersMIT Research Expenditures 1940–2017

Source: Office of the Provost/Institutional Research

$0

$200

$400

$600

$800

$1,000

$1,200

$1,400

$1,600

$1,800

$2,000

1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015

Mill

ions

Lincoln Labs Non Federal

Lincoln Labs Federal

Defense Labs Federal

SMART

Broad Non Federal

Broad Federal

Campus Non Federal

Campus Federal

Total Research C$

Sputnik 1957

Project MAC 1963

Draper Lab Divested

1973

Cancer Center 1974

Whitehead 1982

Faculty Early Retirement

1997

Broad Institute

2004 - 2010

ARRA Funding

2010

80%

97% 94% 96%

92%

82%

86%

75% 73%

83% 78%

71%

64%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

$0

$80

$160

$240

$320

$400

$480

$560

$640

$720

$800

1940

1945

1950

1955

1960

1965

1970

1975

1980

1985

1990

1995

2000

2005

2010

2015 M

illio

ns

Campus Federal

Campus Non Federal

Campus % Federal excluding Broad

85%