Embed Size (px)
Citation preview
Quantitative Literacy Faculty Learning Community An Experiment to Assess the Effectiveness of Simulations to Improve Quantitative Literacy
Conducted Spring Semester 2011 By Lisa L. Verdon February 21, 2012
This quantitative literacy experiment took place in Spring 2011 in the business economics elective course of Marketing (BUEC 230). The experiment involves the repeated use of simulations to improve quantitative literacy. Although marketing is not itself a Q course, the prerequisite of Principles of Economics (ECON 101) is a Q course. The class started with 33 students and 31 students completed the course and all five simulations. Of the 31 students one student was a freshman, 7 students were sophomores, 12 students were juniors, and 11 students were seniors. While the majority of students were economics or business economics majors, 8 students were other majors. The simulations are from the Harvard Business School Publishing and conducted online via their website. There are two types of simulations used. The first simulation is for a medical equipment motor manufacturing company. The second simulation is for a rental car company in Florida. The first simulation was the most complex. This simulation was given at the beginning of the semester, as a sort of pre-‐test, and again at the end of the semester. The simulation had many content pieces including four different types of customers. These four different types of customers all had different concerns ranging from temperature and motor life of the equipment to pricing and customer service. The students had to choose the size of their sales force, what sectors the sales force would be focused on, what price and discount to offer each sector, how much to spend on market research, and how much to spend on research and development to improve the motors. The second simulation was far less complex and was focused primarily on pricing. This scenario was presented using scaffolding. Initially, the rental car scenario was located in one city in Florida. The students had choice over the weekday and weekend price and the size of their fleet. As the simulation progressed the car rental locations increased to multiple cities where the students were allowed to have different weekday and weekend prices in each city, to allocate their fleet between the cities, and adjust the overall size of their fleet. In all scenarios students were presented with numeric information in a variety of formats. A sample screenshot of some of that information is provided in Appendix C. All scenarios included sales figures for their own company and their closest competitors. These sales figures reported price, quantity, and market share. Additionally, students were able to see historic financial statements for their company. All of this information was updated each time the student made a decision. The students were given an opportunity to make different decisions in 12 consecutive periods. Students were fired if they had three consecutive periods of losses.
A description of the simulation assignment is attached in Appendix A. A major component of the grade was the profits made in each scenario. To receive the greatest amounts of points possible, students had to earn profits greater than two standard deviations away from the mean. However, most of the points for the assignments came from a write up of their experience in this simulation. The written portion of the simulation allowed students to reflect on the financial information that they saw, as well as the decisions that they made, and what they would do differently in the future. My primary measure of success and quantitative literacy is based on increased profits from the first simulation to the last simulation. While increase profit from simulations two through four also shows an increase in quantitative literacy, an increase in the fifth simulation demonstrates that students are able to take that knowledge and apply it to a different situation. Although there are other measures in these simulations, the students were told to focus on profits, so other measures may or may not have increased.
Simulation 5 -‐ 1
p-‐value Simulation 4-‐2
p-‐value
Profits $985,932 0.0394 $159,000,000 0.0000 Grade -‐4.0968 0.3754 -‐0.8387 0.7723 Market Share
0.26% 0.5177 19.52% 0.0000
Looking at scenarios 1 and 5 we do see a statistically significant increase in the cumulative profit. We also observed a statistically significant increase in the cumulative profit and market share going from scenarios 2 to 4. There is no significant difference between students of different grade levels. There is also no significant difference in the overall grade for the scenario assignments. Interestingly, there was a difference between different grade levels in the first simulation. There was a statistically significant difference between sophomores and seniors both in market share and cumulative revenue. In both of these variables the sophomore students had greater market share and cumulative revenue then the seniors. However, there was not a statistically significant difference between these groups in their target measure of cumulative profit. This difference seems to suggest that perhaps, seniors follow directions more closely then underclassmen. Overall, these results suggest that simulations are effective in improving quantitative literacy across all grade levels. Generally speaking, I would not take a Business Economics elective course as being representative of a greater body of students. However, this class did include a relatively large number of non-‐majors. Again, all of the students in this class had been through a quantitative course before this one. It is unclear if simulations would have the same impact if this were the students’ first exposure to use of quantitative information.
Appendix A Simulation Assignment Instructions and Details
MarketingSimulationDetailsBUEC230Spring2011
GeneralInformation
Wewillbeusingtwodifferentsimulations.Eachsimulationwillberunwithmultiplescenarios.Foreachscenario,youwillbeaskedtomakeaseriesofmonthlydecisions.Beforeeachdecision,youwillbeprovidedwithappropriateinformationtomakeyourdecisions.Aftereachdecision,theinformationisupdatedtoreflectyourdecisionandthebehaviorofyourcompetitors.Thisupdatedinformationshouldbeusedtomakeyourdecisionforthenextperiod.Thesemonthlydecisionswillresultinprofitsorlossesforyourcompanythatwillaccumulateuntiltheendofthescenario.Instructions
1. Eachscenariowillhaveanopenandclosedate.Youmustcompletethescenario,bymaking12monthsofdecisions,withinthattimeperiod.Failuretocompleteascenariowillresultinafailinggradeforthatsimulation.
2. Onceyouhavecompletedthescenario,youmustanalyzeyourresults.Thisanalysisshouldbenomorethantwopageslong.Itshouldbewrittenasanexecutivesummaryforyourmanager.Yourcompletedanalysismustbeapdffileemailedtoyourinstructorbycloseofbusiness(5pm)ontheduedate.Theanalysisshouldaddressthefollowingquestions:
A. Howwouldyoudescribethemarketforthisproduct?Isitrelatively
competitiveornot?Howdoyouknow?Includespecificinformationthatsupportsthisconclusion.
B. Howwouldyoudescribethebehaviorofyourcompetitors?C. Whatwasyourstrategyatthebeginningofthescenario?Howdid
youimplementthisstrategy?Didyouchangeyourstrategyatanypointinthesimulation?Whyorwhynot?
D. Whatwouldyoudodifferentlyifyouweretorunthisscenarioagain?
Grading
Eachscenariois10percentofyouroverallcoursegrade.Thismeanssimulationsmakeuphalfofyourgradesoputintheappropriatetimeandeffort.Thegradingforeachofthesimulationswillbebasedonyourparticipation,theprofitsyouearn,andyouranalysis.Eachsimulationisworth100pointsTheparticipationportionofyourgradeissimplypass/fail.Ifyoudonot
completeasimulationinthedesignatedtimeframe,youfailthatsimulation.
Youmayearnupto20pointsfortheprofitsofyourcompany.Thepointswillbedeterminedbasedonyourprofitsrelativetothemeanprofitfortheclass,ineach
scenario.Achievingthemeanprofitresultsin10points.Foreveryonestandarddeviationaboveorbelowthemean,thestudentwilleithergain(above)orlose(below)5points.Thesegainsandlossesofpointsareboundsothemostpointsthatcanbegainedorlostis10.Example:Ifthemeanprofitis$100,000andthestandarddeviationis$25,000 Profit Points $30,000 0 $60,000 5 $85,000 10 $100,000 10 $110,000 10 $130,000 15 $165,000 20Theremaining80pointsareearnedthroughyourwrittenanalysis.Thewrittenanalysiswillbegradedbasedprimarilyoncontentbutappropriateform,tone,andlengthwillaffectyourgrade.Failuretodeliveryouranalysisasinstructedwillresultinanautomaticdeductionof10points.Lateassignmentswillnotbeaccepted.
Schedule
Opens Closes AnalysisDueSimulation1 1/17/11 1/26/11 1/28/11Simulation2 1/29/11 2/4/11 2/7/11Simulation3 2/8/11 2/28/11 3/4/11Simulation4 3/5/11 4/1/11 4/4/11Simulation5 4/5/11 4/25/11 4/29/11
SimulationDescriptions
PricingSimulation:UniversalRentalCar
(Simulations2–4)Description:Thisweb‐basedsimulationpresentsanengagingcontextinwhichstudentsdeveloptheirknowledgeofpricingbymanagingarentalcaroperation(Universal)inFloridaandimprovesregionalperformancebydevelopingapricingstrategy.Thesimulationinvolvesthreeregions‐‐Orlando,Tampa,andMiami‐‐whichvaryinsize,marketdynamics,andcustomermix.ThefocusiscompetitionbetweentwocarrentalcompanieswithplayersinputtingdecisionsforUniversal.Thesimulationlastsupto12simulatedmonths.Whetherassignedasindividualsorteams,players
mustsetweekdayandweekendpricesforeachregionforeachperiod(month)and
makefleetcapacitydecisionsatseveralpointsthroughoutthesimulation.
MarketingSimulation:ManagingSegmentsandCustomers
(Simulations1and5)
Description:
Inthissingle‐playersimulation,studentsassumethepositionofCEOofamedical
motormanufacturerandaretaskedwithexecutingasuccessfulbusiness‐to‐
businessmarketingstrategyoveraperiodoftwelvefiscalquarters."Students
determineallaspectsofthecompany'sgo‐to‐marketapproach(includingsales‐
forcedeploymentanddistributionchannelstrategy)andassociatedelementsof
productpolicy,includingpricingandmarketpositioningofthecompany'sproduct
linetobothsmallandlargevolumecustomers.Studentsalsoprioritizethe
manufacturer'seffortsinacquiringandretainingcustomersinordertoachievea
combinationofsustainablerevenuesandprofitsandmaximizecumulativeprofitsat
theendofthesimulation.Thesimulationalsoillustratesthebenefitsofinvestment
inmarketresearch.
Appendix B Written Portion Grading Rubric
Writing Assignment Rubric You writing assignments will be assessed according to the following criteria:
1. Attention to the Writing Task (20 points) Description: The writer fulfills the assignment with respect to length, topic, form, and purpose. This includes providing an answer to all questions posed in the assignment.
1 2 3 4 5
2. Quality of Ideas (30 points) Description: The writer presents a central argument supported by relevant materials. The reasoning is sound and arguments are critiqued appropriately. There is clear differentiation between the author’s work and referenced works.
1 2 3 4 5
3. Organization and Development (25 points)
Description: The writer presents the arguments in a logical ordering. There is a clear thesis, introduction, and conclusion. Each paragraph has a single topic and is connected to surrounding paragraphs. 1 2 3 4 5
4. Clarity and Style (15 points)
Description: The writing assignment is easy to read, uses the appropriate style, and demonstrates clarity. Appropriate variety is combined with proper sentence structures and transitional phrases.
1 2 3 4 5
5. Grammar and Mechanics (10 points)
Description: The writer employs proper grammar and spelling while avoiding run-on sentences and paragraphs, fragments, comma splices, etc. All research is properly referenced. 1 2 3 4 5
Scale 5 = no significant problems 4 = some minor problems, readability is not impaired 3 = many minor problems, still reasonably readable 2 = some significant problems, impairs readability 1 = many significant problems, readability significantly undermined
Appendix C Simulation Screen Shot
Do Not
Copy o
r Pos
t
This document is authorized for use only by Lisa Verdon until October 2011. Copying or posting is an infringement of copyright. [email protected] or 617.783.7860.
________________________________________________________________________________________________________________ This guide was prepared by Professors John T. Gourville, Thomas T. Nagle, and John Hogan for the sole purpose of aiding classroom instructors in the use of Universal Rental Car - Pricing Simulation, HBP No. 2093. This guide and the simulation are developed solely as the basis for class discussion and are not intended to serve as endorsements, sources of primary data, or illustrations of effective or ineffective management. Copyright © 2007 Harvard Business School Publishing. To order copies or request permission to reproduce materials, call 1-800-545-7685, write Harvard Business Publishing, Boston, MA 02163, or go to http://www.hbsp.harvard.edu. No part of this publication may be reproduced, stored in a retrieval system, used in a spreadsheet, or transmitted in any form or by any means—electronic, mechanical, photocopying, recording, or otherwise—without the permission of Harvard Business Publishing. Harvard Business Publishing is an affiliate of Harvard Business School.
FACILITATOR’S GUIDE
Universal Rental Car - Pricing Simulation
2661 R E V . N O V E M B E R 1 2 , 2 0 1 0
Appendix D Simulation Results Comparisons
