Report of Student Learning and Achievement Azusa Pacific ... · Report of Student Learning and Achievement Azusa Pacific University School of ... Student Learning Assessment for

IACBE Public Disclosure of Student Learning 1

ReportofStudentLearningandAchievementAzusaPacificUniversitySchoolofBusiness&Management

ForAcademicYear: 2015-16

MissionoftheAPUSchoolofBusiness&Management

MissionStatement:WeareaChrist-centeredcommunityofscholarsandprofessionalspursuingacademicexcellencetoadvancetheworkofGodintheworld,developingstudentsofcharacterandcompetenceasdifference-makersinbusinessandsociety

StudentLearningAssessmentforBAandBSBusinessDegrees

ProgramIntendedStudentLearningOutcomes

1. CompetentBusinessKnowledge

2. CriticalThinkingAbility

3. ChristianBusinessEthics

4. CapableCommunicationSkills

5. CollaborativeTeamworkAbility

6. ComprehensiveGlobalAwareness

IntendedStudentLearningOutcomes

1. LearningOutcome1:Studentswillbeabletodemonstrateknowledgeinmultiplebusinessdisciplines,includingmanagement,finance,marketing,accountingandeconomics.

2. LearningOutcome2:Studentswillbeabletoidentifyandsolvebusinessproblemsusinganalyticalandcriticalthinkingskills.

3. LearningOutcome3:StudentswillbeabletodemonstratetheabilitytoevaluatebusinessdecisionsbasedonaChristianperspective.

4. LearningOutcome4:Studentswillbeabletoconveyideasclearlythroughprofessionalwrittencommunication.Studentswillalsobeabletoexpressideaseffectivelythroughprofessionaloralpresentations.

5. LearningOutcome5:Studentswillbeabletodemonstratetheabilitytofunctionasaneffectivebusinessteammember.


6. LearningOutcome6:Studentswillbeabletoidentifycultural,economicandpoliticalaspectsofbusinessinaglobalenvironment.

AssessmentInstrumentsforIntendedStudentLearningOutcomes—DirectMeasuresofStudentLearning: PerformanceObjectives(Targets/Criteria)forDirectMeasures:

1. ETSMajorFieldTestforBusiness(BUSI450) Objective(Target/Criterion)forDirectMeasure1:Ascoreof70%orbetterontheETSMFT.GeneralProgramISLOsAssessedbythisMeasure:1

2. CaseStudyAnalysis(BUSI450) Objective(Target/Criterion)forDirectMeasure2:Anaveragescoreof3(4pointscale)ontheCriticalThinkingRubricforeachstudent.GeneralProgramISLOsAssessedbythisMeasure:2

3. SeniorSeminarPaper(BUSI496) Objective(Target/Criterion)forDirectMeasure3:Anaveragescoreof3(4pointscale)ontheChristianCharacterRubricforeachstudent.GeneralProgramISLOsAssessedbythisMeasure:3

4. SeniorSeminarPaper(BUSI496) Objective(Target/Criterion)forDirectMeasure4:Anaveragescoreof3(4pointscale)ontheWrittenandOralCommunicationRubricsforeachstudent.

GeneralProgramISLOsAssessedbythisMeasure:4

5. TeamWorkInventory(BUSI450)GeneralProgramISLOsAssessedbythisMeasure:5

Objective(Target/Criterion)forDirectMeasure5:Ascoreof70%orbetterontheTeamSimulationSurvey.

6. InternationalBusinessFinalExam(BUSI370)GeneralProgramISLOsAssessedbythisMeasure:6

Objective(Target/Criterion)forDirectMeasure6:Anaveragescoreof3(4pointscale)ontheGlobalAwarenessRubricforeachstudent.

AssessmentInstrumentsforIntendedStudentLearningOutcomes—IndirectMeasuresofStudentLearning: PerformanceObjectives(Targets/Criteria)forIndirectMeasures:

1. IndirectMeasure1:AlumniSurvey Objective(Target/Criterion)forIndirectMeasure1:70%oftherespondentsreporteffectivepreparationforeachofthe6ProgramSLOareas.GeneralProgramISLOsAssessedbythisMeasure:1-6

2. IndirectMeasure2:Employer’sSurvey Objective(Target/Criterion)forIndirectMeasure2:70%oftherespondentsreporteffectivepreparationforeachofthe6ProgramSLOareas.GeneralProgramISLOsAssessedbythisMeasure:1-6


AssessmentResults:BAandBSBusinessDegrees

SummaryofResultsfromImplementingDirectMeasuresofStudentLearning:

1. SummaryofResultsforDirectMeasure1:

Trait

StudentsMeanScores

Target(NationalMean)

Accounting 40 41.5

Economics 41 39.8

Management 58 54.3

QuantitativeAnalysis 35 36.4

Finance 41 42.4

Marketing 59 55

Legal&Social 60 59.5

InformationSystems 44 50.1

InternationalIssues 45 40.3

Overall

150

150.3

SummaryofData:Overall,thestudentsperformedwellwith80%ofthestudentsscoring70%orbetterontheETSBusinessMFT.Only20%ofthestudentsscored


below70%ontheMFT.Asawhole,thestudentsscoredattheNationalmean(150.30)fortheentiretest.StudentsperformedabovetheNationalmeanin5ofthe9subjectareasandwellabovethenationalmeaninthesubjectsofManagement(3.70abovethenationalmean),Marketing(4.00abovethenationalmean)andInternationalIssues(4.70abovethenationalmean).Analysis:Overall,studentsperformedatasatisfactorylevelonallareasmeasuredbytheETSMajorFieldTest,however,markedimprovementwasshownintheInternationalIssuesscoreat+4.70abovethenationalmeanversus-5.80belowthenationalmeantheyearbefore.Thismaybeduetotherecentadditionoffull-timeinstructorsteachingBUSI370InternationalBusinessthispastassessmentperiod.Generallyspeaking,onelimitationofusingtheETSBusinessMFTtomeasurebusinessknowledgeisthatthequestionsonthetestmaynotberepresentativeofwhatactuallyistaughtinourcurriculum,thereforestudentsarebeingtestedonmaterialthattheywerenotrequiredtolearninourprogram.ItmaybehelpfultoevaluatethequestionsontheBusinessMFTandexcludethosequestionsthatarenotapplicabletoobtainamoreaccuratescoreforassessmentpurposes.Anotheralternativewouldbetodevelopourownbusinessknowledgecontenttest,asotherschoolshave,toensurethatthequestionsarerepresentativeofwhatistaughtanddeemedimportanttoourprogram.


Trait MeanScore

Target(%) StudentsMeetingTarget(%)

ProblemIdentification 3.03 135(100)

111(82)

EvaluatesQualityofEvidence

3.06 135(100)

103(76)

BusinessAnalysis 3.03

135(100)

94(70)

ConceptsandInterpretations

3.23 135(100)

95(71)

CommunicatesEffectively

3.03 135(100)

103(76)


PointofView 3.06 135(100)

111(82)

Totals 3.07 135(100)

103(76)

Overall,studentsperformedwellwhenmeasuredforcriticalthinkingabilitywith76%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe6criticalthinkingcriteriawas3.07(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsforcriticalthinkingability.However,ofthe6criticalthinkingareasmeasured,BusinessAnalysisandConceptsandInterpretationsseemedtobetheweakestabilities,with30%and29%ofthestudents(respectively)performingatalevelneedingimprovement.ItwouldseemthatstudentsareproficientintheircriticalthinkingabilitieswiththeexceptionoftheirabilitytoperformBusinessAnalysis(Doesthestudentundertakeappropriatequantitativeand/orqualitativeanalysis?)andintheareaofConceptsandInterpretations(Doesthestudentidentifyandaccuratelyexplain/usetherelevantkeyconcepts/followevidenceandreasoninordertoobtainthoughtful,logicalconclusionsand/orsolutions?).WeaknessintheareaofBusinessAnalysismaybecorrelatedtotheSubjectAssessmentscoresfromtheETSBusinessMFTresultsthatindicatedthatourstudentsperformedbelowthenationalmeaninareasthatwerequantitativelyrelated,specificallyAccounting,FinanceandQuantitativeAnalysis.ContinuedweaknessintheareaofConceptsandInterpretationsmaybeduetocoursecurriculumindicateaneedforadditionalinstructioninthatsubjectarea.


Trait

Mean

Superior(%)

MeetsExpectations(%)

NeedsImprovement(%)

EthicalSelfAwareness

3.82

52(76)

16(24)

0

UnderstandingofEthicalPerspectives

3.71

48(71)

19(28)

1(1)

EthicalIssueRecognition

3.80

51(75)

17(25)

0


ApplicationofChristianEthicalPerspectives

3.36

29(43)

34(50)

5(7)

EvaluationofDifferentEthicalPerspectives

3.45

32(47)

33(48)

3(5)

Totals

3.63

41(60)

24(35)

3(5)

SummaryofData:Overall,studentsperformedwellwhenmeasuredforethicalreasoningabilitywith93%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe5ethicalreasoningcriteriawas3.63(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsforethicalreasoningability.Resultsforthisperiodwereverysimilartothepreviouscollectionperiod.Analysis:TheresultsindicatethatstudentsareproficientintheirethicalreasoningabilityandtheirabilitytoApplyChristianEthicalPerspectives(StudentcanindependentlyapplyChristianethicalperspectives/conceptstoanethicalquestion,accurately,andisabletoconsiderfullimplicationsoftheapplication).TheseresultsmayindicatethatachangeintherequirementtoapplyChristianethicalperspectivesandconceptstoimprovetheclarityoftheassignmentparametershaveconsistentlyenabledstudentstomoreadequatelyaddressthiscomponentinthesignatureassignment.


Trait

Mean

Superior(%)

MeetsExpectations(%)

NeedsImprovement(%)

CreatesaClearCentralAssertion

3.79 52(76) 16(24) 0


DemonstratesStrongArgumentation&ReasoningSkills

3.55 37(55) 29(43) 2(2)

ChoosesSignificantSourceMaterial

3.38 31(46) 32(47) 5(7)

DevelopsLogicalCoherenceandFlow

3.71 47(69) 21(31) 0

AnalyzesandIntegratesSourceMaterial

3.33 25(37) 40(59) 3(4)

UsesEngaging&ClearLanguage

3.75 50(74) 16(24) 2(2)

UsesCorrectGrammar&Style

3.48 41(60) 18(26) 9(14)

Totals

3.57

40(59)

24(35)

4(6)

SummaryofData:Overall,studentsperformedwellwhenmeasuredforwrittencommunicationabilitywith92%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe7writtencommunicationcriteriawas3.57(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsforwrittencommunicationability.Resultsforthisperiodwereverysimilartothepreviouscollectionperiod.Overall,studentsperformedwellwhenmeasuredfororalcommunicationabilitywith94%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe5oralcommunicationcriteriawas3.65(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsfororalcommunicationability.Resultsforthisperiodwereverysimilartothepreviouscollectionperiod.Analysis:StudentsaremoreproficientintheirwrittencommunicationabilitywithimprovementintheirabilitytoChooseSignificantSourceMaterial(choosesimportantsourcesforreference&Incorporatessupport,detail,orresearchinsightfullyandeffectively)andAnalyzeandIntegrateSourceMaterial(insightfullyanalyzessourcematerial&demonstratesmasteryoveritseffectiveincorporationtobuildargument).TheseresultsindicatethatadditionalinstructioninChoosingSignificantSourceMaterialandAnalyzingandIntegratingSourceMaterial,mayhaveimprovedtheirconsistentabilitytodoso.ImprovementsintheCorrectUseofGrammarandStylemayhavealsoimprovedduetotheadditional


instruction;however,thereisroomforimprovementinthisarea.StudentsseemproficientintheiroralcommunicationabilityandhaveimprovedtheirabilitytoUseSupportMaterial(useofavarietyoftypesofsupportingmaterials{explanations,examples,illustrations,statistics,analogies,quotationsfromrelevantauthorities}makeappropriatereferencetoinformationoranalysisthatsignificantlysupportsthepresentationorestablishesthepresenter'scredibility/authorityonthetopic)onaconsistentbasis.ItseemsthatbetterinstructionintheuseofSourceMaterialduringoralpresentationshasproducedconsistentlybetterresultsinthisarea.

Trait

Mean

Superio

r (%)

Meets

Expectations (%)

Needs

Improvement (%)

Organization

3.82

55 (81)

13 (19)

0

Language

3.79

54 (79)

14 (21)

0

Delivery

3.79

52 (76)

16 (24)

0

Supporting Material

3.35

23 (35)

43 (63)

2 (2)

Central Message

3.50

38 (56)

29 (43)

1 (1)

Totals

3.65

43 (63)

23 (35)

2 (2)

SummaryofData:Overall,studentsperformedwellwhenmeasuredfororalcommunicationabilitywith94%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe5oralcommunicationcriteriawas3.65(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsfororalcommunicationability.Resultsforthisperiodwereverysimilartothepreviouscollectionperiod.Analysis:


StudentsseemproficientintheiroralcommunicationabilityandhaveimprovedtheirabilitytoUseSupportMaterial(useofavarietyoftypesofsupportingmaterials{explanations,examples,illustrations,statistics,analogies,quotationsfromrelevantauthorities}makeappropriatereferencetoinformationoranalysisthatsignificantlysupportsthepresentationorestablishesthepresenter'scredibility/authorityonthetopic)onaconsistentbasis.ItseemsthatbetterinstructionintheuseofSourceMaterialduringoralpresentationshasproducedconsistentlybetterresultsinthisarea.


Trait Mean Target(%) StudentsMeetingTarget

(%)ContributestoTeamMeetings

3.24 68(100)

64(94)

FacilitatestheContributionsofTeamMembers

3.27 68(100)

63(93)

IndividualContributionsOutsideofTeamMeetings

3.33 68(100)

64(94)

FostersConstructiveTeamClimate

3.29 68(100)

64(94)

RespondstoConflict 3.33 68(100)

64(94)

Totals 3.29 68(100)

64(94)

Overall,studentsperformedverywellwhenmeasuredforteamworkabilitywith96%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe5teamworkcriteriawas3.29(4pointscale)indicatingthatstudentsfarexceededtheSBMexpectationsforteamworkability.Onceagainitwouldappearfromthedatathatthestudentsexcelledineverymeasureofteamworkability,however,itisquitepossiblethatthescoreswereinflatedduetothefactthatthestudentswereaskedtoevaluatetheirpeersandmayhavebeengenerousintheirassessment(thinkingthatthescoresmayadverselyaffecttheirfellowstudent’sgrade).



Trait Mean Target(%) StudentsMeetingTarget(%)

DemonstratesanUnderstandingofFactorsAffectingtheGlobalizationofBusiness

3.06 77(100)

73(95)

DemonstratesanUnderstandingofInternationalTrade

2.16 77(100)

65(84)

Analyzes&SolvesForeignExchangeBusinessProblems

3.06 77(100)

74(96)

EvaluatesDifferentModesofInternationalBusiness

2.78 77(100)

72(94)

MakesSoundDecisionsRegardingFunctionalAreasofInternationalBusiness

2.62 77(100)

73(95)

Totals 2.74 77(100)

71(92)

Overall,studentsperformedadequatelywhenmeasuredforglobalawarenesswith92%ofthestudentsmeetingorexceedingexpectations.Meanscoreforthe5globalawarenesscriteriawas2.74(4pointscale)indicatingthatoverall,studentsmettheSBMexpectationsforglobalawareness.MarkedimprovementwasshowninUnderstandingofInternationalTrade,with84%ofthestudentsmeetingorexceedingexpectations,versus12%fromtheprioryear.EvaluatingDifferentModesofInternationalBusinessandMakingSoundDecisionsRegardingFunctionalAreasofInternationalBusinessandwerealsogreatlyimprovedovertheprioryear,with94%and95%(respectively)ofthestudentsmeetingorexceedingexpectations,versus34%and63%(respectively).


ItwouldseemthatstudentsdemonstrateanadequateunderstandingofglobalawarenesswiththepossibleexceptionoftheabilitytoDemonstrateanUnderstandingofInternationalTrade(anunderstandingofinternationaltradeflowsandappliesthisunderstandingtobusinessdecisionsintheinternationalcontext{InternationalTradeTheory,ForeignDirectInvestment,RegionalEconomicIntegration}).Whilethenumberofstudentsmeetingorexceedingexpectationshasimprovedinthisaspectofglobalawareness,continuedchangestothecurriculuminBUSI370InternationalBusinesswouldlikelyhavethegreatestimpactinimprovingtheunderstandingofinternationaltrade.

SummaryofResultsfromImplementingIndirectMeasuresofStudentLearning:

1. SummaryofResultsforIndirectMeasure1:Nodatacollectedforthisassessmentperiod.


SummaryofAchievementofIntendedStudentLearningOutcomes:

IntendedStudentLearningOutcomes LearningAssessmentMeasures

GeneralProgramISLOs

DirectMeasure1

DirectMeasure2

DirectMeasure3

DirectMeasure4

DirectMeasure5

DirectMeasure6

IndirectMeasure1

IndirectMeasure2

PerformanceTargetWas…








1. ProgramLearningOutcome1 Met NA NA





6. ProgramLearningOutcome6 NotMet NA NA

ProposedCoursesofActionforImprovementinLearningOutcomesforwhichPerformanceTargetsWereNotMet:

1. CourseofActionforCoreISLO1: 1)ContinuetorevisetheBusinessCorecurriculumtoprovideopportunitiesforreviewandretentionofkeyaccounting,economicsandfinanceconcepts. 2)Begintestingstudentsatthepointoflearningtodetermineifadequateformativelearningisoccurring.Ifnot,adjustcoursecurriculumtoimprovecomprehensionandability. 3)DevelopandimplementaBusinessContentTesttomoreaccuratelymeasurethebusinessknowledgetaughtintheSBM.Portionsofthetest


shouldbeadministeredthroughoutthedegreeprogramaswellasatthecompletionofthedegreeprogramtomeasurebothformativeandsummativesubjectspecificlearning.

2. CourseofActionforCoreISLO2:1)Considerrequiringaccounting,financeandquantitativeanalysistutorialsbeforeenrollinginBUSI450StrategicManagement.2)ContinuetorevisetheBusinessCorecurriculumtoprovideopportunitiesforreviewandretentionofkeyaccounting,financeandquantitativeanalysisconcepts.3)ContinuetoreviseBUSI450coursecurriculumtoensureadequatecoverageofBusinessAnalysisandConceptsandInterpretationsprinciplesforuseinmanagementsimulations.

3. CourseofActionforCoreISLO3:1)ContinuetoincludeaspecificrequirementtoapplyChristianethicalperspectives/conceptsandincludeinstructiontoensureproperapplicationasapartoftheBUSI496SeniorSeminar:BusinessEthicscoursecurriculum.2)ContinuetoreviewandreviseBusinessCorecurriculumtoincluderequirementsandinstructiontoapplyChristianethicalperspectivesandconceptsinothercoursesinthedegreeprogram.

4. CourseofActionforCoreISLO4:1)ContinuetosupplementtheBUSI496SeniorSeminar:BusinessEthics’coursecurriculumwithalibraryresearchmoduletaughtbyuniversityresearchlibrarians.2)ContinuetosupplementtheBUSI496SeniorSeminar:BusinessEthics’coursecurriculumwithassistancefromtheUniversityWritingCenter.3)ConsidertheadditionoftheBUSI405BusinessReportWritingcourse,whichincludescurriculumtodeveloporalcommunicationskills,totheBusinessCoreasarequiredcourse.

5. CourseofActionforCoreISLO5:Reevaluatetheteamworkmeasureandtheadministrationoftheinstrumenttoensureanaccurateassessmentofthestudents’teamworkability.Makechangesasappropriate.

6. CourseofActionforCoreISLO6:Continuetoincludeadditionalinstruction,assignmentsandexercisesfocusingonDemonstratinganUnderstandingofInternationalTradeintheBUSI370InternationalBusinesscurriculum.

StudentLearningAssessmentforMBADegree

GeneralProgramIntendedStudentLearningOutcomes

1. CrossDisciplinaryIntegration


2.GlobalPerspective

3. CriticalThinking

4. Character

5. Communication

6. Collaboration


1. LearningOutcome1:Todemonstrateknowledgeofmultiplebusinessdisciplines,including,accounting,finance,marketing,management,andstrategicintegration.

2. LearningOutcome2:Todevelopbusinessstrategiesthatrespondtoemergingopportunitiesandchallengesintheglobalenvironment.

3. LearningOutcome3:Toquicklyandaccuratelyidentifyandanticipatevalidbusinessproblems/opportunitiesusinganalytical,quantitative,andcriticalthinkingskills.

4. LearningOutcome4:Todemonstratetheabilitytogivevoicetoanddefendpersonalvalues.

5. LearningOutcome5:Tocreateanddeliverprofessionaloralbusinesspresentations.

6. LearningOutcome6:Todeveloptheskillsnecessarytosuccessfullyleadandcontributetoateaminadynamiccompetitiveenvironment.


1. ETSMBATest Objective(Target/Criterion)forDirectMeasure1:StudentswillperformbetterthantheNationalAverage(Mean%correct)ontheETSMBATest.GeneralProgramISLOsAssessedbythisMeasure:1

2. CaseStudyAnalysis(BUSI548) Objective(Target/Criterion)forDirectMeasure2:80%ofstudentswilleithermeetorexceedexpectationsaccordingtotheGlobalPerspectiverubric.


3.MarketAnalysis(BUSI527) Objective(Target/Criterion)forDirectMeasure3:75%ofstudentswilleithermeetorexceedexpectationsaccordingtotheCriticalThinkingrubric.


4. LeadershipReflectionPaper(BUSI581) Objective(Target/Criterion)forDirectMeasure4:80%ofstudentswilleithermeetorexceedexpectationsaccordingtotheCharacterrubric.



5.MarketingResearchPresentation(BUSI515)GeneralProgramISLOsAssessedbythisMeasure:5

Objective(Target/Criterion)forDirectMeasure5:80%ofstudentswilleithermeetorexceedexpectationsaccordingtotheCommunicationrubric.

6. SimulationTeamsAnalysis(BUSI527)GeneralProgramISLOsAssessedbythisMeasure:6

Objective(Target/Criterion)forDirectMeasure6:75%ofstudentswilleithermeetorexceedexpectationsaccordingtotheCollaborationrubric.

AssessmentInstrumentsforIntendedStudentLearningOutcomes—IndirectMeasuresofStudentLearning: PerformanceObjectives(Targets/Criteria)forIndirectMeasures:


2. IndirectMeasure2:Employer’sSurvey Objective(Target/Criterion)forIndirectMeasure2:70%oftherespondentsreporteffectivepreparationforeachofthe6ProgramSLOareas.GeneralProgramISLOsAssessedbythisMeasure:1-6

AssessmentResults:MBADegree



Area Target(NationalMean)

ETSMBAResults

Marketing 57.4 54Management 58.5 53Finance 43.9 40Accounting 46.5 39StrategicIntegration 51.3 48


Studentsdidnotmeetourtargetforanyofthe5subjectareasontheETSMBAexam.Itshouldbenotedthatthesamplesizewas11andmaynotberepresentativeoftheentireprogram.Regardless,thereiscertainlyroomforimprovementineachofthesubjectareas.


Area Target ResultsIdentificationofglobalbusinessopportunities

80% 86%

Analysisofglobalbusinesschallenges

80% 89%

Developmentofglobalbusinessstrategies

80% 100%

Studentsperformedwellineachareaofglobalperspective.


Area Target Results

Situationassessment

75% 75%

Opportunityanalysis

75% 65%

Masterofdataandanalyticprocess

75% 80%

Overall,studentperformancehasslightlyimprovedoverlastyear.Whiletargetmetintwoareas,35%ofstudentswerebelowexpectationin‘OpportunityAnalysis.’



Area Target ResultsUtilizationoffaithbasedcontent

80% 93%

FormulationofChristianworldview

80% 93%

Applicationoffaith-basedconcepts

80% 100%

Studentsperformedwellineachareaofcharacterandfaithintegration.


Area Target Results

Introduction 80% 86%Enthusiasm 80% 71%Visualaid 80% 95Conclusion 80% 79%Professionalismofpresentation

80% 79%

TheclasswasbelowthetargetinProfessionalismofpresentationandConclusion.Thelowenthusiasmscoresresultedfromtheclasstakingtheoralpresentationlessseriouslythanthereportandexamthatwasalsodue.Nonetheless,itwasoverallacceptable(82%).


Area Target Results

Goalandactionoriented

75% 75%


Conflictmanagement

75% 85%

Substantivecontributor

75% 85%

Likelastyear,studentscontinuetodowellreflectingonmissedleadershipopportunities.Atthesametime,someofthemcontinuetostrugglewithactionableideasforimprovingboththeirinterpersonalteamdynamicsandtranslatingwhattheyknowaboutleadershipintoaction.Studentscontinuetodoanicejobreflectingonopportunitiestoimprovetheirleadershipbehaviors.






GeneralProgramISLOs

DirectMeasure1

DirectMeasure2

DirectMeasure3

DirectMeasure4

DirectMeasure5

DirectMeasure6

IndirectMeasure1

IndirectMeasure2
















1. CourseofActionforCoreISLO1: WehaveshiftedtheinstructorsforourMBAcoursefromprimarilyadjunctfacultytoprimarilyfull-timefacultytoensurecontinuityofcontentandexpectations.Plansareunderwaytoimplementapre-testandpost-testmodeltoidentifyareasforimprovementearlierintheprogram.In


addition,planstoincreasethenumberofstudentstakingtheETSMBAexamhavebeenproposed.

2. CourseofActionforCoreISLO2:Nochanges

3. CourseofActionforCoreISLO3:ToincreaseprogramrepresentationintheMBAassessmentprocess,wehavemovedthemeasurementofthisgoaltoBUSI521ManagerialEconomicsforthe2014-2015academicyear.

4. CourseofActionforCoreISLO4:WehavedroppedmeasuringthisgoalinBUSI509WorldviewLeadershipFormation,asitsretreatstylemadeitsuboptimaltomeasurethisgoal.Weareaddinga360degreeassessmentofindividualstudentstoestablishafullerpictureofwherestudentsfallshortinsatisfyingthisgoal.

5. CourseofActionforCoreISLO5:Nochanges

6. CourseofActionforCoreISLO6:Wemetanddiscussedwhereconcreteskillsfordiagnosingandresolvinginterpersonalconflictandteamleadershipcanbefurtherinstilledintheprogram.WedecidedtoaddnewcontentforhighperformanceteamsandbestpracticesforcollaborationtoBUSI516OrganizationalBehaviorandBUSI581StrategicLeadership.

StudentLearningAssessmentfortheMastersofManagementDegree

GeneralProgramIntendedStudentLearningOutcomes

1. CrossDisciplinaryCompetence

2. AnalyticalandCriticalThinking/Deciding

3. Character

4. Communication

5. Collaboration


1. LearningOutcome1:Studentswilldemonstrateknowledgeandapplicationofmanagementtheories,conceptsandpractices.


2. LearningOutcome2:Studentswillrecommendstrategicsolutionstoambiguousbusinessproblemsusinganalyticalandcriticalthinkingskills.

3. LearningOutcome3:StudentswillexhibitanunderstandingofleadershipandethicsbasedonaChristianworldview.

4. LearningOutcome4:Studentswillcomposeprofessionalwrittenbusinesscommunicationsthatareclear,concise,andcompelling.Studentswillcreateanddeliverprofessionaloralbusinesspresentations.

5. LearningOutcome5:Studentswillexhibittheabilitytoengageinacollaborativeeffort.


1. AppliedResearchProject(MGMT570) Objective(Target/Criterion)forDirectMeasure1:80%ofthestudentswillmeetorexceedexpectationsineachCompetencytraitarea.


2. ResearchReport(MGMT597) Objective(Target/Criterion)forDirectMeasure2:80%ofstudentswilleithermeetorexceedexpectationsineachCriiticalThinkingtraitarea.


3. LeadershipPaper(MGMT581)) Objective(Target/Criterion)forDirectMeasure3:80%ofthestudentswillmeetorexceedexpectationsineachCharactertraitarea.GeneralProgramISLOsAssessedbythisMeasure:3

4. ResearchReport(MGMT597) Objective(Target/Criterion)forDirectMeasure4::80%ofthestudentswillmeetorexceedexpectationsineachCommunicationtraitarea. GeneralProgramISLOsAssessedbythisMeasure:4

5. AppliedResearchProject(MGMT570)GeneralProgramISLOsAssessedbythisMeasure:5

Objective(Target/Criterion)forDirectMeasure5:80%ofthestudentswillmeetorexceedexpectationsineachCollaborationtraitarea.

AssessmentInstrumentsforIntendedStudentLearningOutcomes—IndirectMeasuresofStudentLearning:

PerformanceObjectives(Targets/Criteria)forIndirectMeasures:


2. IndirectMeasure2:Employer’sSurvey Objective(Target/Criterion)forIndirectMeasure2:70%oftherespondents


GeneralProgramISLOsAssessedbythisMeasure:1-6

reporteffectivepreparationforeachofthe6ProgramSLOareas.

AssessmentResults:MAMDegree


1. SummaryofResultsforDirectMeasure1:Trait: Target: Result:Knowledge 80% 68%

Comparison 80% 68%

Application 80% 59%

Ramification 80% 45%

Analysis 80% 50%

Insights:Theclassapproachedstandardsintheareaofknowledgeoftheories,conceptsandpractices(68%)andalsotheabilitytocompareandcontrastbetweensame(68%).However,studentsdidnotfareaswellintheareasofapplication(59%),understandingappropriatenessofapplication(45%),andanalyzing(50%).Basedonindirectassessment(observationandfeedback),studentsmaytakeMGMT570priortotakingthosecourseswhichprovidethefoundationalknowledgerequiredtocompletethiscourse.Studentswhoregisterforadvancedcourseswithouthavingtheproperfoundationarenotequippedwiththenecessaryskills,knowledge,andabilityrequiredandthereforenotpreparedfortherigorsofthiscourse.Basedonindirectassessment,anumberofstudentswhoenteredtheprogramwithlimitedordeficientacademicorbusinessfoundationwerenotabletomeetclassroomassignmentstandardsandfeltoverwhelmedandunequippedfortherigorsofthecourse.Somestudentsindicatedthiswasthefirsttimetheyencounteredacourserequiringthismuchrigor.

2. SummaryofResultsforDirectMeasure2:Trait: Target: Result:OpinionsandLogic

80% 83%

Viewpoint 80% 92%

BiasandJudgment

80% 83%

Choices 80% 75%


Insights:Theclass,asawhole,metstandardsbutfewexceededstandardsintheareaofaccuratelyidentifyingvalidbusinessproblems.Theweakestperformanceareaswerebias,judgment,andchoices.Studentsputforthlimitedchoices;theirwritingandresearchdidnotreflectmuchcreativityororiginality.Theclasswaswillingtodissect,diagnose,andanalyzeatthe"meetstandards"levelbuttherewasnoindicationofcriticalthinkingbeyondthatlevel.

3. SummaryofResultsforDirectMeasure3:Trait: Target: Result:IdentificationofPersonalLeadership

80% 55%

Structure&CommunicatingwithClarity

80% 66%

EvidenceofDepthofThought

80% 77%

Insights:Thestudentsinthisclassrepresentedawiderangeofability.Overall,thisgroupwasmorediversethanusualincompetencelevelsandabilitytograsptheconceptualprinciplesassociatedwithgraduatelevelstudy.Moreover,theentiregrouptendedtobelessverbalandexpressive.Agreatpercentagelackedtheinitiativeandincisivecompetencegenerallyassociatedwithgraduatelevelstudents.However,inperformanceafewstudentgroupsshowedgooddepthofthought,andabilitytointegrateprinciplesintotheircontext.Overall,anumberofstudentsappearedquiteincapableofengaginganddidnotachievealevelofintegrationexpected.Inparticular,writingskillswerequiteweakinsomestudents.Additionally,conceptualandthinkingskillswerealsoinadequate.Thedisparitybetweenthesuperiorstudentsandtheotherstudentswasthemostnoticeabletodate.

4. SummaryofResultsforDirectMeasure4:Trait: Target: Result:Nonverbalskills 80% 65%

Enthusiasm 80% 64%

Intro 80% 54%

Clarification 80% 57%

Time 80% 85%


Visuals 80% 77%

Professionalism 80% 58%

Interesting 80% 74%

Insights:Themajorityoforalpresentationsweremiddleoftheroad(average).Thetwoweakestareasweretheintroductionandconclusion.Thestrongestareasweretime,professionalism,visuals,andinteresting.Withthelater,studentsdidverywellwithPPTorPrezitomakethepresentationinteresting.Moreworkisneededonimprovingscoresfromaveragetoaboveaverage.Themajorityoforalpresentationswerebelowaverageonallcategoriesexcepttime.Thesestudentshadbeentrainedandassessedpreviouslyinthisareawiththesamerubric.Theissuewasthetopic.Itisanacademic,quantitativetopic.Nonetheless,theyareexpectedtomakeitinterestingandengaging,thatdidn'thappen.Moreover,studentsindicatedthisisthefirsttimetheyrequiredthismuchrigorintheircourse.

Trait: Target: Result:

ProblemStmt 80% 75%

LitEvaluation 80% 54%

Methods 80% 65%

Results 80% 71%

Summary 80% 60%

Organization 80% 80%

Tone 80% 81%

SentStructure 80% 63%

WordChoice 80% 57%

WritingMechanics 80% 78%

Length 80% 100%

APA 80% 60%

UseofRefs 80% 75%

Insights:Themeatofthepaper--literaturereview,methods,results,andsummary--werefairlyweak.Thisisahardpapertowrite(similartoathesis);however,graduatestudentsareexpectedtowriteatthislevel.Oftenwritingskillsweresimple,choppy,orwordy.Theresearchwasadequatebutrequiredlotsofhandholding.Itdidn'tseemtheycoulddoitontheirownwithouthelp.Studentsneedhelpfindingcurrentquality,scholarlyreferences.Theystruggledwiththisinclass.

Theweakestareas--literaturereviewsandqualityofreferences--gohand-in-hand.Studentsshowedalackofunderstandinginhowtodoqualityresearch--


specificallyfindingtopqualityjournalarticlesusinglibraryresources.Writingmechanics(grammar,punctuation),sentencestructure,andwordchoicewereallbelowstandards.

5. SummaryofResultsforDirectMeasure5:Trait: Target: Result:Communication 80% 68%

PositiveAttitude 80% 91%

Contribution 80% 64%

Performance 80% 64%

Skills 80% 32%

Attitude 80% 32%

Leadership 80% 32%

Insights:Themajorityofstudentsareexperiencedwithgroupworkandcommunicatingthroughdifferentandvariouscommunicationchannels.However,themajorityofthegroupsdidnotperformaswellaspossibleduetolackofskillinraisingandresolvingconflict.Infact,thelackofabilitytoraiseandresolveconflictwaspoor(32%).Inconjunction,theattitudeinconflictresolutionwasalsopooroverall(32%).Additionally,theleadershipabilityinunderstandingandmediatingconflictwaspoor(32%).Interestingly,manyofthestudentsseemedunawareofbasicgroupdynamicsandconflictmanagementtheories,concepts,andpractices.






GeneralProgramISLOs

DirectMeasure1

DirectMeasure2

DirectMeasure3

DirectMeasure4

DirectMeasure5

DirectMeasure6

IndirectMeasure1

IndirectMeasure2
















1. CourseofActionforCoreISLO1: Herearethefollowingrecomendations:• IntroducecoursesequencingasrecommendationbytheMAMcommittee.• Introduceconceptsofapplicationinpriorcoursework(somewhereinthecoursecontentduringtheprogram).Thiswouldincludehowto

analyzeprosandcons,understandtheramificationsofmodels,theories,concepts,etc.,andapplication.

2. CourseofActionforCoreISLO2:Herearethefollowingrecomendations:• IntroducecoursesequencingasrecommendationbytheMAMcommittee.• Requirecourseworktobringconditionalstudentsuptoseniorlevelundergraduateacademicexpectationsinskills,knowledgeandabilities.Provideassistanceforstudentsneedingbasicskillsinbusinesscommunicationsandbasicmanagementknowledge.

3. CourseofActionforCoreISLO3:Herearethefollowingrecomendations:• Requirecourseworktobringconditionalstudentsuptoseniorlevelundergraduateacademicexpectationsinskills,knowledgeandabilities.• Provideassistanceforstudentsneedingbasicskillsinbusinesscommunicationsandbasicmanagementknowledge.

4. CourseofActionforCoreISLO4:Thefollowingareprogrammaticrecomendations:

• RequirestudentstocompleteaWrittenCommunication(includingpresentations)Orientationpriortoenteringtheprogram.Orientationshouldincludeinformationanddemonstrationofknowledgeofwritingandpresentingskills.

DuringOrientationconsidervideotapingstudentssotheycangetfeedbackontheirpresentation.Thismayhelpthemtobetterunderstandwheretheyarefallingshort.• Considerhavingcontinuityinprofessorassignments.• IntroducemorescholarlytyperesearchpaperswithAPAformattingthroughouttheprogram.

• Consideramandatorylibraryorientationandresearchorientationforincomingstudents.Thiswouldincludeunderstandinghowtoresearch,locatescholarlysources(beyondGoogle),APA,useofreferences,andotherresearchedorientedandwritingskillareas.


• Ensurethatallstudentsmeetbasicadmissionstandards.• MAMProgramCommitteetoroutinely(quarterly)reviewtheprogressofconditionalstudents.Producecourseworktobringconditionalstudentsuptoexpectations.

5. CourseofActionforCoreISLO5:Herearethefollowingrecomendations:• MAMCommitteerecommendscourseassignmentsbemadewithProgramChairandCommitteetoensurecoursesareassignedtoqualified

instructors.• Considerreviewingassignmentsinallcoursestoensureadequatecoverageoftopics.Ensurethatsyllabicoverallrequiredtheories,

conceptsandpracticesasoutlinedbytheMAMProgramCommittee.MovethecollectionofcollaborationdatatoMGMT561.Considervideotapingthestudentsotheycangetfeedbackontheirgroup,conflict,andfeedbackskills.Thismayhelpthemtobetterunderstandwheretheyarefallingshort.

Documents

Report of Student Learning and Achievement Azusa Pacific ... · Report of Student Learning and Achievement Azusa Pacific University School of ... Student Learning Assessment for