Embed Size (px)
Citation preview
1CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Growing an Growing an Economical Economical
Student Admin Student Admin DatamartDatamart
Using Common Using Common ToolsTools
Jim JanossyJim JanossyRuss PattersonRuss Patterson
DePaul UniversityDePaul UniversityInfo ServicesInfo ServicesCUMREC 2002CUMREC 2002
©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Copyright © 2002 DePaul University. This work is the intellectual property of DePaul University. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from DePaul University, Office of the President, Chicago, IL 60604.
2CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
DePaul DePaul Adhoc Adhoc
Student Student DatamartDatamart
Jim JanossyJim JanossyRuss PattersonRuss Patterson
DePaul UniversityDePaul UniversityInfo ServicesInfo ServicesCUMREC 2002CUMREC 2002
©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
3CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
DePaulDePaul University University
• Chicago & suburbs, 6 campusesChicago & suburbs, 6 campuses
• BA/BS, MA/MS, Ph.D, CE programsBA/BS, MA/MS, Ph.D, CE programs
• 8 colleges8 colleges
• 20,600 students (12,400 U, 8,200 G)20,600 students (12,400 U, 8,200 G)
• 3,500 employees (1,800 F, 1,700 S)3,500 employees (1,800 F, 1,700 S)
4CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
PeopleSoftPeopleSoftat DePaulat DePaul
• HR, Finance, SA 7.6 systemsHR, Finance, SA 7.6 systems
• Replaced homegrown SA, SCT / IAReplaced homegrown SA, SCT / IA
• Implementation began 1999Implementation began 1999
• Full cutover done July 1, 2001Full cutover done July 1, 2001
• HP 9000 servers, HP-UX, OracleHP 9000 servers, HP-UX, Oracle
5CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
PeopleSoftPeopleSoftdata loadingdata loading
• Student data loaded, 1981 forward.Student data loaded, 1981 forward.
• Loaded June 2001, still reconciling!Loaded June 2001, still reconciling!
• STDNT_ENRL: STDNT_ENRL: 2.8 million rows2.8 million rows
• STDNT_CAR_TERM: STDNT_CAR_TERM: 1.5 million rows1.5 million rows
• TERM_TBL: TERM_TBL: 109 terms to present109 terms to present
6CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
PeopleSoftPeopleSoftdata transitiondata transition
• Old system data loaded to “ODS”Old system data loaded to “ODS”
• Old data available through ODBCOld data available through ODBC
• Reconciliation accesses old dataReconciliation accesses old data
• Old IBM mainframe out July 2001Old IBM mainframe out July 2001
• Economize on old license feesEconomize on old license fees
7CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
PeopleSoft SAPeopleSoft SA usage at DePaul usage at DePaul
• Approx 980 on-line admin usersApprox 980 on-line admin users
• 338 PS-Query users338 PS-Query users
• 81,000 CampusConnect user ids81,000 CampusConnect user ids
• 70 power users (data extraction)70 power users (data extraction)
• 10 subject matter experts (SME)10 subject matter experts (SME)
8CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reporting Reporting instanceinstance
PeopleSoft Student PeopleSoft Student Admin system Admin system
9CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reporting Reporting instanceinstance
PeopleSoft Student PeopleSoft Student Admin systemAdmin system
Reporting Reporting instance is ainstance is afull copyfull copy of ofthe on-linethe on-linesystemsystem
10CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reporting Reporting instanceinstance
PeopleSoft Student PeopleSoft Student Admin systemAdmin system
Copied Copied nightlynightly
11CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reporting Reporting instanceinstance
PeopleSoft Student PeopleSoft Student Admin systemAdmin system
Why?Why?* Efficiency* Efficiency* Static data* Static data* Security* Security* Preservation* Preservation
Copied Copied nightlynightly
PRD1PRD1PRD0PRD0
12CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reporting Reporting instanceinstance
PeopleSoft Student PeopleSoft Student Admin systemAdmin system
PS-Query and PS-Query and most reporting most reporting runs from the runs from the reporting reporting instanceinstance
PRD1PRD1
13CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Data accessData access
• On-line is PS-providedOn-line is PS-provided
• Reporting side is do-it-yourself!Reporting side is do-it-yourself!
• 130 old reports abandoned130 old reports abandoned
• 70 old report outputs 70 old report outputs still relevantstill relevant!!
• Adhoc extractions Adhoc extractions still relevantstill relevant!!
14CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
DePaul DePaul infrastructureinfrastructure
IS:IS: systems, prod reports, ad hocs systems, prod reports, ad hocs
Enrollment Mgmt/Registrar:Enrollment Mgmt/Registrar: university-wide day/day reportinguniversity-wide day/day reporting
Office of Institutional Planning and Office of Institutional Planning and Research (OIPR):Research (OIPR): long term stats long term stats
15CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
College office College office reporting?reporting?
Whatever you Whatever you can’tcan’t get from get from
• • production reporting (IS)production reporting (IS)• PS online panels• PS online panels• Enrollment Management• Enrollment Management• Institutional Planning . . .• Institutional Planning . . .
16CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
...get for yourself?...get for yourself?
PS-QueryPS-Query 338 users338 users
• spreadsheet outputspreadsheet output
• easy to learn tooleasy to learn tool
• feed Crystal Reports or Accessfeed Crystal Reports or Access
17CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Serious LimitationsSerious Limitations
• spreadsheet maximum 65K rowsspreadsheet maximum 65K rows
• hard to learn data structures!hard to learn data structures!
• awkward outer joinsawkward outer joins
PS-QueryPS-Query 338 users338 users
18CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Serious LimitationsSerious Limitations
• spreadsheet maximum 65K rowsspreadsheet maximum 65K rows
• hard to learn data structures!hard to learn data structures!
• awkward outer joinsawkward outer joins
PS-QueryPS-Query 338 users338 users
19CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Fatal LimitationsFatal Limitations
• must work with must work with raw tablesraw tables
• views aren’t enoughviews aren’t enough of a cure of a cure
• not enough capabilitiesnot enough capabilities
PS-QueryPS-Query 338 users338 users
20CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Direct data accessDirect data access
ODBCODBCraw PS tablesraw PS tables
22 users22 users
Registrar, institutional Registrar, institutional planning, programmer/planning, programmer/analyst accessanalyst access
Adhoc reports done Adhoc reports done by Info Servicesby Info Services
Steep learning curve!Steep learning curve!
21CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Data access goalData access goal
Easy to useEasy to useStable data formatsStable data formatsStandard sourceStandard sourceUsers do most adhocsUsers do most adhocs
PS-QueryPS-Query 338 users338 users
?ODBCODBC
raw PS tablesraw PS tables22 users22 users
?
22CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Data access goalData access goal
Easy to useEasy to useStable data formatsStable data formatsStandard sourceStandard sourceUsers do most adhocsUsers do most adhocs
PS-QueryPS-Query 338 users338 users
?ODBCODBC
raw PS tablesraw PS tables22 users22 users
?
23CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Data access goalData access goal
Easy to useEasy to useStable data formatsStable data formatsStandard sourceStandard sourceUsers do most adhocsUsers do most adhocs
PS-QueryPS-Query 338 users338 users
?ODBCODBC
raw PS tablesraw PS tables22 users22 users
?
24CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Data access goalData access goal
Easy to useEasy to useStable data formatsStable data formatsStandard sourceStandard sourceUsers do most adhocsUsers do most adhocs
PS-QueryPS-Query 338 users338 users
?ODBCODBC
raw PS tablesraw PS tables22 users22 users
?
25CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Reaching the goalReaching the goal
ODBCODBCDatamart tablesDatamart tables
70 users70 users
PS-QueryPS-Query 338 users338 users
ODBCODBCraw PS tablesraw PS tables
22 users22 users
26CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Adhoc student Adhoc student datamartdatamart
How does it look?How does it look?
How do users user it?How do users user it?
27CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
PeopleSoft operator PeopleSoft operator id and passwordid and password
28CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
UsageUsage
29CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
StudentStudent datamart datamart functional functional looklook
• 20 tables20 tables
• Enrolled student dataEnrolled student data
• Assemble data for convenient useAssemble data for convenient use
• Focus on 20% to meet 80% needsFocus on 20% to meet 80% needs
• Row control by ACAD_GROUPRow control by ACAD_GROUP
30CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Student datamart Student datamart technicaltechnical look look
• Separate Oracle databaseSeparate Oracle database
• Oracle define, create, securityOracle define, create, security
• Extract data with Access or SQRExtract data with Access or SQR
• Load with SQR or SQL*LoaderLoad with SQR or SQL*Loader
• Pilot first, engage users, perfect itPilot first, engage users, perfect it
31CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Student datamart Student datamart technicaltechnical look look
• Separate Oracle databaseSeparate Oracle database
• Oracle define, create, securityOracle define, create, security
• Extract data with Access or SQRExtract data with Access or SQR
• Load with SQR or SQL*LoaderLoad with SQR or SQL*Loader
• Pilot firstPilot first, engage users, perfect it, engage users, perfect it
32CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Student datamart Student datamart technicaltechnical look look
• Separate Oracle databaseSeparate Oracle database
• Oracle define, create, securityOracle define, create, security
• Extract data with Access or SQRExtract data with Access or SQR
• Load with SQR or SQL*LoaderLoad with SQR or SQL*Loader
• Pilot first, Pilot first, engage usersengage users, perfect it, perfect it
33CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Student datamart Student datamart technicaltechnical look look
• Separate Oracle databaseSeparate Oracle database
• Oracle define, create, securityOracle define, create, security
• Extract data with Access or SQRExtract data with Access or SQR
• Load with SQR or SQL*LoaderLoad with SQR or SQL*Loader
• Pilot first, engage users, Pilot first, engage users, perfect itperfect it
34CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Adhoc datamartAdhoc datamart- pilot- pilot
• Design for 600 most recent adhocsDesign for 600 most recent adhocs
• Pilot and use in AccessPilot and use in Access
• Focus onFocus on common problems common problems
• Roll out best contact data firstRoll out best contact data first
• Coordinate with mailing servicesCoordinate with mailing services
35CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
?
36CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Low- Low- hanging hanging fruit!fruit!
37CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Best Contact Data Best Contact Data Table FormationTable Formation
ADDRESSESADDRESSESNAMESNAMESPERSONAL_PHONEPERSONAL_PHONEEMAIL_ADDRESSESEMAIL_ADDRESSESHR: employeesHR: employeesCBORD: dorm assignsCBORD: dorm assigns
DP701ABest Contact
Data
38CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
FromCBORDQuery
Adhoc_Get_Best_Address
.mdb
PS_NAMES
370,000 rows
PS_PERSONAL_
PHONE366,000 rows
PS_EMAIL_
ADDRESSES182,000 rows
PS_ADDRESSES359,000 rows
Adhoc_Get_Best_Name
.mdb
DP150_BestName
305,000 rows
DP170_Phones
274,000 rows
DP160_BestEmail
76,000 rows
DP100_BestAddress
294,000 rows
Adhoc_Get_Phones
.mdb
Adhoc_Get_Best_Email
.mdb
Adhoc_DP701A_
Get_BestContactData
EMPLOYEES.XLS
MAILPREP.EXEFACSTAFF
.mdb3,902 rows
DP701A_BestContact
Data305,000 rowsROOM_
ASSIGNS.XLS
Adhoc_Get_DormRooms
.mdb
DP180_DormRooms2,560 rows
From HumanResources
Query
Formation of theDP701A_BestContactData Table
DePaul SA Adhoc Datamartn:\IAS_ADHOC_GROUP\SA_Datamart_Feeder_Backups\DP701A_Formation_colors.vsd J.Janossy 11/28/01 Rev.3 2/21/02
Counts of rows in tables are as of11/26/01 and are approximate, toestablish a perspective only OneOne
datamart datamart table!table!
39CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Adhoc datamartAdhoc datamart- engage users- engage users
• Get key users involved (SMEs)Get key users involved (SMEs)
• Publicize documentation sitePublicize documentation site
• Publish a schedule for more tablesPublish a schedule for more tables
• Use for a regular periodic processUse for a regular periodic process
• Use to replace other downloadsUse to replace other downloads
40CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Adhoc datamartAdhoc datamart- perfect it- perfect it
• Tune table format, contentTune table format, content
• Expand to admission dataExpand to admission data
• Convert loads to SQR, SQL*Loader Convert loads to SQR, SQL*Loader
• Absorb data from Enroll. Mgmt?Absorb data from Enroll. Mgmt?
• Expand longitudinallyExpand longitudinally
41CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
TABLENAME ROWCOUNTDP701A_BestContactData 339308DP701B_DePaulDegrees 64499DP701C_Enrolled 20595DP701D_AbleReg 42331DP701F_GPA 179049DP701G_Advisees 16029DP701H_VisaData 18885DP701J_CreditHoursSummary 31544DP701K_FirstLatestTerm 179049DP701N_PotentialGrads 4429DP701P_PersonalDemog 334268DP701R_Roster 59785DP701S_AllStudents 189667DP701Y_PSids_SSN 334305DP701Z_TermRef 307
TablesTables
42CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
RequirementsRequirements
• Clear documentationClear documentation
• Clear naming conventionClear naming convention
• Create simple data structuresCreate simple data structures
• Interpret coded values for useInterpret coded values for use
• Use common tools, common skillsUse common tools, common skills
43CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
RequirementsRequirements
• Clear documentationClear documentation
• Clear naming conventionClear naming convention
• Create simple data structuresCreate simple data structures
• Interpret coded values for useInterpret coded values for use
• Use common tools, common skillsUse common tools, common skills
44CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Column Description FormatA00_Seq Primary database key, just a
sequential number generatedat loading. This column has noinherent meaning for end-useraccess, and its use is notrecommended. (Indexes areseparately applied on othercolumns likely to be used injoin operations.)
Char 8
A01_Emplid Identifier from NAMES file. Char 11A01B_LDFlag Living or deceased flag, set to
"L" if the PERSONAL_DATA.DT_OF_DEATH column is null,but set to "D" if that columncontains data. Note: there issome inconsistency in howpersonnel are recording thefact that the person assignedan EmplID is deceased. Thecorrect way is for a date to beentered in DT_OF_DEATH,but in some cases the onlything that has been done is tomake all address fieldsinactive. Either methodremoves the addresses fromthe DP701A, but the correctDT_OF_DEATH indication alsocauses related phone numbersand e-mail address to beremoved.
Char 1
45CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
RequirementsRequirements
• Clear documentationClear documentation
• Clear naming conventionClear naming convention
• Create simple data structuresCreate simple data structures
• Interpret coded values for useInterpret coded values for use
• Use common tools, common skillsUse common tools, common skills
46CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
RequirementsRequirements
• Clear documentationClear documentation
• Clear naming conventionClear naming convention
• Create simple data structuresCreate simple data structures
• Interpret coded values for useInterpret coded values for use
• Use common tools, common skillsUse common tools, common skills
47CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
RequirementsRequirements
• Clear documentationClear documentation
• Clear naming conventionClear naming convention
• Create simple data structuresCreate simple data structures
• Interpret coded values for useInterpret coded values for use
• Use common tools, common skillsUse common tools, common skills
48CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
ApproachApproach
Common toolsCommon tools
Common skillsCommon skills
Common centsCommon cents
ODBC
MS-Access
SQR
SQL*Loader
Oracle roles
49CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Common toolsCommon tools
Common skillsCommon skills
Common centsCommon cents
Existing skills
MS-Access
Excel
Basic PC training
Views by college
ApproachApproach
50CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Common toolsCommon tools
Common skillsCommon skills
Common centsCommon cents
No new software
No new hardware
No new staff
No consultants
Got sense? Use it!
ApproachApproach
51CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
ResourcesResources
One analyst, 50% time +One analyst, 50% time +
One analyst, 20% time +One analyst, 20% time +
One student worker =One student worker =
8 months8 months
How do it faster? Start with a guideHow do it faster? Start with a guide
52CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Why?Why?
You need to know the data!You need to know the data!
You have needs todayYou have needs today
ETLs and warehouses are expensiveETLs and warehouses are expensive
Common tools do workCommon tools do work
You need to know the data, so start!You need to know the data, so start!
53CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
But, you need to:But, you need to:
Be organizedBe organized
Start with something manageableStart with something manageable
Show useful products earlyShow useful products early
Listen to users & the “adhoc guy”Listen to users & the “adhoc guy”
Know the dataKnow the data
54CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
How?How?
Get help from users, PS ERDsGet help from users, PS ERDsGet data access via ODBCGet data access via ODBCBuild own binder of data samplesBuild own binder of data samplesUse PS Query to build SQL in stepsUse PS Query to build SQL in stepsGet support of your DBAGet support of your DBAGet started!Get started!
55CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
Alternative Alternative approachapproach
• Build views instead of a datamartBuild views instead of a datamart
• Build extra tables with SQRBuild extra tables with SQR
• Define extra tables to PSDefine extra tables to PS
• Access with PS-Query or ODBCAccess with PS-Query or ODBC
• Problems: PS-skills, efficiencyProblems: PS-skills, efficiency
56CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
A deep thought...A deep thought...
If you do not require a budget, you If you do not require a budget, you are not constrained by the lack of are not constrained by the lack of one.one. - Jack Handy- Jack Handy
57CUMREC 2002CUMREC 2002©© 2002 Information Services at DePaul University 2002 Information Services at DePaul University
For more For more information...information...
Download a single zipped file from the web. Download a single zipped file from the web. This contains Word documents for sample This contains Word documents for sample formats, design spreadsheets, and SQR code:formats, design spreadsheets, and SQR code:
www.depaul.edu/~jjanossywww.depaul.edu/~jjanossy
Please understand:Please understand:The material is presented for your review “as is.” We The material is presented for your review “as is.” We are happy to discuss it, but we provide no support!are happy to discuss it, but we provide no support!
