NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research

Report Automation in Institutional Research

Richard Hetherington

Merima Babic

Courtney Raeford

temple.edu/ira

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 1

Table of Contents

NEAIR 2015 Presentation – Report Automation in Institutional Research ......... 2

Oracle Client Install Instructions ........................................................................................ 14

Visual Basic.NET Code ............................................................................................................. 16

Resources ...................................................................................................................................... 20

Visual Basic.NET ..................................................................................................................... 20

Structured Query Language (SQL) .................................................................................. 20

Excel ............................................................................................................................................. 20

Report Automation in Institutional Research

Rich Hetherington Associate Director

Merima Babic Research Analyst

Courtney Raeford Research Analyst

Data Analysis and Reporting

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Overview

Automation of cyclic reporting VB.net application Excel templates (VLOOKUP) Structured Query Language (SQL)

Process to our solution Demonstration of automated report Adaptability to IR offices

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Located in Philadelphia, Pennsylvania 18 Schools/Colleges 5 Professional Schools

9 Campuses (including Rome & Japan) 450+ Academic Programs 38,000+ Students Approximately 3,500 Faculty www.temple.edu/ira

Temple University At-A-Glance

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Volume of reports Consistency Resources

Goal: More time for

analytical research

Why Report Automation?

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Streamline and Automate Cyclic Reporting while having: Data consistency Process flexibility Self Sufficiency Transparency Scalability

Design Constraints

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Excel Output PDF Output Existing SQL code Ability to schedule Uncomplicated configuration Existing Excel Templates Minimal IT Unit support

Design Constraints

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

VB.net Application

(.EXE)Database

ConnectionSend SQL Request

Set resulting table to variable

Open Excel Template

Go to Worksheet (“Data”)

Paste Data Result

Refresh Report Worksheet

(VLOOKUPS)

Save Excel File Create and Save PDF

Send Notification

Windows Task Scheduler

Report Output

Automation Process

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Excel Template

Automation Process

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Demonstration

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Institutional Research & AssessmentNEAIR 2015: Report Automation in IRA

Scalability Scheduling Distribution Leverage Resources In-office skills Institutional Resources Online Resources

Efficiency Consistency

Overview

Temple At-A-Glance

Why Automation?

Design Constraints

Automation Process

Demonstration

Conclusion

Questions

Conclusion

Questions?Questions?

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 14

Oracle Client Install Instructions

In order to connect to the oracle environment via Visual Studio, the Oracle Data Access Components (ODAC) must be installed. The version used by the visual basic code provided, can be downloaded from: http://www.oracle.com/technetwork/topics/dotnet/utilsoft-086879.html

Once the ODAC client has been installed, add the reference to your project in Visual Studio. Under

Project, select Add Reference.

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 15

Navigate to the .NET tab and find the Oracle.DataAccess component – double click to add to you

project.

Next, return to the Add Reference dialog box, navigate to the COM tab, find the Oracle Data Control component – double click to add to your project.

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 16

Visual Basic.NET Code

1. Imports System.Text 2. Imports System.IO 3. Imports System.Xml 4. Imports Oracle.DataAccess.Client 5. Imports Oracle 6. Imports System.Net 7. Imports System.Windows.Forms 8. Imports System.Management 9. 'Must add References (.NET, Microsoft.Office.Interop.Excel) 10. Imports Microsoft.Office.Interop.Excel 11. Imports Microsoft.Office.Interop 12. Module Module1 13. 14. Sub Main() 15. Dim uglist1 As New ArrayList 16. Dim uglist2 As New ArrayList 17. 'Reads ini to get path and filename for sql, excel template file, Output filename and

output directory 18. Dim myini As String 19. Dim aPath As String 20. Dim aName As String 21. Dim line4 As String = "" 22. 'Determine the location of the ini file based on exe path 23. aName = System.Reflection.Assembly.GetExecutingAssembly. _ 24. GetModules()(0).FullyQualifiedName 25. aPath = System.IO.Path.GetDirectoryName(aName) 26. myini = aPath & "\Param.ini" 27. 28. Try 29. ' Open the file using a stream reader. 30. Using sr As New StreamReader(myini) 31. line4 = File.ReadAllText(myini) 32. End Using 33. Catch er As Exception 34. End Try 35. 'Parse out the components of the ini file and set to variables 36. Dim myparams As String = line4 37. Dim mypparts As String() = myparams.Split(";") 38. Dim mysqlin As String = mypparts(0) 39. Dim myxls As String = mypparts(1) 40. Dim myoutputfolder As String = mypparts(2) 41. Dim myfilename As String = mypparts(3) 42. Dim line As String = "" 43. '************************************** 44. Try 45. ' Open the sql file using a stream reader.

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 17

46. Using sr As New StreamReader(mysqlin) 47. line = sr.ReadToEnd() 48. End Using 49. Catch er As Exception 50. 'Console.WriteLine("The file could not be read:") 51. End Try 52. '************************************** 53. '**** OPEN ORACLE CONNECTION ********* 54. '************************************** 55. Dim orapass As String = My.Resources.ORA_PASS 56. Dim oradb As String = "Data Source=(DESCRIPTION=" _ 57. + "(ADDRESS=(PROTOCOL=TCP)(HOST=prd-rac1.erp.temple.edu)(PORT=1521))" _ 58. + "(CONNECT_DATA=(SERVICE_NAME=PROD8)));" _ 59. + "User Id= ADD_ORACLE_USERNAME;Password= " & orapass & ";" 60. '************************************** 61. Dim conno As New OracleConnection(oradb) 62. conno.Open() 63. Dim sqlo As String 64. Dim cmdo As New OracleCommand 65. Dim slbcount As Integer 66. '****** check for connection *********** 67. If (conno.State = 0) Then 68. MsgBox("Oracle Connection Failure") 69. Else 70. 'MsgBox("open") 71. End If 72. '*************************************** 73. sqlo = "" 74. sqlo = line 75. cmdo.CommandType = CommandType.Text 76. cmdo.CommandText = sqlo 77. cmdo.Connection = conno 78. Dim dro As OracleDataReader = cmdo.ExecuteReader() 79. 'Passes sql into oracle connection - waits for returning data 80. While dro.Read() 81. If dro.HasRows = False Then 82. MsgBox("No Data Selected") 83. GoTo SKIPifblank 84. Else 85. uglist1.Add(dro.Item("METRIC")) 86. uglist2.Add(dro.Item("VALUE")) 87. Try 88. Catch ex As Exception 89. 'Console.WriteLine(ex.Message) 90. Return 91. End Try 92. End If 93. skipifblank: 94. End While 95. dro.Dispose()

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 18

96. eoj: 97. slbcount = uglist1.Count 98. cmdo.Dispose() 99. conno.Dispose() 100. 'Load sql output to excel template file 101. Dim xlApp As New Excel.Application 102. Dim xlWorkBook As Excel.Workbook 103. Dim xlWorkSheet As Excel.Worksheet 104. Dim xlSourceRange As Excel.Range 105. Dim xlDestRange1 As Excel.Range 106. Dim xlDestRange2 As Excel.Range 107. Dim myxlfile As String 108. myxlfile = myxls 109. xlWorkBook = xlApp.Workbooks.Open(myxlfile) 110. 'Display Excel 111. xlApp.Visible = True 112. 'Set the source worksheet 113. xlWorkSheet = xlWorkBook.Sheets("Data") 114. 'Set the source range 115. xlSourceRange = xlWorkSheet.Range("A2:B455") 116. 'Clear current data 117. xlSourceRange.Delete() 118. 'Set the destination range 119. xlDestRange1 = xlWorkSheet.Range("A2:A455") 120. xlDestRange1.Value =

xlApp.Application.WorksheetFunction.Transpose(uglist1.ToArray) 121. xlDestRange2 = xlWorkSheet.Range("B2:B455") 122. xlDestRange2.Value =

xlApp.Application.WorksheetFunction.Transpose(uglist2.ToArray) 123. xlWorkSheet = xlWorkBook.Sheets("REPORT") 124. xlWorkSheet.Activate() 125. Dim mydatestamp As String = DateTime.Now.ToString("yyyy-M-d") 126. Dim myexceltitle As String = myfilename & "_" & mydatestamp & ".xlsx" 127. xlWorkSheet.SaveAs(myoutputfolder & myexceltitle) 128. ' xlWorkSheet.ExportAsFixedFormat(XlFixedFormatType.xlTypePDF, myoutputfolder

& myexceltitle, Excel.XlFixedFormatQuality.xlQualityStandard, True, False, 1, 10, False)

129. xlWorkSheet.ExportAsFixedFormat(XlFixedFormatType.xlTypePDF, myoutputfolder & myfilename & "_" & mydatestamp, Excel.XlFixedFormatQuality.xlQualityStandard, True, False, 1, 10, False)

130. 131. xlWorkBook.Close() 132. xlApp.Quit() 133. 134. ReleaseObject(xlDestRange1) 135. ReleaseObject(xlDestRange2) 136. ReleaseObject(xlWorkSheet) 137. ReleaseObject(xlWorkBook) 138. ReleaseObject(xlApp) 139. ' MsgBox("Process Complete")

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 19

140. End Sub 141. 142. Private Sub ReleaseObject(ByVal obj As Object) 143. 'This function is required in order to close all files and connections 144. Try 145. Dim intRel As Integer = 0 146. Do 147. intRel = System.Runtime.InteropServices.Marshal.ReleaseComObject(obj) 148. Loop While intRel > 0 149. 'MsgBox("Final Released obj # " & intRel) 150. Catch ex As Exception 151. 'MsgBox("Error releasing object" & ex.ToString) 152. obj = Nothing 153. Finally 154. GC.Collect() 155. End Try 156. End Sub 157. End Module

NEAIR 2015 – Burlington, Vermont Report Automation in Institutional Research 20

Resources

Visual Basic.NET For more information about connecting to an oracle database environment using Visual Studio: http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/dotnet/GettingStartedVBVersion/GettingStartedNET_VBVersion.htm For more information about How to automate Microsoft Excel from Visual Basic .NET: https://support.microsoft.com/en-us/kb/301982 Visual Basic Quick Syntax Reference: http://www.digilife.be/quickreferences/QRC/VB.NET%20Quick%20Reference.pdf Microsoft’s Visual Basic Programming Guide: https://msdn.microsoft.com/en-us/library/y4wf33f0(v=vs.100).aspx

Structured Query Language (SQL) Oracle SQL Online Language Guide: https://docs.oracle.com/cloud/latest/db112/SQLRF/toc.htm For Microsoft SQL Server environments, the following resources can assist with connection strings and syntax: https://www.connectionstrings.com/sql-server/ https://support.microsoft.com/en-us/kb/308656

Excel VLOOKUP Reference: https://support.office.com/en-us/article/VLOOKUP-function-0bbc8083-26fe-4963-8ab8-93a18ad188a1