Making auditing great again! Office 365

SharePoint Saturday Belgium 2017 • October 21 • Brussels Track: IT PRO | Level: 200

Making auditing great again!

Paul Hunt - MVP

Pla

tin

um

Go

ldSi

lver

• Solutions Architect for Trustmarque

• Co-organiser of SUGUK London Region

• Member of the SharePoint community since 2007

• Third time Office Server & Services MVP in 2017

• Woodturner

Who am I?

• Paul Hunt

• @Cimares

• www.myfatblog.co.uk

• www.trustmarque.com

• Solutions Architect for Trustmarque

• Co-organiser of SUGUK London Region

• Member of the SharePoint community since 2007

• Third time Office Server & Services MVP in 2017

• Woodturner

Who am I?

• Paul Hunt

• @Cimares

• www.myfatblog.co.uk

• www.trustmarque.com

SharePoint Saturday Belgium 2017 • October 21 • Brussels

Agenda

The importance of records

Office 365 Audit comparison

SharePoint Site Collection auditing

SharePoint Audit in the Unified Audit Log

Extracting the Unified Audit Log

The importance of records

Beware of false knowledge; it is more dangerous than ignorance.George Bernard Shaw

The importance of records

“If I were to run, I’d run as a republican. They’re the dumbest group of voters in the country. They believe anything on fox News. I could lie and they’d still eat it up. I bet my numbers would be terrific”

FALSE

The importance of records

• People magazine keep every copy of every magazine that has been printed.

• There was no record of a 1998 interview.

• No article printed in the 80s or 90s contain mention of the Republican party in articles about Donald Trump.

The importance of records

“Not a lot of people know that..”

Michael CaineFALSE

The importance of records

• https://youtu.be/hY85a15n5QY

• Peter Sellers apparently used this on his answering machine and repeated it in a Parkinson interview in the 70s.

• Michael Caine has confirmed he never used the phrase until it was added as an in-joke to the film Educating Rita in 1983.

GDPR – Helping to prove compliance

• GDPR does not mandate auditing of data.

• Audit data assists in proving compliance but does not make you compliant.

• Helps to identify unauthorised data access

You don’t need to audit everything!

• Targeted auditing is easier:• To manage

• To report on

• To monitor

• Auditing is pointless is you cannot interrogate and understand the data.

Understanding your organisation’s audit needs is NOT an IT function!

IT should facilitate, not drive the need for Audit.

Audit everything is not a good option!

SharePoint Saturday Belgium 2017 • October 21 • Brussels

Audit availability in Office 365

Auditing availability in Office 365

SharePoint Online Auditing

• Configured per site collection

• 90 day limit enforced (30 day minimum?)

• Extracted automatically (if configured)

• Can’t configure in EDGE!

• Doesn’t record VIEW activities

• OneDrive auditing difficult to manage.

Office 365 Unified Audit

• Broad spectrum of coverage (Beyond just SharePoint!)

• 90 day limit

• Manual/App based extraction

• Doesn’t record LIST ITEM activities. (This includes changing DOCUMENT metadata!)

• Integrates with ASM (E5)

SharePoint Saturday Belgium 2017 • October 21 • Brussels

SharePoint Site Collection AuditConfiguration

Configuring Site Collection Auditing

• Configured on a per site collection basis.

• Limited to a maximum of 90 days

Configuring Site Collection Auditing

• Configured on a per site collection basis.

• Limited to a maximum of 90 days

Audit log view link

Classic team site Modern site

/_layouts/15/Reporting.aspx?Category=Auditing

Demo: SharePoint Site Collection Audit

SharePoint Saturday Belgium 2017 • October 21 • Brussels

Office 365 Unified AuditConfiguration

Unified Audit functionality in Office 365

• User Activity• SharePoint & OneDrive

• Exchange Online (requires mailbox audit logging!)

• Sway*

• PowerBI

• Teams (Not messages!)

• Yammer*

• Dynamics 365

• Flow (On it’s way!)

• Admin Activity• Azure Active Directory

• SharePoint Online

• Exchange Online

• Sway*

• PowerBI

• Teams

• Yammer*

• eDiscovery

• Flow (On it’s way!)

Note: This list is slowly being increased!

Unified audit lag timesWorkload 30 Mins 24 Hours

SharePoint Online and OneDrive for Business X

Exchange Online X

Azure Active Directory (User login events) X

Azure Active Directory (admin events) X

Sway X

PowerBI X

Yammer X

Security & Compliance Centre (eDiscovery) X

Teams X

Dynamics 365 X

Flow (When it arrives) X

Turning on Unified O365 auditing

• Sign in to Security & Compliance Centre.

• Select Search & Investigation/Audit Log Search

Searching the Unified Audit log

Searching the Unified Audit log

Searching the Unified Audit log - Filter & Export

Demo: Configuring & Searching the O365 Unified Audit log

Additional steps for Exchange

• Connect using Exchange Online PowerShell Module.*

• Set-mailbox “name” –AuditEnabled $true

• Default Audit gives:

*Now supports MFA & ADFS

Admin Delegate Owner

Update Update None

Move

MoveToDeletedItems

SoftDelete SoftDelete

HardDelete HardDelete

FolderBind

SendAs SendAs

SendOnBehalf

Create Create

Audit actions available

Action Admin Delegate Owner

Copy Yes No No

Create Yes* Yes* Yes

FolderBind Yes* Yes** No

HardDelete Yes* Yes* Yes

MailboxLogin No No Yes***

MessageBind Yes No No

Move Yes* Yes Yes

MoveToDeleteItems Yes* Yes Yes

SendAs Yes* Yes No

SendOnBehalf Yes* Yes No

SoftDelete Yes* Yes* Yes

Update Yes* Yes* Yes

Bind = Open or Read (including preview pane)

* - Default action auditing when enabled.** - Aggregated for a 24 hour period*** - Only applies to POP3/IMAP4 or Oauth logins. Does not track NTLM or Kerberos logins

But I need more than 90 days worth of audit!

SharePoint Saturday Belgium 2017 • October 21 • Brussels

Extracting the O365 Unified Audit Log using the Management API

Options for Extracting the Unified Audit log

Pull method

• Register your APP!

• Register a collector subscription

• Download a manifest file

• Download content blobs listed in Manifest.

• Process data into backend storage

Push method

• Register your APP!

• Register a collector subscription

• Register a WebHook

• Content blob manifests are pushed to the Webhook.

• Download content blobs when notified.

• Process data into backend storage

Note: Subscribed data is available for 7 days only!

Registering Your APP ID in Azure AD.

• Requires Web app/API configuration

• And Tenant level permissions.

Don’t forget to GRANT permissions

Registering a collector subscription

Available for 5 Content Types

• Audit.AzureActiveDirectory

• Audit.Exchange

• Audit.SharePoint

• Audit.General (Sway, Yammer etc)

• DLP.All

Notes:

• When a subscription is registered, it can take up to 12 hours for the first content to be available.

• DLP.All is only available to users with the “Read DLP Sensitive Data” permission.

Retrieving the Blob Manifest

• Returns a collection of JSON objects

contentUri : https://manage.office.com/api/v1.0/d3c8c691-7321-4cc4-ac08-7ca6f05be84c/activity/feed/audit/20170809160530886001699$20170809160530886001699$audit_sharepoint$Audit_SharePoint

contentId : 20170809160530886001699$20170809160530886001699$audit_sharepoint$Audit_SharePoint

contentType : Audit.SharePointcontentCreated : 2017-08-09T16:05:30.886ZcontentExpiration : 2017-08-16T16:05:30.886Z

Retrieving the Blob Content

• Returns a collection of JSON objectsCreationTime : 2017-08-15T10:30:58Id : 93c5b9d0-f916-46d0-7a2f-08d4e3c8b7dbOperation : FileUploadedOrganizationId : d3c8c691-7321-4cc4-ac08-7ca6f05be84cRecordType : 6UserKey : i:0h.f|membership|10037ffe9e27c68a@live.comUserType : 0Version : 1Workload : SharePointClientIP : 52.169.28.217ObjectId : https://wharfconsulting.sharepoint.com/sites/audit-test-c/Audit Samples/Prime Minister without Education and skills.txtUserId : joan.jett@wharf-media.co.ukEventSource : SharePointItemType : FileListId : 7db7d957-69fc-4c2d-b191-82868c1928beListItemUniqueId : b081f0c2-f055-437f-b128-8666bead8dddSite : ad4040da-0b0a-4059-958c-5f6c27d181e6WebId : 97c2f404-3aa8-4efd-8e34-6736c3aefcecSourceFileExtension : txtSiteUrl : https://wharfconsulting.sharepoint.com/sites/audit-test-c/SourceFileName : Prime Minister without Education and skills.txtSourceRelativeUrl : Audit Samples

Where to put all that data?

Gotchas!

• Subscription content expires 7 days after collection.

• Watch out for oAuth token expiry.

• Beware the back-off command. MS will throttle excessive requests.

Demo: Extracting the Unified Audit Log

Reporting on your audit data from Azure SQL

Questions?

References

• Office 365 Management Activity API Reference

SharePoint Saturday Belgium 2017 • October 21 • Brussels

Thank You!