Microservices in the Brightspace Web viewMicroservices in the Brightspace Cloud. ... This flexibility allows our teams to develop and refine new ... If you are located in The United

Microservices in the Brightspace Cloud


Contents

ContentsDOCUMENT CHANGE HISTORY....................................................................................................................... 3

ABOUT MICROSERVICES IN THE BRIGHTSPACE CLOUD....................................................................................5

MICROSERVICES AND ON-PREMISE CLIENTS...................................................................................................6

MICROSERVICES AND DATA........................................................................................................................... 6

MICROSERVICES ARCHITECTURE.................................................................................................................... 7

OVERVIEW OF RELEASED MICROSERVICES.....................................................................................................8

Activity Feed Service.........................................................................................................................................12

Brightspace Assignment Grader Transcoding Service.......................................................................................13

Authentication Service.....................................................................................................................................14

Brightspace Binder Data Store..........................................................................................................................15

Content Service................................................................................................................................................16

Course Image Catalog Service...........................................................................................................................18

Dates Service....................................................................................................................................................19

Distributed Event Framework Service..............................................................................................................21

Document Conversion Service..........................................................................................................................22

EduDentity Authentication Service...................................................................................................................23

Feed Service.....................................................................................................................................................25

Hypermedia Proxy Service................................................................................................................................28

Landlord Service...............................................................................................................................................29

LMS Discovery Service......................................................................................................................................32

Brightspace Polling Service...............................................................................................................................33

User Info Service...............................................................................................................................................34

Video Analysis Service......................................................................................................................................35

Video Note Service...........................................................................................................................................36

Wiggio Service..................................................................................................................................................37

MICROSERVICES AND THE BRIGHTSPACE DATA PLATFORM..........................................................................38

ABOUT D2L.................................................................................................................................................. 40

© 2023 by D2L Corporation. All rights reserved. 2


Document Change History

Document Change HistoryThis version of the document replaces all previous versions. The following table describes the most recent changes to this document.

Revision Date Summary of Changes

June 1, 2017 Updated the Activity Feed Service topic to include the locations, in which the service is available.

Updated the Overview of released microservices and Video Note Service topics to include the availability of the Video Note Service in AWS Asia Pacific (Singapore).

Updated the Overview of released microservices, Brightspace Polling Service, and Video Analysis Service topics to include the availability of the services in AWS Asia Pacific (Sydney) and AWS Canada Central (Montreal).

May 4, 2017 Updated Activity Feed Service topic and Microservices architecture topics with Activity Feed Service information.

Updated Overview of released microservices and Document Conversion Service topics to indicate Document Conversion Service is depended on by Brightspace Assignment Grader for iOS and Android.

April 6, 2017 Added the Document Conversion Service topic.

Updated the Microservices architecture and Overview of released microservices topics to reflect all current microservices.

Updated formatting for consistency.

Added Activity Feed Service topic and updated Overview of released microservices topics with Activity Feed Service information.

March 2, 2017 Added the Course Image Catalog Service and Content Service topics.

Updated the Microservices architecture and Overview of released microservices topics to reflect all current microservices.

Removed the Caliper Gateway Service topic to reflect that the service is not currently used in the Brightspace Cloud.

January 5, 2016 Updated Brightspace Valence naming to Brightspace APIs and Brightspace developer platform.




October 6, 2016 Updated the Overview of released microservices, Feed Service, LMS Discovery Service, Dates Service, Video Note Service, and User Info Service topics to reflect the new discussions functionality in Brightspace Pulse.




About microservices in the Brightspace CloudAs the Brightspace platform continues to improve and evolve, some of its functionality is now delivered using a pattern known as Microservice Architecture. This architecture involves separating software otherwise bundled together into independent and lightweight components (microservices or simply known as services) that communicate across a network (typically, via https) rather than being bundled directly together. The location of each microservice in the Brightspace Cloud is based on many factors including expected usage patterns, availability, resiliency, and dependencies on other microservices. As a result, some microservices reside in D2L data centers or Amazon Web Services™ (AWS). For the most part, the locations of microservices have no end-user impact on how the Brightspace platform is used. Some Brightspace products also use microservices that store data outside of D2L data centers. For example, the Brightspace Data Platform uses AWS for data storage and the Dates Service uses IBM® Cloudant® for database storage. If applicable, data storage considerations are covered as part of the D2L master agreement (MA) and/or amendments.

Development and operations teams at D2L experience many of the direct benefits of microservices, but that change and renewal also lets us further improve experiences and functionality for our users. These benefits flow from one key idea: narrowly focused system components that exchange functional services with other components via well-defined network API boundaries.

The narrowly focused and separated components give our teams the option to employ a variety of technologies and scalability strategies, rather than settling for those intended for combined application. For example, the Brightspace Data Platform takes advantage of the distributed processing provided by Apache™ Hadoop® clusters when performing its aggregation and analysis. This technique would not be relevant to other Brightspace product areas such as discussion posts.

Additionally, the separation also helps our teams effectively and quickly adapt to new technologies and approaches as they become available. For example, we have been able to create new user interfaces that leverage specialized web-side user interface frameworks and interact directly with microservices. This flexibility allows our teams to develop and refine new workflows for our users using the most effective technology.

Our test-focused staff also can make effective use of this architectural change because they can take advantage of alternatives around testing microservices that emerge because of the formal service boundaries. Our Brightspace developer platform community can also take advantage of these boundaries, because each of them naturally becomes an API candidate for users looking to develop custom workflows or tools that integrate into the Brightspace platform.

The implementation of microservices and the coordination of development and operations teams has enriched D2L's approach to network infrastructure and deepened our expertise in a variety of more specialized technology platforms.




Microservices and on-premise clientsOn-premise clients access the same D2L microservices in AWS as hosted clients, but they access them through their on-premise Brightspace instances. D2L microservices in AWS do not require any installation. D2L microservices in AWS cannot be hosted in on-premise environments.

To access certain features in Brightspace platform 10.6, on-premise clients must agree to permit access to centrally hosted microservices. For example, the Landlord Service and Authentication Service are required for accessing Brightspace Insights and Brightspace Pulse. Specifics on how to do this depend on how each client's environment is configured. For example, a client may have specific firewall restrictions that their IT department must adjust to permit traffic to D2L microservices in AWS.

Health checks that report on the availability of D2L microservices in AWS are monitored by D2L and are not available to on-premise clients.

Microservices and dataDepending on its function, transmission and storage of data is a consideration for microservices. The location of each data store is based on many factors, including the location of the microservice itself, security of the data, availability of the data, and dependencies on other microservices. D2L works with customers in many regions, jurisdictions, and markets with different needs and requirements around data privacy. We provide information on the data that is transmitted and stored for each microservice, allowing organizations to review it as needed.




Microservices architectureThe following architecture diagram displays current D2L microservices, their deployment locations in the Brightspace Cloud, and the dependencies among them with new Brightspace products and other microservices. For detailed information, refer to the sections for individual microservices in this guide.

Figure 1: An overview of microservices in the Brightspace Cloud




Overview of released microservices




Microservice Name Role Released In

Instance Location Depends On Depended On By

Activity Feed Service Provides the ability to use the Activity Feed widget.

10.7.0 1 instance per region

AWS U.S. East (N. Virginia)

AWS Canada Central (Montreal)

AWS EU (Ireland)

AWS Asia Pacific (Sydney)

AWS Asia Pacific (Singapore)

Authentication Service

Landlord Service

N/A

Brightspace Assignment Grader Transcoding Service

Converts files from one format to another for Brightspace Assignment Grader to consume.

Pre 10.3 1 global instance Azure (West U.S.)

EduDentity Authentication Service

Brightspace Assignment Grader


Provides user and service-level authentication and authorization via the OAuth2 protocol.

10.5.0 1 global instance AWS U.S. East (N. Virginia)

Landlord Service Brightspace Pulse

Brightspace Insights

Brightspace Data Platform

Caliper Gateway

Brightspace Binder Data Store

Not a service but a storage area for Binder documents.

Pre 10.3 1 global instance Azure (South Central U.S., West U.S.)

N/A Brightspace Pulse




Content Service Houses Brightspace content, and currently provides and standardizes functions such as import/export, storage, preview, permissions, and basic search of SCORM objects in the Brightspace platform.


AWS (U.S. East) by default, but administrators can specify region based on their location


LTI Outcomes v2

The new Brightspace SCORM solution

Course Image Catalog Service

Provides a catalog of images available that can be associated with a course.


N/A My Courses widget when Daylight is enabled

The course image banner, which may be added to widget-based homepages when Daylight is enabled.

Dates Service Provides an API for learners' personal dates. Currently, only used by Brightspace Pulse.



AWS - EU (Ireland)

Landlord Service


User Info Service

Brightspace Pulse

Distributed Event Framework Service

Provides awareness of Brightspace Learning Environment events for other Brightspace services such as Brightspace Insights.

Available to clients on 10.4+

1 instance per Data Center

D2L Data Center

Landlord Service Brightspace Data Platform




Document Conversion Service

Converts documents uploaded to the Content tool, Assignments tool, Brightspace Learning Repository, or Brightspace ePortfolio into PDFs, which display inline in the tool or product.


U.S. East (N. Virginia)

Canada Central (Montreal)

Asia Pacific (Sydney)

Landlord Service


Content tool

Assignments tool

Brightspace Learning Repository

Brightspace ePortfolio

Brightspace Assignment Grader for iOS and Android

EduDentity Authentication Service

Stores, manages, and authenticates users independent of Brightspace Learning Environment.

Pre 10.3 1 global instance Azure (South Central U.S., West U.S.)

N/A Brightspace Binder Data Store

Feed Service Provides an API for learner updates to the Announcements, Grades, Content, and Discussions tools. Currently, only used by Brightspace Pulse.



AWS - EU (Ireland)

Landlord Service


User Info Service

Brightspace Pulse

Hypermedia Proxy Service

Acts as a proxy or mediator to learning paths within Brightspace platform.


Landlord Service


Brightspace platform

Landlord Service Provides each Brightspace instance with a TenantId, a permanent globally unique identifier.


N/A Authentication Service







LMS Discovery Service Provides a list of Brightspace instances so app users (such as Brightspace Pulse) don't need to know their instance URL.



AWS - EU (Ireland)


Brightspace Polling Service

Enables organizations to do quick surveys. The first integration is with Brightspace Capture for Live Events.

10.6.3 and Brightspace Capture 9.2

1 instance per region




N/A Brightspace Capture

User Info Service Provides storage of user preferences and filters user information between Brightspace Pulse and Brightspace Learning Environment.



AWS - EU (Ireland)


Dates Service

Feed Service

Video Analysis Service Acts as a conduit between Brightspace Capture and the Brightspace Data Platform.





Brightspace Capture and Brightspace Learning Environment





Video Note Service Enables learners and instructors to record short videos with a webcam. These videos can be added where video attachments are supported and when the HTML Editor’s Insert Stuff option is available.

Pre 10.3 1 instance per region


AWS - EU (Ireland)

AWS Asia Pacific (Sydney, Singapore)

N/A N/A

Wiggio Service Enables learners and instructors to collaborate in an informal setting.

10.3 1 global instance AWS (U.S. East)

N/A N/A

Activity Feed ServiceDescriptionThe Activity Feed Service provides the ability to use the Activity Feed homepage widget. The Activity Feed widget creates a unique "feed" or "stream" for a course. The feed is a way for instructors to provide access to activities learners need to complete, deliver information they need to know, and facilitate learner engagement using simplified workflows in an intuitive and friendly interface.

LocationOne instance per region in AWS:



EU (Ireland)


Asia Pacific (Singapore)

Dependencies Depends on Authentication Service and Landlord Service.




Data Transmitted/StoredData related to the Brightspace Learning Environment and course that launched the service:

TenantId

OrgUnitId

Data related to the user generated post or comment:

User ID - a programmatic identifier for the user who created the post or comment such as User ID = 123

Timestamp - when the post or comment was created

Message text - the post or comment

Weblinks and titles related to any attachments on the post or comment

How the Service Works1. A course administrator adds the Activity Feed widget to a course homepage.

2. An administrator sets up Brightspace Learning Environment’s IMS configuration and maps Brightspace Learning Environment roles to the IMS roles for Instructor and Student.

3. An instructor views the class feed, creates new posts, contributes comments, and moderates all learner comments.

4. Learners view the class feed and contribute comments.

5. All user created posts and comments are visible inside the widget and stored in the Activity Feed Service.

Brightspace Assignment Grader Transcoding ServiceDescriptionThe Brightspace Assignment Grader Transcoding Service converts documents from a given format into a format that can be read by Brightspace Assignment Grader. Brightspace Assignment Grader requires this functionality to support annotating files submitted by learners for grading.

LocationOne global instance in Microsoft Azure West U.S.

Dependencies Depends on EduDentity Authentication Service.

Depended on by Brightspace Assignment Grader.

Data Transmitted/Stored Data is cached for five days, after which is it is automatically deleted. There is no long term storage.

A programmatic identifier for the user such as User ID = 123.




The converted file and the identified file type.

How the Service Works1. Brightspace Assignment Grader submits a file to the service.

2. The service converts the file and returns a link.

3. The file and associated data is deleted five days after the request is made.

Authentication ServiceThe Authentication Service (or Auth) is an OAuth 2.0 security token microservice. Its primary responsibility is to issue security tokens to authorized clients (software applications, including free-range apps) to enable them to interact with D2L microservices.

By design, the Authentication Service, on which Brightspace Pulse is dependent, does not support self-signed, expired, or invalid certificates. Organizations using any of these will not be able to use Brightspace Pulse.

The Authentication Service is enabled by default. As a result, Brightspace features or products that depend on the Authentication Service, such as Brightspace Pulse, can be accessed. Currently, all features or products that depend on the Authentication Service are turned off by default. If those features or products are enabled, it is possible for data to flow into them.

LocationA globally accessible D2L microservice that resides in AWS U.S. East (N. Virginia).

Dependencies Depends on the Landlord Service. Before using the Authentication Service, on-premise clients must register their

org with the Landlord Service.

Depended on by multiple microservices, Brightspace Pulse, and Brightspace Insights.




Data StoredThe Authentication Service stores the URLs of authorized clients (software applications, including free-range apps) and provisions access tokens for these clients for service-to-service authentication used by Brightspace products. It stores the userId as part of the context for user authentication - for example, when authenticating a user of the data API for the Brightspace Data Platform.

Using a proxy server with the Authentication Service for on-premise clientsThe Authentication Service supports proxy servers. This allows on-premise clients that use proxy servers to take advantage of Brightspace products that depend on the Authentication Service such as Brightspace Pulse.

For on-premise clients using a proxy server, allow outbound traffic from Brightspace Learning Environment to https://auth.brightspace.com.

Important: You must specify the host name (not the IP address) and port 443.

How the Service WorksThe Authentication Service facilitates service-level and user-level authentication and authorization. The following example summarizes user-level authentication and authorization.

1. A learner navigates to a tool that depends on the Authentication Service.

2. The Learning Management System (LMS) contacts the Authentication Service, provisions an Auth token (JSON Web Token) for the learner, and provides the Auth token to the tool/application.

3. While using the tool, JavaScript running in the learner’s browser can call secured D2L microservices directly, providing the Auth token during each request.

4. Microservices extract and authenticate the Auth token, then ensure that the caller is authorized to perform the requested operation before proceeding.

In this way, the learner’s browser is less tightly coupled to the LMS, which improves performance and robustness, and facilitates the development of new Brightspace features.

Brightspace Binder Data StoreDescriptionThe Brightspace Binder Data Store contains Binder documents on behalf of a user. It is not a microservice, but is a centralized repository that is used by Brightspace Binder.

LocationOne global instance in Microsoft Azure South Central U.S. and West U.S.

Dependencies Depended on by Brightspace Binder.

Data Transmitted/Stored A programmatic identifier for the user such as User ID = 123.




Files related to the user, including tags, annotations, and metadata.

How the Service Works1. Brightspace Binder submits documents for storage in the Brightspace Binder data store.

2. At a later point in time, Brightspace Binder requests a document on behalf of a user.

3. An authentication check happens for the user.

4. The requested document is retrieved.

Content ServiceDescriptionThe Content Service houses Brightspace content, and currently provides and standardizes functions such as import/export, storage, preview, permissions, and basic search of SCORM objects in the Brightspace platform. It makes use of a new SCORM Engine to play back SCORM objects in a new window.

Connections from the Content Service to Brightspace Learning Environment are made through Brightspace APIs.

LocationOne instance per region deployed by default in AWS U.S. East (N. Virginia), but can be changed by organization administrators based on their location. Administrators first set the d2l.Tools.ContentService.Endpoint configuration variable to on, and then depending on the AWS region they select, based on their location, enter the corresponding URL.Note: When selecting the AWS region from the list below, confirm with those responsible within your organization whether there are any geographic or other restrictions regarding where your organization's data must reside. The following general guidelines might help with your selection, but you must first ensure that the region you select is consistent with local law, and with the contract that your organization signed with D2L:

If you are located in The United States, Central America, or South America, choose the US region; enter https://api.us-east-1.content-service.brightspace.com (default).

If you are located in Europe, Africa, or the Middle East, choose the Ireland region; enter https://api.eu-west-1.content-service.brightspace.com.

If you are located in Australia, New Zealand, Oceania, China, India, or the rest of Asia (excluding Singapore), choose the Australia region; enter https://api.ap-southeast-2.content-service.brightspace.com.

If you are located in Singapore, choose the Singapore region; enter https://api.ap-southeast-1.content-service.brightspace.com.

Important: Once you specify the AWS region and your users start to add SCORM content to their courses, you cannot change the region without loss of data.

Dependencies Depends on the Authentication Service, the Landlord Service, and LTI Outcomes.




Depended on by the new Brightspace SCORM solution.

Data Transmitted/StoredThe following data is transmitted through/stored in the Content Service, the SCORM Engine, or S3:

Data related to upload sessions.

Data related to the conversion process of content objects.

Data related to user sessions, which can include:

User ID.

First and last name.

Where in Brightspace Learning Environment the user launched from.

LTI roles.

User roles in the LMS.

Data related to the content object, which can include:

Where its stored on S3.

Revisions of the content object.

Who owns the content.

Metadata such as title or type.

Where the content object is put in the LMS.

Report information on SCORM progress, attempts, score, user ID, and first and last name.

SCORM Engine stores reporting data, progress in the SCORM package, and user ID.

S3 stores data on the SCORM packages uploaded.

How the Service WorksEmbedding a SCORM package into the Content tool:

1. When a user clicks the option to add a SCORM package, an LTI (ContentItemSelectionRequest) launch is made to the Content Service.

2. The user selects a SCORM package (zip file) from their system. API calls are made to the Content Service to register the new object and sign requests to upload the package to S3.

3. After the SCORM package is uploaded to S3, processing occurs to unzip the package, re-upload the unzipped package to S3, and import it into the SCORM Engine.

4. When the processing of the package is complete, the user is notified that they can insert the content topic.

5. When the user inserts the content topic, a request is made from the Content Service to Brightspace Learning Environment that assigns a URL to the uploaded object, which is then embedded into the content topic.




Playing or interacting with a SCORM package in the Content tool:

1. When a user clicks on a content topic that includes a SCORM package, a button is displayed indicating that they can open the object in a new window. The user clicks the button and a new window is opened, which performs an LTI (basic-lti-launch-request) launch to the Content Service.

2. The Content Service checks the user’s IMS role from the LTI launch to determine whether the user is an instructor or learner.

3. The Content Service makes API calls to the SCORM Engine to register the launch and request a signed preview or launch URL. The preview URL is requested if the user is an instructor and the launch URL is requested if the user is a learner.

4. The Content Service redirects to the SCORM Engine preview or launch URL and the SCORM package displays to the user.

5. As the user interacts with the SCORM package, progress updates are sent to the SCORM Engine, which are then forwarded to the Content Service for its own reporting.

Course Image Catalog ServiceDescriptionThe Course Image Catalog Service provides a list of publicly available images that can be associated with a course in the Learning Management System (LMS). It is used by the new My Courses experience and the new Course Image Banner when Daylight is enabled.

LocationDefault for all clients: a global cluster that resides in AWS U.S. East (N. Virginia).

Dependencies Does not depend on any other microservice.

Depended on by the My Courses widget when Daylight is enabled.

Depended on by the course image banner, which may be added to widget-based homepages when Daylight is enabled.

Data Transmitted/StoredThe Course Image Catalog Service contains a set of images stored on the Brightspace CDN along with keywords for searching through the catalog when choosing an image to associate with a course.




The Course Image Catalog Service does not store the association of an image to a course. This is stored in the LMS. There is no user or course information stored with the service.

Search terms used in requests are logged anonymously. These may be used to help prioritize future improvements to the service, such as search index improvements or possible catalog expansion.

How the Service Works1. If a course does not have a course image assigned to it, the LMS calls the Course Image Catalog Service for a

random abstract image to assign to the course.

2. If users have permission to change a course image, they can search using the Course Image Catalog Service for a list of images they can assign to the course.

Dates ServiceDescriptionThe Dates Service provides an API for learner dates, for example, Brightspace Pulse uses the Dates Service to provide details on assignment due dates and scheduled exams.

LocationDefault for all clients: a global cluster that resides in AWS U.S. East (N. Virginia). The Dates Service Database resides in IBM Cloudant (Rackspace, U.S. - Chicago).

For configured EMEA clients: a global cluster that resides in AWS - EU (Ireland). The Dates Service Database resides in IBM Cloudant (SoftLayer, London).

Dependencies Depends on the Authentication Service and Landlord Service, and User Info Service.

Depended on by Brightspace Pulse.




Data Transmitted/StoredTo communicate with one another, devices and microservices use the HTTPS networking protocol.

Data at rest residing in AWS - CouchDB (the primary database) is encrypted, while data at rest residing in IBM Cloudant (the high availability database) is not encrypted. While in transit, data is encrypted using SSL.

The user ID in data is a composite key along with the course offering ID; it is not a universally accepted global ID for the user. The user ID cannot be linked to a user's name or identity. In the database, user IDs appear as a series of repeated numbers.

Regarding data retention, D2L requires the user ID to report on general user data; it is not used to report on the activity of a specific user. For example, the user ID may be used to report on how many users have an average of three or more dates per month. D2L would not use the data to report on how many times John Smith looks at his deadlines. D2L retains the data as long as required to generate reports based on general user data. The reports are subject to change at D2L’s discretion and client-specific data is subject to the terms specified in the MA, including data retention past contract termination.

Dates Service Database Data Scope Retention Policy

Calendar/Personal dates

Note: Personal dates are created by learners (midterms, assignments, etc.); the information doesn't currently exist in the LMS. However, after creation, the dates persist between devices.

Course offering ID (OrgunitId), userId, tenantId, title, description, eventTimestamp, type (test, assignment), weight, progress status

All course offerings for all instances at a given data center

As long as required for analytical purposes

How the Service Works1. When a learner uses Brightspace Pulse, an API request is made to retrieve date information, for example, scheduled

exams.

2. The request is forwarded to the Dates Service.

3. If the data is stored in Brightspace Learning Environment, the data is retrieved using the Brightspace APIs. If the data is not stored in Brightspace Learning Environment, the data is retrieved from the Dates Service database.

4. The date information is then sent to the device.




Figure 2: How the Dates Service works when accessing dates from Brightspace Pulse

Distributed Event Framework ServiceDescriptionThe Distributed Event Framework Service provides awareness of Brightspace Learning Environment events for other products such as Brightspace Insights.

LocationOne instance per Data Center.

Dependencies Depends on the Landlord Service.

Depended on by the Brightspace Data Platform.

Data StoredThe Distributed Event Framework Service transmits and stores data in the form of events. The Distributed Event Framework Service uses the Advanced Message Queuing Protocol (AMQP) with encryption. While in transit, all events are encrypted. The events that are transmitted contain programmatic identifiers for the user, the context of the event, and the type of the event. For example:


A programmatic identifier for courses such as Course ID = 987.

Events such as logins, tool access, and content visits are identified by the programmatic identifier for the user.

Events are stored in the Data Center and transmitted to services such as the Brightspace Data Platform (located in AWS).

How the Service WorksThe following example illustrates how the Distributed Event Framework Service works with Brightspace Insights.

1. User events are generated in Brightspace Learning Environment.




2. These events are then stored in the Main DB split.

3. The Telegraph Service pulls batches of events from the Main DB split and prepares to publish them to the Distributed Event Framework Service.

4. Before publishing events, the Telegraph Service must attach a TenantId to each event. It first looks for the TenantId in the Memcache (where a cached copy of the TenantId may be stored). If the TenantId is not there, it requests it from the Landlord Service.

5. The Landlord Service returns the unique TenantID to the Telegraph Service, which attaches the TenantId to each event and then publishes the events to the Distributed Event Framework Service.

6. The Distributed Event Framework Service processes the events for usage by other products/services. For example, the Distributed Event Framework Service streams events to the Brightspace Data Platform.

Figure 3: How the Distributed Event Framework Service works with Brightspace Insights




Document Conversion ServiceDescriptionThe Document Conversion Service converts documents uploaded to the Content tool, Assignments tool, Brightspace Learning Repository, or Brightspace ePortfolio into PDFs, which display inline in the tool or product.

Documents are converted during upload instead of during user access, which reduces wait times when rendering documents to users.

Valid file types include: .ppt, .pps, .pptx, .ppsx, .doc, .docx, .rtf, .odt, .odp, .xls, .xlsx.

Converted documents display as inline PDFs in the tool or product. If an uploaded document is an invalid file type, it fails to convert and render in the tool or product.

When users download documents, they are saved in their native file formats.

LocationOne instance per region. The Document Conversion Service is currently in the following AWS locations:




A call from the Brightspace platform is routed to the appropriate region based on its geographic location using the Amazon Route 53 DNS web service. Currently, Canadian servers are routed to Canada Central, Australian servers to Asia-Pacific, and all others default to U.S. East.

Dependencies Depends on the Authentication Service

Depended on by the Content tool, Assignments tool, Brightspace Learning Repository, or Brightspace ePortfolio

Depended on by Brightspace Assignment Grader for iOS and Android

Data Transmitted/StoredUploaded documents and converted PDFs are stored in AWS S3 and cleaned up every 180 days. There is no long-term storage.

How the Service Works1. User uploads a document to the Content tool, Assignments tool, Brightspace Learning Repository, or Brightspace

ePortfolio.

2. The Brightspace platform sends the file to the Document Conversion Service for conversion.

3. The service converts the file to a PDF, which is stored in AWS S3.

4. A link to the PDF file is returned to the Brightspace platform for display in the tool or product.




5. After 180 days, the uploaded document and converted PDF are deleted.

EduDentity Authentication ServiceDescriptionThe EduDentity Authentication Service allows users to verify their identity, similar to the login process in Brightspace Learning Environment. This service is independent of any particular instance of Brightspace Learning Environment, allowing users to log in even if they are not associated with any particular organization. This service is used with products where users may not need to have any affiliation to a specific institution or implementation of Brightspace Learning Environment, such as Brightspace Binder.

LocationOne global instance in Microsoft Azure South Central U.S. and West U.S.

Dependencies Depended on by Brightspace Assignment Grader Transcoding Service.

Depended on by Brightspace Binder system, including the Brightspace Binder Data Store, Content Publishing Service (CPS), and Binder apps.

Depended on by MyDesire2Learn.

Depended on by Open Courses.

Data Transmitted/StoredFor each user registered in the system:


A hash of the password (but not the password itself to prevent decryption).

A security question and three hashes for the answers (but not the answers themselves to prevent decryption).

Email address.

First name, last name, and display name.

The date the user was created.

Whether or not the user has been verified and the deadline for verification.

Whether or not the user is currently active and the date of deactivation (if applicable).

The last successful login date, the number of failed login attempts, and the date the user was locked out (if applicable).

Whether or not this is a dummy user and an expiry date (if applicable).




How the Service Works1. A separate solution verifies the credentials of a user with the service. The service authenticates the user and passes

the results back to the calling solution.

2. Alternatively, a separate solution requests the security question of the service. The response is checked against the stored hashes.

Feed ServiceDescriptionThe Feed Service provides an API for learner updates to the Announcements, Grades, Content, and Discussions tools. It sends user notifications (the ones that appear in the minibar in Brightspace Learning Environment) to the Apple Push Notification Service (APNS) and Google Cloud Messaging (GCM) for use by the Apple iOS and Google Android platforms, respectively.

Note When users log in with Brightspace Pulse, data starts collecting automatically. To prevent data collection while you are evaluating this product for your environment, disable the Feed Service.

LocationDefault for all clients: a global cluster that resides in AWS U.S. East (N. Virginia). The Feed Service Database resides in IBM Cloudant (Rackspace, U.S. - Chicago).

For configured EMEA clients: a global cluster that resides in AWS - EU (Ireland). The Feed Service Database resides in IBM Cloudant (SoftLayer, London).

Dependencies Depends on the Authentication Service.





Data Transmitted/StoredThe Feed Service stores Announcements notifications that are pushed from the LMS as they happen (in the LMS) for Brightspace Pulse users. This service itself does not return to the LMS to retrieve historical data.

To communicate with one another, devices and microservices use the HTTPS networking protocol.


The user ID in data is a composite key along with the course offering ID; it is not a universally accepted global ID for the user. The user ID cannot be linked to a user's name or identity. In the database, user IDs appear as a series of repeated numbers.

Regarding data retention, D2L requires the user ID to report on general user data; it is not used to report on the activity of a specific user. For example, the user ID may be used to report on how many users have an average of three or more dates per month. D2L would not use the data to report on how many times John Smith looks at his deadlines. D2L retains the data as long as required to generate reports based on general user data. The reports are subject to change at D2L’s discretion and client-specific data is subject to the terms specified in the MA, including data retention past contract termination.




Feed Service Database Data Scope Retention Policy

Grades Event: Released Grade

Course offering ID (key), user ID (key), grade value

All users for all course offerings for all instances globally


Grades Event: Updated Grade

Course offering ID (key), user ID (key), grade value



Announcements Event: New Announcements Item

Course offering ID (key), title, description, posted date

All course offerings for all instances globally


Announcements Event: Updated Announcements Item

Course offering ID (key), title, description, posted date

All course offerings for all instances globally


Discussions Event: New Forum

Course offering ID (key), Forum ID (key)



Discussions Event: Update Forum

Course offering ID (key), Forum ID (key)



Discussions Event: New Topic

Course offering ID (key), Forum ID (key), Topic ID (key)



Discussions Event: Update Topic

Course offering ID (key), Forum ID (key), Topic ID (key)



Discussions Event: New Post

Course offering ID (key), Forum ID (key), Topic ID (key), Post ID (key)



Discussions Event: Reply to Post

Course offering ID (key), Forum ID (key), Topic ID (key), Post ID (key)






User/Device Mapping User ID (key), Device ID (key)

Note: The Device ID is an identifier supplied by Apple/Google servers so Brightspace can send push notifications to the user's device.

All devices for all users globally


Note The Feed Service does not transmit information from courses with an End Date that has passed or that have the Is Active setting disabled.

How the Service Works1. When a learner uses Brightspace Pulse for the first time, their device is registered and a unique Device ID is

assigned.

2. In Brightspace Learning Environment, events are generated that need to be sent to the Feed Service as push notifications, for example, an exam grade.

3. The event is stored in the Feed Service database.

4. The D2L Mobile Push Notification Service looks up the Device ID in the Mobile Push Notification Service Database to determine who the intended recipient is.

5. The D2L Mobile Push Notification Service sends the Device ID to the third party Push Notification Service (i.e. Apple, Google), which retrieves the event directly.

6. The 3rd party Push Notification Service sends a push notification to the device. No data is sent with the request, only a notice that information is available such as an exam grade.




Figure 4: How the Brightspace Feed Service works when receiving notifications in Brightspace Pulse

Hypermedia Proxy ServiceDescriptionThe Hypermedia Proxy Service acts as a proxy or mediator to learning paths within Brightspace platform.

Connections from the Hypermedia Proxy Service to Brightspace Learning Environment are made through the Brightspace APIs.

LocationA global cluster that resides in AWS U.S. East (N. Virginia).

Dependencies Landlord Service - If unavailable, this service will also be unavailable.

Depended on by Brightspace platform:

Activity Sequence Viewer

Data Transmitted/StoredNo data is stored alongside this service. The data passed through this service is:

OrgUnitId




For content modules and topics:

ID

Parent ID, child IDs, sibling IDs

Name

Completion state

Entity data representing files, links, LTI activity launch information, etc. Specifics depend on the entity type and are dictated by Brightspace Learning Environment.

How the Service Works1.

Landlord ServiceDescriptionThe Landlord Service is a global microservice that supports multi-tenancy and Service Oriented Architecture (SOA)-based solutions. It provides each Brightspace instance with a TenantId, a permanent globally unique identifier.

Note If a Brightspace instance cannot connect to the Landlord Service, a unique TenantId is not assigned and any features that require a TenantId are unavailable. Users receive a message that their organization's system is not set up.

LocationA global instance that resides in AWS U.S. East (N. Virginia).

DependenciesDepended on by:





Brightspace Pulse

Data Stored The TenantId.

The primary domain as well as any aliases of your Brightspace instance.

The main database split server as configured in the instance.config file and the database name.




API CallsLandlord allows the following public read-only API calls:

Given primary domain, database server name, and database, retrieve a TenantId. All three values are required to get a TenantId.

Given a TenantId, retrieve the primary domain.

Provisioning a TenantId (on-premise clients only)If you do not have a TenantId, submit a ticket to D2L Support requesting a TenantId with the following information for each of your sites:

the Brightspace site

the DNS CNAME, where the value is the database server name

name of the main database split

Important The DNS CNAME and name of the main database split must match the corresponding information in the instance.config file. The easiest method for providing this information to D2L Support is to copy the connection string element from instance.config for the main database split (excluding the password). For example:

<connectionString value="Data Source=MYSQLServer;Initial Catalog=D2L_Main;..." />

Having your TenantID provisioned using a DNS CNAME instead of a host name ensures that if you need to make an unplanned change to your database server, applications that rely on the TenantId are unaffected. For example, if your site fails over to a mirror database, you update the CNAME value to the new host name. In this situation, no changes to the TenantID are required. For example, Name: LVUDB, Type: CNAME, and Value: winsql01.lvu.com.

Confirming that a TenantId has been provided (on-premise clients only)As of 10.5.0, on-premise clients can check the ORG_ORGANIZATIONS table in their main database split to see if the TenantIdCachedForQueryString and CachedTenantId columns are populated for their org. If they are not populated, then Brightspace is unable to get the TenantId from the Landlord Service, meaning the TenantId is not provisioned yet or Brightspace cannot connect to the Landlord Service. The System Error Log should include a message that describes the issue.

Configuring Brightspace for the Landlord Service (on-premise clients only)If your institution's firewall configuration does not allow outbound traffic, you must use one of the following methods to establish an outgoing HTTPS connection to https://landlord.brightspace.com (https://landlord.brightspace.com):

Add a firewall rule to allow outbound connections (port 80 and port 443) from all web and scalable servers to https://landlord.brightspace.com (https://landlord.brightspace.com).

Configure a proxy server on the network by setting up the following configuration variables: d2l.System.Infrastructure.ProxyAddress - The address of the proxy server. It normally takes the form


https://landlord.brightspace.com/

https://landlord.brightspace.com/



http://myproxy:8080/ or https://myproxy:8080/ where myproxy is the host name or IP address and 8080 is the port. d2l.System.Infrastructure.ProxyBypassAddresses - Addresses or address patterns that should not go through the proxy server. Address patterns take the form scheme://hostname:port/path where scheme is either http or https; hostname can be set as a * wildcard; port can be a specific number or a * wildcard to apply to all port numbers; and path is optional and can also contain a * wildcard.

Changes to TenantID values (on-premise clients only)Features that require the Landlord Service could experience problems if the primary domain of the Brightspace site, the database server name, or name of the main database split changes.

If you need to change the primary domain of your Brightspace site, the database server name, or the name of the main database split, contact your D2L Technical Account Manager.

LMS Discovery ServiceDescriptionThe LMS Discovery Service provides a list of LMSs so users don't need to know their Brightspace instance URL. For example, when a learner uses Brightspace Pulse, they need to connect the app to a specific Brightspace instance. However, the learner may only know the name of the school and not the Brightspace instance URL. The LMS Discovery Service allows learners to enter the school name and the service provides them with the applicable Brightspace instance URL.

LocationDefault for all clients: AWS U.S. East (N. Virginia).

For configured EMEA clients: AWS - EU (Ireland).

DependenciesDepended on by Brightspace Pulse.

Data StoredData at rest is encrypted and stored in Amazon S3. While in transit, data is encrypted using SSL.

Regarding data retention, D2L requires the user ID to report on general user data; it is not used to report on the activity of a specific user. D2L retains the data as long as required to generate reports based on general user data. The reports are subject to change at D2L’s discretion and client-specific data is subject to the terms specified in the MA, including data retention past contract termination.

LMS Discovery Service Database

Data Scope Retention Policy

Institution names (currently only Higher Education, U.S./Canada)

Institution name, Instance URL, Location

Stored in AWS regions Service life time




How the Service Works1. From a device, a learner launches Brightspace Pulse for the first time and enters the name of their school.

Brightspace Pulse contacts the LMS Discovery Service, which retrieves the Brightspace instance URL used by the school.

2. After entering the URL in Brightspace Pulse, the learner logs in, authenticating directly with the school’s Brightspace instance.

Figure 5: How the LMS Discovery Service works with Brightspace Pulse

Brightspace Polling ServiceDescriptionThe Brightspace Polling Service allows organizations to run quick polls during webcasts (Example: What is your favorite subject?).

LocationOne instance per region. The Brightspace Polling Service is currently in the following AWS locations:







Dependencies Depended on by Brightspace Capture.

Data Transmitted/Stored Content: Questions and choices created by the presenter and answers created by the participants.

User information: UserID (numerical identifier for each user, i.e. no portion of the name is stored)

Platform data: Tenant ID, and the context (e.g. Video ID including tags, annotations, and metadata).

How the Service Works1. In Brightspace Capture, a presenter creates a webcast and chooses to create a poll.

2. The poll is created in the Polling Service.

3. During the event, the presenter initiates the poll, which gets the information from the Polling Service.

4. Answers by participants are sent to the Polling Service.

5. After the event, the presenter can access the results of the poll from the Polling Service.

User Info ServiceDescriptionTo enhance the performance and scalability of microservices that support Brightspace Pulse, the User Info Service acts as a proxy microservice responsible for:

Modifying or filtering user information between Brightspace Pulse and other sources of information (currently, Brightspace Learning Environment only).

Storing user preferences for Brightspace Pulse.




By default, the User Info Service is turned on and cannot be disabled.

Connections from the User Info Service to Brightspace Learning Environment are made through the Brightspace API. On-premise clients must ensure the Brightspace API can be publicly accessed to allow connections from the User Info Service to Brightspace Learning Environment.

LocationDefault for all clients: a global cluster that resides in AWS U.S. East (N. Virginia). The User Info Service Database resides in IBM Cloudant (Rackspace, U.S. - Chicago).

For configured EMEA clients: a global cluster that resides in AWS - EU (Ireland). The User Info Service Database resides in IBM Cloudant (SoftLayer, London).

Dependencies No dependencies on other microservices.


Data Transmitted/StoredThe User Info Service stores the following user data for course offering enrollments in a User Info Service database in IBM Cloudant:

Composite key of tenantId and userId

orgUnitId

Org unit name

Org unit code

Org unit type

Color (Brightspace Pulse only)

Active flag (Brightspace Pulse only - was returned in previous enrollments, but is no longer returned)

Customized course names (edited in Brightspace Pulse)




To communicate with one another, devices and microservices use the HTTPS networking protocol.


Regarding data retention, D2L requires the user ID to report on general user data; it is not used to report on the activity of a specific user. D2L retains the data as long as required to generate reports based on general user data. The reports are subject to change at the discretion of D2L and client-specific data is subject to the terms specified in the MA, including data retention past contract termination.

User Info Service Database

Data Scope Retention Policy

Course Enrollment Tenant ID (key), Course offering ID (key) user ID (key)

User As long as required for analytical purposes

Org Unit: Personal Settings

Org unit name, Org unit code, Org unit type, Colour, Active flag

User As long as required for analytical purposes

Video Analysis ServiceDescriptionThe Video Analysis Service acts as a conduit between Brightspace Capture and Brightspace Data Platform.

LocationOne instance per region. The Brightspace Polling Service is currently in the following AWS locations:




Dependencies Depends on Brightspace Capture.

Depends on Brightspace Learning Environment

Depended on by Brightspace Data Platform.

Data Transmitted/Stored Course ID (org-unit ID)

User ID, user role




Video ID

Number of segments in the video

Video segment #: indicates which segment of the video was watched, a number between 1 and the number of segments in the video

How the Service Works1. As a user watches a video, the data is sent to Brightspace Data Platform.

2. This data is stored in Brightspace Data Platform, and is aggregated to create the analytics dashboard in Brightspace Learning Environment.

3. The aggregation calculation counts the number of times each user has watched each segment of the video.

Video Note ServiceDescriptionVideoNote is a light-weight video recording service that allows learners and instructors to record short videos with a webcam. These videos can be added where video attachments are supported and when the HTML Editor’s Insert Stuff option is available.


Depended on by Brightspace Data Platform.

LocationDefault for all clients: AWS U.S. East (N. Virginia).

For configured EMEA clients: AWS EU (Ireland).

For configured APAC clients: AWS Asia Pacific (Sydney or Singapore).

Data Transmitted/Stored The Brightspace Learning Environment instance name

The organization name

orgID

orgUnitID

Username

Title of the video

Description of the video

Date/time video was created




The location of the video

The video – being recorded and played back

Wiggio ServiceDescriptionWiggio is an ad-hoc group collaboration product that allows instructors and learners to work together in a less formal setting that than Brightspace Learning Environment.


LocationAWS U.S. East (N. Virginia)

Data Transmitted/Stored Brightspace Learning Environment user name

User email address

User names (first and last)

orgUnitID

orgID

User mobile phone number (user provided, optional)

Wiggio group memberships (which Wiggio groups the user is part of)

Personal and group files uploaded to Wiggio (includes folder names)

Email and SMS notification settings (user configured)

Social media account information (user configured)

User photo (if provided by user)

Wiggio specific:

Group calendar (Meetings and event dates, times, and invitees)

ToDo list (list name, task name, and assignee)

Polls and poll questions

Discussion topics, messages, and responses

Group website links




Microservices and the Brightspace Data PlatformDescriptionThe Brightspace Data Platform is D2L’s analytics solution. The Brightspace Data Platform stores raw events, and computes and stores aggregated data which can be accessed through an API.

LocationA cluster of nodes in AWS regions, influenced by our client base. D2L works with customers in many regions, jurisdictions, and markets with different needs and requirements for data privacy and residency.

Dependencies Depends on the Distributed Event Framework Service.

Depends on the Caliper Gateway Service.

Depends on the Landlord Service.

Depends on the Authentication Service.

Depended on by Brightspace Insights.

Data Transmitted/StoredThe Brightspace Data Platform stores and transmits analytics events and aggregated data. Events provide information about actions performed by the user. For example, a content visit event is triggered when a user opens a content topic. These events are aggregated across meaningful dimensions, for example, course access by all students in a course. The aggregated data can be transmitted via the Data API, for example, to a Brightspace Insights report.

Events contain programmatic identifiers for the user, the context of the event, and the type of the event. For example:


A programmatic identifier for courses such as Course ID = 987.

A programmatic identifier for the event type such as Login = 4545.

Stored data is encrypted with unique keys generated by D2L and are unique to each region. The data is stored on encrypted volumes to guard against back-end services being compromised. When transmitting data, the Brightspace Data Platform uses the HTTPS networking protocol. While in transit, all events are encrypted. Data access is restricted on a per-customer basis using the TenantID of the originating Brightspace instance. API access is governed by user and system-level permissions.

How the Service Works1. User events are generated in Brightspace Learning Environment. For example, when a user logs in to the system, a

Login Event is generated.




2. The Distributed Event Framework Service sends events to the Brightspace Data Platform.

3. Events are stored in Brightspace Data Platform (BDP) Storage.

4. The Brightspace Data Platform aggregates data, and stores the aggregated data in BDP Storage. For example, Login Events could be aggregated along hourly, daily, and weekly dimensions.

5. Aggregated data is sent to Brightspace Learning Environment in response to API requests. For example, API requests could be used to generate a report showing the Login Events generated for learners in a course.

Figure 6: How microservices work with the Brightspace Data Platform



About D2L

About D2LA global leader in EdTech, D2L is the creator of Brightspace, the world’s first integrated learning platform.

The company partners with thought-leading organizations to improve learning through data-driven technology that helps deliver a personalized experience to every learner, regardless of geography or ability. D2L’s open and extensible platform is used by more than 1,100 clients and almost 15 million individual learners in higher education, K–12, healthcare, government, and the enterprise sector—including Fortune 1000 companies.

The company has operations in the United States, Canada, Europe, Australia, Brazil, and Singapore. www.brightspace.com (http://www.brightspace.com) | www.D2L.com (http://www.d2l.com)

Contact Us

Phone: 1.519.772.0325 (Worldwide)

Toll Free: 1.888.772.0325 (North America)

0.808.234.4235 (United Kingdom and Europe)

0.800.452.069 (New Zealand)

1.800.656.210 (Australia)

0.800.891.4507 (Brazil)

Fax: 1.519.772.0324

Email: [email protected]

Twitter: @Brightspace

Web: www.brightspace.com (http://www.brightspace.com) | www.D2L.com (http://www.d2l.com)

© 2017 D2L Corporation.The D2L family of companies includes D2L Corporation, D2L Ltd, D2L Australia Pty Ltd, D2L Europe Ltd, D2L Asia Pte Ltd, and D2L Brasil Soluções de Tecnologia para Educação Ltda.Brightspace, D2L, and other marks ("D2L marks") are trademarks of D2L Corporation, registered in the U.S. and other countries. Please visit d2l.com/trademarks for a list of other D2L marks.


http://www.d2l.com/

http://www.brightspace.com/

http://www.d2l.com/

http://www.brightspace.com/


About D2L

Amazon Web Services and AWS are trademarks, registered trademarks or trade dress of AWS in the U.S. and/or other countries.Apache, Apache Hadoop, and Hadoop are trademarks of The Apache Software Foundation. Used with permission. No endorsement by The Apache Software Foundation is implied by the use of these marks.IBM and Cloudant are registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml (www.ibm.com/legal/copytrade.shtml).All other trademarks are property of their respective trademark holders.
