DISCOVERY DOCUMENT OPEN EDX PLATFORM - JUNIPER …...DISCOVERY DOCUMENT OPEN EDX PLATFORM - JUNIPER RELEASE Last Modified Authors

DISCOVERY DOCUMENT

OPEN EDX PLATFORM - JUNIPER RELEASE

Last Modified

Authors

Arbiter

Status

Created

References https://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniper

https://edunext.co/mailto:[email protected]:[email protected]://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniper

TABLE OF CONTENTS

GENERAL SCOPE 3

1.1. Applications/Services Included 3 1.2. Location of Code (Upstream Repositories) 3 1.3. URLs 3 1.4. Existing Documentation or Resources 4

INSTALLATION PROCEDURES 4 2.1. What Procedures / Documentation Was Followed? 4 2.2. What Problems Did We Run Into and How Did We Overcome Them? 4 2.3. Where are the Settings? 5

THEMING 5 3.1. Consistency Across Screens 5 3.2. Responsive Design 6 3.3. Right to Left 6 3.4. Unified Branding Configurations 6

TESTING OF NEW / CHANGED FEATURES 6 IN-DEPTH REVIEW PER FEATURE 13 FINAL NOTES 20

1. GENERAL SCOPE

1.1. Applications/Services Included

● LMS - STUDIO (juniper.alpha1) ● E-COMMERCE (not installed) ● INSIGHTS (not installed) ● MOBILE APPS (not installed) ● GRADEBOOK (juniper.alpha1) ● COURSE DISCOVERY SERVICE (not installed) ● LEARNER RECORDS(credentials? not installed) ● MICROFRONTENDS

○ Profile (juniper.alpha1) ○ Account (juniper.alpha1) ○ Order History (master (0f64b30ee1eba147d7f4b3311288c6cc9707f872))

● EDX ENTERPRISE (not installed) ● VIDEO PIPELINE (not installed)

1.2. Location of Code (Upstream Repositories)

● https://github.com/edx/edx-platform/tree/open-release/juniper.alpha1 ● https://github.com/edx/frontend-app-ecommerce/tree/0f64b30ee1eba147d7f4b3311288c6cc9707f872 ● https://github.com/edx/frontend-app-gradebook/tree/open-release/juniper.alpha1 ● https://github.com/edx/frontend-app-profile/tree/open-release/juniper.alpha1 ● https://github.com/edx/frontend-app-account/tree/open-release/juniper.alpha1

1.3. URLs

● LMS: https://openedx-edunext-co-j-alpha.edunext.co/ ● CMS: https://studio-j-alpha.edunext.co/ ● Gradebook: https://gradebook-j-alpha.edunext.co/ ● Profile: https://profile-j-alpha.edunext.co/u/admin_edunext ● Account: https://account-j-alpha.edunext.co/ ● Order History (not fully functional due to the lack of e-commerce in this environment):

https://history-j-alpha.edunext.co/

https://github.com/edx/edx-platform/tree/open-release/juniper.alpha1https://github.com/edx/frontend-app-ecommerce/tree/0f64b30ee1eba147d7f4b3311288c6cc9707f872https://github.com/edx/frontend-app-gradebook/tree/open-release/juniper.alpha1https://github.com/edx/frontend-app-profile/tree/open-release/juniper.alpha1https://github.com/edx/frontend-app-account/tree/open-release/juniper.alpha1https://openedx-edunext-co-j-alpha.edunext.co/https://studio-j-alpha.edunext.co/https://gradebook-j-alpha.edunext.co/courseshttps://profile-j-alpha.edunext.co/u/admin_edunexthttps://account-j-alpha.edunext.co/https://account-j-alpha.edunext.co/

1.4. Existing Documentation or Resources

URL COMMENTS

Technical docs https://docs.edx.org/

User docs https://docs.edx.org/

Release notes https://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniper

These notes are good. They highlight a lot of issues we encountered and some we have not yet touched. Lets link to them in the appropriate items

Other relevant resources

2. INSTALLATION PROCEDURES

2.1. What Procedures / Documentation Was Followed?

● Followed documentation: https://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniper

● We got and tested some new features for LMS. Additionally to that page, we read the changelog of the Juniper version to know what variables were changed and what are the new variables on Juniper.

2.2. What Problems Did We Run Into and How Did We Overcome Them?

● We had problems with the login between LMS and CMS. The reason was the value of the variable SESSION_COOKIE_DOMAIN, also, I had to configure the EDXAPP_LOGIN_REDIRECT_WHITELIST. The session needs to be shared between Studio and the LMS. Also discussed in slack.

● We had problems with MFE + Basic Auth, we had to disable basic auth in order to get that working properly. Also if we want to add an MFE we need to configure EDXAPP_LOGIN_REDIRECT_WHITELIST and use https, otherwise, the platform won’t redirect to the MFE page after a successful login.

https://docs.edx.org/https://docs.edx.org/https://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniperhttps://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniperhttps://openedx.atlassian.net/wiki/spaces/COMM/pages/940048716/Juniperhttps://discuss.openedx.org/t/deprecation-removal-complete-removal-of-studio-signin-and-signup-pages-relates-to-depr-6/962https://discuss.openedx.org/t/deprecation-removal-complete-removal-of-studio-signin-and-signup-pages-relates-to-depr-6/962

2.3. Where are the Settings? Currently the settings for CMS and LMS are in /edx/etc/.yml but you can still see that in the ansible installations they are created the .env.json and auth.json files in /edx/app/edxapp/..json. Nevertheless, they are not being used. edxapp only uses the yml file. Here are some questions we considered important to be raised and answered.

● What is still set as DUMMY KEY CHANGE BEFORE GOING TO PRODUCTION? This is FERNET_KEY, and this is using the default value. This was removed from the master version of edx/configuration: https://github.com/edx/configuration/pull/5431 still not sure why. In edx-platform the references to FERNET_KEYS have not seen removed.

● Why is there still a MICROSITE_CONFIGURATION: and MICROSITE_ROOT_DIR: key? Is this not removed for Juniper? It has not been removed from configuration (Juniper version). In master that was removed: https://github.com/edx/configuration/pull/5431. Nevertheless this is not being used in edx-platform.

● What should we be doing about: USERNAME_REPLACEMENT_WORKER: OVERRIDE THIS WITH A VALID USERNAME? We must configure this with an usernamer that is going to use these APIs https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/lms/djangoapps/discussion/rest_api/views.py#L602, https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/openedx/core/djangoapps/user_api/accounts/views.py#L1022

3. THEMING

3.1. Consistency Across Screens Profile, account and order pages have different header and footer compared to the other pages because they are micro-frontend elements. But turning them off will look like the previous version of the platform, without conflicts (Use the Flags: ENABLE_PROFILE_MICROFRONTEND ENABLE_ORDER_HISTORY_MICROFRONTEND ENABLE_ACCOUNT_MICROFRONTEND). The other pages have no visual problems, they look like the older version. For a very long course name, it breaks styles and text is not contained within the block.

https://github.com/edx/configuration/pull/5431https://github.com/edx/configuration/pull/5431https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/lms/djangoapps/discussion/rest_api/views.py#L602https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/lms/djangoapps/discussion/rest_api/views.py#L602https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/openedx/core/djangoapps/user_api/accounts/views.py#L1022https://github.com/edx/edx-platform/blob/03a3c201cb1065f11936deac88ec085ee69f1110/openedx/core/djangoapps/user_api/accounts/views.py#L1022

3.2. Responsive Design For micro-frontend elements there are two menus, one is the dropdown with the user info and the other is the menu with general links. Some elements don’t look good in mobile version: wiki, sequence menu inside a course, problems with images, sidebar of about page, the course image is hidden on the dashboard, inputs on instructor dashboard.

3.3. Right to Left For micro-frontend pages, there is no translation of texts although the platform is in Arabic and other pages translate well. On the about page are some different footer styles than other pages.

3.4. Unified Branding Configurations Not evaluated during this process.

4. TESTING OF NEW / CHANGED FEATURES

COMPONENT NAME OF FEATURE DESCRIPTION COMMENTS

LMS Sysadmin Dashboard -Enhanced interface Will it be usable for our internal requirements this time? Users -> maybe Courses -> maybe Staffing -> maybe gitlogs fails -> no

INSTRUCTOR DASHBOARD

View of Learner Grades and Enrolment Status

It is easier for the instructor to access the enrollment status of a particular learner.

Section Locator New way to select the locator of a section or problem easily in the instructor dashboard.

User Reports New columns, date_joined, and last_login.

STUDIO New Navigation Bar New way to navigate units from Studio.

Enhanced Circuit Simulator The Studio circuit simulator has been improved

Enhanced Problem Editor The basic problem editor has more help options, which is very useful for those who do not have coding skills.

Enhanced Advanced Problem

More options and improvements to create Advanced Problems in Studio.

Problem Editor New fonts in the text editor.

LMS ADMIN Manual unlocking of users A new ction from the django admin panel to unlock users.

Useful for superadmin support. No intermediate admins since it is on the admin panel.

E-COMMERCE -- -- Not installed during this test.

INSIGHTS -- -- Not installed during this test.

MOBILE APPS -- -- Not installed during this test.

5. IN-DEPTH REVIEW PER FEATURE

Name of feature JWT_TOKENS

Business case It is a compact URL-safe means of representing claims to be transferred between two parties.

Limitations

Potential problems

Will definitely break the wordpress plugin at some point. We need to add support for JWT there.

Settings This configuration is placed in settings.JWT_AUTH, which in edx/configuration is: JWT_AUTH: JWT_ISSUER: "{{ EDXAPP_LMS_ISSUER }}" # DEPRECATED JWT_AUDIENCE: "{{ EDXAPP_JWT_AUDIENCE }}" # DEPRECATED JWT_SECRET_KEY: "{{ EDXAPP_JWT_SECRET_KEY }}" # DEPRECATED JWT_ISSUERS: # DEPRECATED - ISSUER: AUDIENCE: SECRET_KEY JWT_AUTH_REFRESH_COOKIE: # DEPRECATED JWT_PUBLIC_SIGNING_JWK_SET: JWT_SIGNING_ALGORITHM: JWT_PRIVATE_SIGNING_JWK: JWT_AUTH_COOKIE_HEADER_PAYLOAD: JWT_AUTH_COOKIE_SIGNATURE Description of the configurations:

● JWT_PUBLIC_SIGNING_JWK_SET: A JSON Web Key Set (JWK Set) is a JSON object that represents a set of JWKs. The JSON object MUST have a "keys" member, with its value being an array of JWK objects. This JSON object MAY contain white space and/or line breaks.

● JWT_AUTH_COOKIE_HEADER_PAYLOAD: Contains only the header and payload portions of the JWT.

● JWT_AUTH_COOKIE_SIGNATURE: Contains only the public key signature portion of the JWT.

● JWT_SIGNING_ALGORITHM: Signature algorithm used ( we specify RS512 above to identify RSASSA-PKCS1-v1_5 using SHA-512 as the signature algorithm value as described in the JSON Web Algorithms (JWA) spec.

We can use this command to generate values for JWT_PUBLIC_SIGNING_JWK_SET, JWT_PRIVATE_SIGNING_JWK and JWT_SIGNING_ALGORITHM that needs to be filled in into /edx/etc/lms.yml.

python manage.py lms generate_jwt_signing_key

On the other hand JWT_AUTH_COOKIE_HEADER_PAYLOAD and JWT_AUTH_COOKIE_SIGNATURE are the header names used. The JWT_AUTH_COOKIE_HEADER_PAYLOAD cookie is used in the configuration of the MFE.

Multitenancy considerations

Since this relies on configurations that are in the Django settings, we can use eox-tenant to define different configurations per tenant if needed.

i18n N/A

a12y N/A

UX N/A

Mobile support This is not supported on mobile yet.

https://github.com/edx/edx-drf-extensions/blob/491d7e3e7878428f42681d143afd41db4f7a9406/edx_rest_framework_extensions/settings.py#L2https://github.com/edx/edx-drf-extensions/blob/491d7e3e7878428f42681d143afd41db4f7a9406/edx_rest_framework_extensions/settings.py#L2https://github.com/edx/edx-drf-extensions/blob/491d7e3e7878428f42681d143afd41db4f7a9406/edx_rest_framework_extensions/settings.py#L2https://github.com/edx/edx-drf-extensions/commit/00185b9ee41ef3cee78d8985d3c351fad6ad7aa1https://tools.ietf.org/html/rfc7518#section-3.3

Theming N/A

Customization

Documentation ● https://github.com/edx/edx-platform/blob/61e1eda20df2825a409db3e2d36c69d7c36d3e2d/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0009-jwt-in-session-cookie.rst

● https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0008-use-asymmetric-jwts.rst

● https://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0010-csrf-protection.rst

Questions to be resolved

Name of feature Gradebook Business case Instructors can manually edit student grades.

Limitations This only in courses that do not have the course grades frozen.

Potential problems

We have to solve authentication problems and the way that the MFE can get the configurations. MFEs without a PATH in the url will cause difficulties for routing when working in a multitenant env.

Settings We can enable/disable this per course with this waffle flag: grades.writable_gradebook, We need to enable persistent grades, the

grades.assume_zero_grade_if_absent waffle switch and we need to

set the WRITABLE_GRADEBOOK_URL lms django settings with the URL

of gradebook, i.e. 'https://gradebook.edunext.co'


We can enable/disable this per course with this waffle flag: grades.writable_gradebook.Eox-tenant allows us to define a different one: WRITABLE_GRADEBOOK_URL per tenant but we still need to do some

customizations of the MFE side to solve problems related to

login.

i18n Each component of the MFE has its own translations the footer and the header components are in Transifex. The actual strings of this MFE are not in Transifex.

a12y I don’t know

UX I don’t know

Mobile support No

Theming We can brand this, using our own implementation of the front-end components like the footer and the header. We need to find a way to do a tenant-aware theming.

https://github.com/edx/edx-platform/blob/61e1eda20df2825a409db3e2d36c69d7c36d3e2d/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0009-jwt-in-session-cookie.rsthttps://github.com/edx/edx-platform/blob/61e1eda20df2825a409db3e2d36c69d7c36d3e2d/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0009-jwt-in-session-cookie.rsthttps://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0008-use-asymmetric-jwts.rsthttps://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0008-use-asymmetric-jwts.rsthttps://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0010-csrf-protection.rsthttps://github.com/edx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0010-csrf-protection.rsthttps://gradebook.edunext.co/

Customization We need to find a way to solve the auth problems in a multi-tenant environment, and also the branding for a multitenant. The problem with auth is that right now, the LMS url is a static configuration and therefore it redirects to a defined LMS if it does not find the cookies of that site.

Documentation https://github.com/edx/edx-platform/tree/master/lms/djangoapps/grades/docs/decisions


We need to find a way to solve the auth problems in a multi-tenant environment, and also the branding for a multitenant.

Name of feature Profile Business case MFE that allows users to change their profile info and also to see the profile of other users.

Limitations

Potential problems

Problems with AUTH and with branding. We need to modify the APIs to make sure that only are displayed the profiles of the users that are part of the current tenant.

Settings We can enable/disable this globally with the waffle flag: learner_profile.redirect_to_microfrontend. We need to set in the site configuration ENABLE_PROFILE_MICROFRONTEND as true. In the Django settings, we need to define the URL of the MFE in PROFILE_MICROFRONTEND_URL: i.e https://profile-j-alpha.edunext.co/u/ NOTE: the /u/ is important in the current implementation of this MFE


We can enable/disable this per tenant with PROFILE_MICROFRONTEND_URL, we need to solve the auth and branding problems. We need to make sure that only are displayed the profiles of the users that are part of the current tenant.

i18n Each component of the MFE has its own translations the footer and the header components are in Transifex. The actual strings of this MFE are in https://www.transifex.com/open-edx/edx-platform/frontend-app-profile/

a12y I don’t know

UX I don’t know

Mobile support No


Customization We need to find a way to solve the auth problems in a multi-tenant environment, and also the branding for a multitenant. The problem with auth is that right now, the LMS URL is a static configuration and therefore it redirects to a defined LMS if it does not find the cookies of that site.

Documentation



https://github.com/edx/edx-platform/tree/master/lms/djangoapps/grades/docs/decisionshttps://profile-j-alpha.edunext.co/u/https://www.transifex.com/open-edx/edx-platform/frontend-app-profile/

Name of feature Account Business case MFE that allows users to change their account settings.

Limitations

Potential problems

Problems with AUTH and branding.

Settings We can enable/disable this globally with the waffle flag: account.redirect_to_microfrontend. We need to set in the site configuration ENABLE_ACCOUNT_MICROFRONTEND as true. In the Django settings, we need to define the URL of the MFE in ACCOUNT_MICROFRONTEND_URL, i.e https://account-j-alpha.edunext.co.


We can enable/disable this per tenant with ENABLE_ACCOUNT_MICROFRONTEND, we need to solve the auth and branding problems.

i18n Each component of the MFE has its own translations the footer and the header components are in Transifex. The actual strings of this MFE are in https://www.transifex.com/open-edx/edx-platform/frontend-app-account/

a12y I don’t know

UX I don’t know

Mobile support No


Customization We need to find a way to solve the auth problems in a multi-tenant environment, and also the branding for a multitenant. The problem with auth is that right now, the LMS url is a static configuration and therefore it redirects to a defined LMS if it does not find the cookies of that site.

Documentation



Name of feature STUDIO Login

Business case Users will be forced to log in to the LMS in order to get authenticated in STUDIO.

https://www.transifex.com/open-edx/edx-platform/frontend-app-account/

Limitations The LMS and Studio will now require that both are subdomains of the same domain. E.g. courses.site.com, studio.site.com since studio does not properly implement the oauth login as ecommerce or insights do.

Potential problems

Currently studio redirects to the LMS page when a user needs to be authenticated, in a multitenant environment currently there is not a solution to do a redirection to a different LMS URL in the function of the user. Will need to create routes for every studio to be independent.

Settings N/A


We need to solve the way we authenticate in STUDIO, it might be with creating a new plugin, or using eox-tenant (and create a Studio per tenant and therefore redirect in Studio in the function of the tenant configs).

i18n N/A

a12y N/A

UX N/A

Mobile support N/A

Theming N/A

Customization We need to customize this in order to make it works properly for our use case.

Documentation https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256


Can we support SSO in studio if we use the plugin approach?

Name of feature STAFF GRADED PROBLEMS

Business case Might replace the SGA xblock in the future.

Limitations The export/import feature seems to fail in studio and only works in the LMS.

Potential problems

● If this is going to replace SGA there is no place to send the answer to. ● Very useful for offline exercises. ● If we can't find a way to send files, we could make an xblock for that, but it defeats the

purpose of this as a SGA replacement

Settings Nothing.


Nothing.

i18n This is supported out of the box, so translations are in transifex and subject to the same process as the rest of the core code.

a12y Supported out of the box.

https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256

UX Nothing special.

Mobile support Not tested.

Theming Not tested, but it is a course component.

Customization

Documentation https://github.com/edx/staff_graded-xblock


Is this PR needed https://github.com/edx/edx-platform/pull/23108? Is this a CAPA type problem or just an xblock?

https://github.com/edx/staff_graded-xblockhttps://github.com/edx/edx-platform/pull/23108

6. FINAL NOTES

Our conclusions/recommendations after the exercise.

- Document the deployment of MFEs and think of alternatives for routing. - Implement the solution to the studio login problem described in

https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256 - We must think of one solution for the CMS login on the multitenant sites.

Note on Python 3 We did not make changes to use Python3. We are using the same Ubuntu version as ironwood, so no special step was needed to get Python3 with the default settings. Note on Inter-service Communication Impact (We need to install e-commerce to test this):

From the Open edX Wiki: “

7. An update was made to the oAuth+SSO flow using auth-backends EdXOAuth2 backend that requires access to the 'user_id' scope. This is not a default scope. DOT applications like ecommerce-sso will stop working without adding the 'user_id' scope to LMS admin/oauth_dispatch/applicationaccess/. This will be handled automatically for newly provisioned devstacks in this PR: https://github.com/edx/devstack/pull/394. This uses an updated management command in LMS, which can also be used to add the Application Access: https://github.com/edx/edx-platform/pull/20076. (Robert Raposa)

8. Ecommerce expects to find a unique user id (LMS user id) for each user, either in a JWT or social auth. If it cannot find this id, a MissingLmsUserIdException will be raised; this exception can be silenced by enabling the allow_missing_lms_user_id waffle switch. The LMS user id can be back-filled for existing users by running the ecommerce/core/management/commands/import_user_ids.py management command. See ecommerce/docs/decisions/0004-unique-identifier-for-users.rst. (crice)

9. DOP→DOT Migration (Nimisha Asthagiri, Julia Eskew, Troy Sankey, Robert Raposa, Diana Huang, Jeremy Bowman)

a. DEPR-47 - django-oauth2-provider and EdXOpenIdConnect REMOVING b. People already running openedx will need to create new DOT credentials and put

them in their IDA configs in order to not break auth to IDAs. TBD. Need to include step-by-step information for this migration.

“

https://github.com/edx/edx-platform/pull/19845#issuecomment-559154256https://github.com/edx/devstack/pull/394https://github.com/edx/edx-platform/pull/20076https://openedx.atlassian.net/wiki/people/557058:6c4236f8-d4a1-4519-b6f3-6e6c8b8f1316?ref=confluencehttps://github.com/edx/ecommerce/blob/master/ecommerce/core/management/commands/import_user_ids.pyhttps://github.com/edx/ecommerce/blob/master/docs/decisions/0004-unique-identifier-for-users.rsthttps://openedx.atlassian.net/wiki/people/5bd360b4cdfb443b578dda07?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:f6d630b7-0530-46b5-833c-1a59877c9b21?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:23418869-24ef-48b1-83cd-86f8bf9bec79?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:04bc06a1-ecc2-4f07-adc8-1d9d4d2b84db?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:6c4236f8-d4a1-4519-b6f3-6e6c8b8f1316?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:424946fc-0fa2-4cbe-8d00-9db90dde56dd?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:424946fc-0fa2-4cbe-8d00-9db90dde56dd?ref=confluencehttps://openedx.atlassian.net/wiki/people/557058:8df1e4f9-85e3-4bdf-9cd6-e8a950e67497?ref=confluencehttps://openedx.atlassian.net/browse/DEPR-47

Documents

DISCOVERY DOCUMENT OPEN EDX PLATFORM - JUNIPER …...DISCOVERY DOCUMENT OPEN EDX PLATFORM - JUNIPER RELEASE Last Modified Authors