Decoupled Demystified: The Present and Future of Drupal’s Web Services

Preston So | @prestonso

#SrijanWW | @srijan

#SrijanWW | @srijan

Welcome!

Preston So has been a web developer and designer since 2001, a creative professional since 2004, and a Drupal developer since 2007. As Development Manager of Acquia Labs, Preston leads new open-source and research initiatives at Acquia. Preston has presented keynotes at conferences on three continents in both English and Portuguese and speaks about diverse topics such as decoupled Drupal, responsive design, front-end development, and user experience.

drupal.org/u/prestonsopreston.so@acquia.com

#SrijanWW | @srijan

What we’ll cover

● Drupal web services at a glance

● Drupal 8.0: WSCII and foundations

● Drupal 8.2: CORS, configuration entities, and DX

● Drupal 8.3: User registration and DX

● What’s ahead in Drupal 8.5+

● JSON API, GraphQL, and RELAXed Web Services

● Epilogue: The wider web services landscape

#SrijanWW | @srijan

Drupal web services at a glance

#SrijanWW | @srijan

Why are web services important?

● Web services enable communication between Drupal and other

systems, most commonly decoupled front ends or other back ends.

● Decoupled Drupal, or API-first Drupal, is the process of employing

Drupal as a data service which exposes data for consumption by other

applications.

● A REST API is a common entry point for other applications.

#SrijanWW | @srijan

Why are web services important?

HTTP request

HTTP response (JSON, XML)

Site or repository built in Drupal

Decoupled application

Web servicesDecoupled application

Software development

kit (SDK)

#SrijanWW | @srijan

Drupal web services at a glance

● Drupal’s web services can be split into three categories:

○ Core REST (internal storage or HAL normalization)

○ Contributed REST (JSON API, RELAXed, Services)

○ Non-REST web services (GraphQL)

#SrijanWW | @srijan

Drupal web services at a glance

#SrijanWW | @srijan

Drupal 8.0: WSCII and foundations

#SrijanWW | @srijan

WSCII

● The Web Services and Context Core Initiative (WSCII), led by Larry

Garfield, enabled RESTful web services in Drupal.

● The original goal was to enable server-to-server communication, but in

recent years actual usage has evolved more toward server-to-client.

● The default REST API available out of the box in Drupal 8 core is fully

REST-compliant.

#SrijanWW | @srijan

The API-first initiative

● The API-first initiative (WSCII), led by Wim Leers, is the successor to

WSCII and aims to expand Drupal’s web services capabilities.

● The API-first initiative captures use cases that are applicable to both

fully decoupled and progressively decoupled (in-Drupal) issues.

● Meetings are held monthly on the third Monday of every month from

6-7pm GMT, and core conversations often take place at DrupalCon.

#SrijanWW | @srijan

Core REST

● The core REST modules allow for all content entities (nodes, users,

taxonomy terms, comments) to be exposed as JSON+HAL or as JSON

representing Drupal’s internal storage, and Views natively supports

“REST export” as a new display type.

● There are many issues with REST in core; please consider contributing

to RX (REST experience) tagged issues.

#SrijanWW | @srijan

Core REST modules

● Serialization is able to perform serialization by providing normalizers and encoders. First, it normalizes Drupal data (entities and their fields) into arrays with a particular structure. Any normalization can then be sent to an encoder, which transforms those arrays into data formats such as JSON or XML.

● RESTful Web Services allows for HTTP methods to be performed on existing resources including but not limited to content entities and views (the latter facilitated through the “REST export" display in Views) and custom resources added through REST plugins.

#SrijanWW | @srijan

Core REST modules

● HAL builds on top of the Serialization module and adds the Hypertext

Application Language normalization, a format that enables you to

design an API geared toward clients moving between distinct resources

through hyperlinks.

● Basic Auth allows you to include a username and password with

request headers for operations requiring permissions beyond that of an

anonymous user. It should only be used with HTTPS.

#SrijanWW | @srijan

Setting up RESTful Drupal

$ drush en -y hal basic_auth serialization rest

$ drush dl restui && drush en -y restui

#SrijanWW | @srijan

Fetching an individual node

GET /node/1?_format=json HTTP/1.1

Host: drupal-backend.dd:8083

Accept: application/json

Cache-Control: no-cache

Postman-Token: 6c55fb8b-3587-2f36-1bee-2141179d1c9c

#SrijanWW | @srijan

Creating a new node

POST /entity/node HTTP/1.1

Host: drupal-backend.dd:8083

Accept: application/json

Authorization: Basic YWRtaW46YWRtaW4=

Content-Type: application/json

Cache-Control: no-cache

Postman-Token: 7776d489-e9bb-cad2-d289-24aa76f8f8a6

#SrijanWW | @srijan

Creating a new node

{

"type": [

{"target_id": "article"}

],

"title": [

{"value": "Lorem ipsum dolor sit amet adipiscing"}

],

"body": [

{"value": "This is a totally new article"}

]

}

#SrijanWW | @srijan

Updating an individual node

PATCH /node/23 HTTP/1.1

Host: drupal-backend.dd:8083

Accept: application/json

Authorization: Basic YWRtaW46YWRtaW4=

Content-Type: application/json

Cache-Control: no-cache

Postman-Token: c1e4df7e-b17b-2256-75c8-55629c8329c7

#SrijanWW | @srijan

Updating an individual node

{

"nid": [

{"value": "23"}

],

"type": [

{"target_id": "article"}

],

"title": [

{"value": "UPDATE UPDATE UPDATE UPDATE"}

],

"body": [

{"value": "Awesome update happened here"}

]

}

#SrijanWW | @srijan

Cross-origin resource sharing (CORS)

# Apache 2

Header set Access-Control-Allow-Origin "*"

#SrijanWW | @srijan

Cross-origin resource sharing (CORS)

$ drush dl cors && drush en -y cors

*|http://localhost:3003

$ drush cr

#SrijanWW | @srijan

Waterwheel SDK ecosystem

● Waterwheel is a collection of SDKs which make it easier for developers

to build Drupal-backed applications in various technologies.

● The Waterwheel module includes resource discovery (content schema

exports to the client side) and generated Swagger API documentation.

● github.com/acquia/waterwheel.js

● github.com/acquia/waterwheel.swift

● drupal.org/project/waterwheel

#SrijanWW | @srijan

Drupal 8.2: CORS, configuration entities, and DX

#SrijanWW | @srijan

Changes in Drupal 8.2

● Configuration entity GET support

● Opt-in CORS support

● RPC endpoints for login, status, logout, and password reset

● REST configuration converted to configuration entities

● Comments can be updated via REST

● Various developer experience benefits

#SrijanWW | @srijan

Configuration entity GET support

● Configuration entities can now be retrieved via GET, meaning you can

now view labels of configuration entities like Vocabularies and Content

Types, which is particularly helpful for client-side visibility.

curl --user admin:admin --request GET

"http://drupal.d8/entity/taxonomy_vocabulary/tags?_format=json"

curl --user admin:admin --request GET

"http://drupal.d8/contact/feedback?_format=json"

#SrijanWW | @srijan

Opt-in CORS support

● Fully decoupled applications on domains distinct from the Drupal back

end are blocked from issuing asynchronous requests to Drupal due to

the same-origin policy unless cross-origin resource sharing is enabled.

● In the past, this was done either via Apache 2 configuration or via the

CORS module, but there is now core support.

● This is not enabled by default due to security consequences of allowing

other domains to access Drupal.

#SrijanWW | @srijan

Opt-in CORS support: default.services.ymlcors.config:

enabled: false

# Specify allowed headers, like 'x-allowed-header'.

allowedHeaders: []

# Specify allowed request methods, specify ['*'] to allow all possible ones.

allowedMethods: []

# Configure requests allowed from specific origins.

allowedOrigins: ['*']

# Sets the Access-Control-Expose-Headers header.

exposedHeaders: false

# Sets the Access-Control-Max-Age header.

maxAge: false

# Sets the Access-Control-Allow-Credentials header.

supportsCredentials: false

#SrijanWW | @srijan

RPC endpoints for login, status, logout, and reset

● You can now log into Drupal, check a user’s status, log out, and reset a

password entirely through RPC endpoints.

#SrijanWW | @srijan

RPC endpoints for login, status, logout, and reset

curl --header "Content-type: application/json" --request POST \

--data '{"name":"admin", "pass":"admin"}' \

http://drupal.d8/user/login?_format=json

curl --header "Content-type: application/json" --request GET \

http://drupal.d8/user/login_status?_format=json

curl --header "Content-type: application/json" --request POST \

http://drupal.d8/user/logout?_format=json

#SrijanWW | @srijan

REST configuration before: rest.settingsresources:

entity:node:

GET:

supported_formats:

- hal_json

supported_auth:

- basic_auth

POST:

supported_formats:

- hal_json

supported_auth:

- basic_auth

PATCH:

supported_formats:

- hal_json

supported_auth:

- basic_auth

DELETE:

supported_formats:

- hal_json

supported_auth:

- basic_auth

#SrijanWW | @srijan

REST configuration after: rest.resource.entity.nodeid: entity.node

plugin_id: 'entity:node'

granularity: method

configuration:

GET:

supported_formats:

- hal_json

supported_auth:

- basic_auth

POST:

supported_formats:

- hal_json

supported_auth:

- basic_auth

PATCH:

supported_formats:

- hal_json

supported_auth:

- basic_auth

DELETE:

supported_formats:

- hal_json

supported_auth:

- basic_auth

dependencies:

module:

- basic_auth

- hal

- node

#SrijanWW | @srijan

Various developer experience benefits

● Entity creation via REST now returns the created entity in the response

● Entity update via REST now returns the updated entity in the response

#SrijanWW | @srijan

Drupal 8.3: User registration and DX

● User registration is now possible through REST

● Various developer experience advancements (changes in module scope

and REST plugin writing process)

#SrijanWW | @srijan

What’s ahead in Drupal 8.5+

#SrijanWW | @srijan

Prospective improvements in Drupal 8.5+

● Translation support

● File upload support

● Configuration entity POST, PATCH, DELETE support?

● Others?

#SrijanWW | @srijan

JSON API, GraphQL, and RELAXed Web Services

#SrijanWW | @srijan

RELAXed Web Services

●

#SrijanWW | @srijan

JSON API

● JSON API is a specification for REST APIs in JSON popular among JavaScript developers and adopted by the Ember and Rails communities.

● JSON API provides a standard way to query single entities, but it also provides all relationships contained therein and query operations via query string parameters.

● JSON API allows you to fetch lists of content entities (filter, sort, pagination) — which is currently only possible via multiple requests or Views REST exports.

#SrijanWW | @srijan

JSON API

● JSON API is likely slated for incorporation into Drupal 8.5 core, though this is prospective. The module is very close to entering beta for stability in contrib.

● See also: JSON API by Mateu Aguiló Bosch (drupal.org/project/jsonapi)

#SrijanWW | @srijan

OAuth2

● Currently Basic Auth is the only way to authenticate into Drupal’s REST API, and it is less secure than other available options such as OAuth2.

● OAuth2 is more secure for authenticating requests, and it is a widely used standard among REST APIs in the wider landscape.

● See also: Simple OAuth by Mateu Aguiló Bosch (drupal.org/project/simple_oauth)

#SrijanWW | @srijan

GraphQL

● GraphQL, originally created by Facebook to power its data fetching, is a query language that enables fewer queries and limits response bloat.

● Rather than tightly coupling responses with a predefined schema, GraphQL overturns this common practice by allowing for the client's request to explicitly tailor a response so that the client only receives what it needs.

● GraphQL shifts responsibility from the server to the client: the server publishes its possibilities, and the client publishes its requirements instead of receiving a response dictated solely by the server.

#SrijanWW | @srijan

GraphQL

● Typical REST APIs tend to be static (or versioned, in many cases, e.g. /api/v1) in order to facilitate

backwards compatibility for applications. However, in Drupal's case, when the underlying content

model is inevitably augmented or otherwise changed, schema compatibility is no longer guaranteed.

● With GraphQL's native schema introspection and client-specified queries, the API is much less opaque

from the client's perspective in that the client is aware of what response will result according to its

own requirements.

● See also: GraphQL by Sebastian Siemssen (drupal.org/project/graphql)

#SrijanWW | @srijan

Summary: Current core REST

#SrijanWW | @srijan

Summary: JSON API

#SrijanWW | @srijan

Summary: GraphQL

#SrijanWW | @srijan

Epilogue: The wider web services landscape

#SrijanWW | @srijan

Content as a service

● Contentful

● Prismic

● Built.io Contentstack

● CloudCMS

● Kentico Cloud

#SrijanWW | @srijan

GraphQL

● GraphQL

● React/Relay

● Apollo Data

#SrijanWW | @srijan

Thank you!

Preston Sodrupal.org/u/prestonsopreston.so@acquia.com

Learn more about Acquia Labsacquia.com/resources/acquia-labs