Composer and Drupal - bobbis.com · Drupal 8 is really getting off the Drupal Island. ... * Drupal needs modules, themes, profiles, core, and drush commands installed in a specific

Composer and DrupalCIDUG Meeting December 13, 2018

John Rearick

composer.key - December 14, 2018

Composer"Dependency manager for PHP"

* Similar to other dependency managers such as: yum, apt, brew, macports, npm, pip, etc.* Helps manage dependency hell.* Lots of dependencies* Not only project dependencies, but also versions of projects as well* Long chains of dependency - A requires B requires C requires ….* Not 100% solution, but makes it managable.


Get off the Drupal island

PHP is a much larger community than Drupal. With the basis in Symfony Drupal 8 is really getting off the Drupal Island.In order to more easily use the solutions the rest of the PHP community has come up with, Drupal decided that playing nicely was required. So we now use composer.Circular work. Composer has made Drupal development easier, and Drupal has made composer nicer for the benefit of other PHP projects.


Building a Drupal WebsiteNormally think about “building a drupal website” so we start with Drupal. Then we add things to that.


Building a PHP Website with Drupal as a dependency

However, we’re really building a PHP project that uses Drupal. So our website’s first dependency is Drupal itself.This allows us to only have our own code in the repository and we don’t have to keep track so much with Drupal or any other dependencies


Get and install Composer Globally

• copy commands from getcomposer.org/download/

• mv composer.phar /usr/local/bin/composer

• Add /Users/username/.composer/vendor/bin to your PATH

Update composer with composer self-update


Start a project with drupal-composer/drupal-project

https://github.com/drupal-composer/drupal-project

Kickstart a plain Drupal project with composer:

composer create-project drupal-composer/drupal-project:8.x-dev some-dir --stability

dev --no-interaction

composer - the composer command, installed globallycreate-project - creates a folder with all the files we expect drupal-composer/drupal-project:8.x-dev drupal-composer - org drupal-project - project/repo name :8.x-dev - defined version. (this is not the drupal version, this is the drupal-project version)some-dir - Whatever directory you want this to create—stability dev - Just go with it.—no-interaction - Please don’t bother asking questions, just go with defaults. Thanks


Let’s Build

https://github.com/drupal-composer/drupal-project to find the command

Replace some-dir with mysite


Where does it get code from?

https://packagist.org/

Packagist is a Repository or database of PHP projects. Configured by default with composer.Not a git repository (necessarily) Similar to something like the subscription channels in RHEL


Where does it get code from?

packages.drupal.org

Your own private repositories!

Other repositories are available for composer.Usually one per project or look at Satis, an open source repository generator https://github.com/composer/satis


WARNING

Projects from public repositories, such as Packagist, may not be relied upon.

• Project could be pulled off GitHub• SCM Repository could get compromised• Use trusted repositories and projects

Examples: * left-pad in npm was pulled when the owner had a disagreement of

philosophy and removed all of their own projects from the repository. NPM had to restore the left-pad project after builds broke. https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/

* event-stream in NPM (again). Commit access was given to someone unvetted and added bitcoin code to the project, which, in turn got deployed to numerous websites https://www.theregister.co.uk/2018/11/26/npm_repo_bitcoin_stealer/

* drupal.org code is pretty well reviewed and there is a policy for managing projects in the Drupal repository


Done Building?

Check out the build log from composer and what we got in finder


click to zoom


notice: we have a .gitignore, but not a .git folder. This isn’t a git repo. Yetnotice: composer.jsonnotice: composer.locknotice: web folder.


Composer.json * simple json file* Don’t usually need to edit this by hand. Please use the composer

commands if you can* Starts with basic information* Note the lack of comma on the last time of a list. Always catches me off

guard* composer validate to check your composer.json for errors* be in the root of the directory (though it has gotten smarter about that

recently)


* Repositories are the repositories we talked about earlier. * Not always git repositories* Can define a git repo for a single private project if needed* here we see the Drupal 8 project repository maintained by the DA


* Here we see the project’s dependencies. * format of org/project* Note: cweagans/composer-patches* Semantic Verstioning Major.Minor.Patch. * Major - breaking changes. remove deprecated functions (Drupal 9)* Minor - New features. Generally shouldn’t break anything, but you know

how that goes* Patch - quick bug fixes that can’t wait for a minor release or security

updates usually* Noticing the ^ has some meaning, we’ll get to that in a bit* Fuzzy version dependency. You can define a range of versions.* add these dependencies with the command line. We’ll get to that.* composer has a concept of a dev version so you can add dependencies

when in development mode. (I haven’t really used this yet)* perhaps can be used with something like devel module or features module


* Extra stuff can be added to enhance composer or configure other plugins.

* Here we can see that we’re configuring patch level to -p2 so that patches can be applied in the web folder

* Patches can be defined here too thanks to the composer-patches requirement we had earlier (we can cover that if you want later)

*


* Installer paths define where code should go. * by default code ends up in the the /vendor/org/project folder* Drupal needs modules, themes, profiles, core, and drush commands

installed in a specific place* type is defined in a project’s composer.json at the top.


* Drupal Scaffold is a plugin written for this composer project so that files like index.php, .htaccess, etc get placed properly

* Here we see a configuration to move the .editorconfig and .gitattributes files to the root of the project instead of the web folder

* When updating drupal core check the git diff to review any possible changes you may have made to these files.

* You can configure drupal-scaffold to ignore certain files by default as well. We have .htaccess ignored because shibboleth config


Composer.lock * Don’t edit this file manually!* Is used to build out all the files as defined in composer.json* locks the specific version of dependency code.* sort of like a cache


Composer.lock Merge Conflict

http://blog.doh.ms/2016/11/28/solving-conflicts-in-composer-lock/

It’s easy to get conflicts in this file when two people update the lock file. I found http://blog.doh.ms/2016/11/28/solving-conflicts-in-composer-lock/ to be a good strategy to manage those conflicts.


.gitignore next slide to zoom in


.gitignore * We don’t have a git repo yet. But it has suggestions for what to put in .gitignore.

* Notice we don’t commit core, contrib, or libraries!* These files are all managed by composer and brought in. So, there really

isn’t any need to track these files.


Set up Drupal

fiddle stuffmkdir -p config/sync


Install Modules

composer require drupal/token


Define Versions

composer require drupal/token:^1.5

^ Latest stable version according to semver~ Allows only last digit to go up

Other constraints such as * < > work too.

getcomposer.org/doc/articles/versions.md#next-significant-release-operators

Same as before.


Other Examples

composer require ‘drupal/token:^1.5'

composer require 'drupal/simple_fb_connect:~3.0'

composer require 'drupal/ctools:3.0.0-alpha26'

composer require 'drupal/token:1.x-dev'

* Token 8.x-1.5 to 8.x-1.n* simple_fb_connect 8.x-3.0 to 3.n* ~ and ^ mean almost the same thing in Drupal land for now, because

semver translates by adding an extra .0 at the end of the versions.* ctools specifically 8.x-3.0-alpha26* token 8.x-1.x-dev


Install token

* composer require drupal/token* See the change in composer.json* composer.lock also updated


Updating Modules

composer update drupal/token --with-dependencies

* `composer update` without anything will update everything. Be careful!* --with-dependencies is important because we want to update any

dependencies as required usually.


Updating Drupal Core

composer update drupal/core webflo/drupal-core-require-dev symfony/* --with-dependencies

* Go to the composer project page and look at the readme for recommended steps: https://github.com/drupal-composer/drupal-project#updating-drupal-core

* Git diff any changes to confirm stuff. Especially for scaffold files that you modified like .htaccess

*


Git

* git init* git add .* git commit -m “inital commit”* Install another module drupal/redirect* observe changes only in composer.json and composer.lock* take a gander at the diff* clone a new copy* cd ..* mkdir other* cd other/* git clone ../some-dir/ .* realize there is no vendor, core, or nothing in modules directory!* Let’s bring it all back in.* composer install* reads the composer.lock file and fetches all the stuff and puts them in

place* git status shows no changes* see our files are there!* oh hey, where’s redirect? (we didn’t commit it…)* got back to ~/Sites/some-dir* git add .* git commit -m “adding redirect module”* cd ..* cd other* git pull* composer install to get us the redirect module YAY


Deployment

composer install can be a bit of a resource hog, and some may not want that running on production.

Use a build server or something to create an artifact to deploy

* composer install as part of your deployment is fine


Artifact

• Adds complexity to the build/deployment process

• Adds production code auditing.

• Reduces resources required for production servers.

* There’s a lot of moving parts to keep track of. Need a place to make the build and move the files

* Code on the production server can be committed to a repo and it’s easier to see if files were changed, with git or some such.

* composer building requires a lot of memory to run and can be resource intensive. Better to offload that to once in a while on circleCI or something ephemeral and cheap than needing to run an Amazon Larger that spends most of its time idle. or trying to figure out how to scale containers.


Artifact

1. Deploy to a build server2. build server runs composer install3. rsync or commit/push/pull the code files to a

production server

* Build server something like circleCI or travis, can be configured to work on web hooks, for automatic deployment

* Build server does the hard work of building everything* Build server will need to have access to commit or trigger a deployment

with Tower or something. Keys can be added to the build server in an encrypted store

* Could be rsync, a separate git repo or even a branch on the original repo* Using git, be sure that the .gitignore doesn’t ignore the important files like

vendor, core, modules, etc…


Private RepositoriesWe can’t use a public repository like Packagist

• Private/proprietary code on private git repo

• Doesn’t make sense to share with the world

* Remember, not git repo, package repo


Private RepositoriesPrivate Packagist

• Open source project

• Requires infrastructure to build and host

• https://github.com/composer/satis

* Made by the people who made composer* It’s a PHP project that can be installed and managed with composer (of

course)* Has a build process that needs to run any time there’s an update to any

project* Build process is usually done on cron every few minutes* I haven’t done this, but to looks pretty easy to set up


Private RepositoriesProject specific repo

• A project can also be a repository

• Easy to setup

• Cumbersome if you have too many

* A couple lines in the project’s composer.json* can be hosted on any git repository* Bringing into project you need to define both a repo and project

requirement. * If your org has a lot of private projects, can be hard to manage since

there’s not central place


Set up project repo

* See https://github.com/jrearick/drush_example* Name matches github path* description is nice* type is “drupal-module” to make sure it goes into modules/contrib/

drush_example* Perhaps this can be changed to “org-module” or something and update

the base project composer.json to handle it* repositories. Define itself with type ‘git’ and ‘url’* a composer type is used for other repositories like drupal.org, satis. Added

here in case this module wants to define dependencies itself* Go back to our local composer.json* {* "type": "git",* "url": "[email protected]:jrearick/drush_example.git"* },* composer validate* Ignore warning about lock file for now, see that the file is valid* "jrearick/drush_example": “master",* master means master branch. Always get the master. You can probably

define tags here with semver?* composer validate* See warning about trying to avoid unbound version constraints. * composer update jrearick/drush_example* composer validate should no longer warn about outdated composer.lock

(avoided updating everything else…)* git diff to see the composer.lock changes* cd web/modules/contrib/drush_example to see the code is there* cd back to the project root and commit


Patch Management

composer can download and apply patches for you!

* The killer app for composer* With the super fast development of Drupal 8 and Drupal 8 modules, it’s

nice to be able to get patches and manage them


Patch Management

* we have an issue https://www.drupal.org/node/1091852* Have a patch on comment 80* Copy raw patch link https://www.drupal.org/files/issues/

2018-11-08/1091852-80.patch* edit composer.json* ,* "patches": {* "drupal/core": {* "1091852-80 Patches states for ajax fields": "https://

www.drupal.org/files/issues/2018-11-08/1091852-80.patch* } * }* composer validate* composer update drupal/core* local patches can be applied too, instead of a url give a path “patches/

my_custom.patch”* when patches don’t apply, get you notified in update process.


Composer Resources• The quick install: https://getcomposer.org/download/

• Using composer to manage a Drupal project: https://www.drupal.org/docs/develop/using-composer/using-composer-to-manage-drupal-site-dependencies

• composer.lock resolving merge conflict: http://blog.doh.ms/2016/11/28/solving-conflicts-in-composer-lock/

• Satis composer repository generator: https://github.com/composer/satis

• Tips for Managing Drupal 8 projects with composer: https://www.jeffgeerling.com/blog/2017/tips-managing-drupal-8-projects-composer


Documents

Composer and Drupal - bobbis.com · Drupal 8 is really getting off the Drupal Island. ... * Drupal needs modules, themes, profiles, core, and drush commands installed in a specific