DockerfileBest Practices - NVISIA · 2019-09-19 · NVISIA -Confidential and Proprietary Example Dockerfile FROMubuntu:18.04 COPY. /app RUNmake /app CMDpython /app/app.py What does

Dockerfile Best Practices

September 17th, 2019

NVISIA - Confidential and Proprietary

Example DockerfileFROM ubuntu:18.04COPY . /appRUN make /appCMD python /app/app.py

What does each step do?• FROM creates a layer from the

ubuntu:18.04 Docker image.• COPY adds files from your Docker

image’s current directory.• RUN builds your application with

make.• CMD specifies what command to run

within the container

What is a Dockerfile?

Build Context


Dockerfile 59bytes

example.txt 19bytes

README.md 54bytes

DockerfileFROM alpine:3.10.1COPY example.txt .CMD cat example.txt

Current Directory

Simple Build Context Example

Build Commanddocker build –t context-1 .


DockerfileFROM alpine:3.10.1COPY example.txt .CMD cat example.txt

Current Directory

More Complex Build Context Example

Build Commanddocker build \–t context-2 \–f docker/Dockerfiledata-1

.git 28,995bytes

.gitignore 7bytesdata-1-->example.txt 19bytesdata-2-->additional.txt 26bytesdata-3-->retropie-4.5rpi2_rpi3.img.gz 813,960,280bytesdocker-->Dockerfile 59bytesREADME.md 79bytes


.git 28,995bytes

.gitignore 7bytesdata-1-->example.txt 19bytesdata-2-->additional.txt 26bytesdata-3-->retropie-4.5rpi2_rpi3.img.gz 813,960,280bytesdocker-->Dockerfile 59bytesREADME.md 74bytes

docker build \–t context-3 \–f docker/Dockerfile.

DockerfileFROM alpine:3.10.1COPY data-1/example.txt .COPY data-2/additional.txt .CMD cat example.txt additional.txt

Current Directory


Build Command


DockerfileFROM alpine:3.10.1COPY data-1/example.txt .COPY data-2/additional.txt .CMD cat example.txt additional.txt

Current Directory


Build Command

.git 28,995bytes

.gitignore 7bytesdata-1-->example.txt 19bytesdata-2-->additional.txt 26bytesdata-3-->retropie-4.5rpi2_rpi3.img.gz 813,960,280bytesdocker-->Dockerfile 59bytes.dockerignore 13bytesREADME.md 74bytes

docker build \–t context-4 \–f docker/Dockerfile.


Build Context

• Contextisthegivendirectoryanddown• Canbeaurl• CannotCOPYfilesfromoutsideofthecontext

• Typicallyit’sgoingtobethecurrentdirectory,butitcanbemodified• DoesnothavetobethesamedirectorythathastheDockerfile• Usethe-foptiontospecifytheDockerfile ifithasadifferentnameorisinadifferentdirectory

• Usea.dockerignore filetoskipunneededfilesfromgettingaddedtothecontext

Image Caching


Dockerfile Project

• SimpleSpringBootproject• FollowstheMavenprojectstructure

• BuiltwithGradle&Docker• Takesroughly110seconds(150secondsthefirsttime)tobuildthejardocker run \

--rm \-w /home/gradle/project \-v $(pwd):/home/gradle/project \gradle:4.9.0-jdk8-alpine \gradle clean build

• .dockerignore fileignores(.git,.gradle,bin)• Needsaruntimeimage

• Preferably,somethingthatcanbebuiltduringCI


Dockerfile Caching Example

BuildTime(Initial) ~180SecondsBuildTime(Anyfilechange) ~150SecondsImageSize 597MB

Build Stats

Copyingineveryfile.Anychangebreaksthecache.

FROM ubuntuCOPY . /appRUN apt-get updateRUN apt-get -y install openjdk-8-jdkRUN apt-get -y install sshRUN apt-get -y install vimCMD ["java", "-jar", "/app/build/libs/docker-app-ds.jar"]


PackageinstallsaregettingInvalidatedonjarchange.

BuildTime(Initial) ~180SecondsBuildTime(Anyfilechange) ~150SecondsImageSize 597MB

Build Stats


BuildTime(Initial) ~180SecondsBuildTime(JARchange) ~150SecondsImageSize 596MB

Build Stats

Onlycopyinthejar.

FROM ubuntuCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarRUN apt-get updateRUN apt-get -y install openjdk-8-jdkRUN apt-get -y install sshRUN apt-get -y install vimCMD ["java", "-jar", "/app/docker-app-ds.jar"]




Build Stats

Movedtorightbeforeit’sneeded.

Theupdategetscachedinaseparatelayer.


Build Stats

FROM ubuntuRUN apt-get updateRUN apt-get -y install openjdk-8-jdkRUN apt-get -y install sshRUN apt-get -y install vimCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Everylinegetsitsownupdate.

FROM ubuntuRUN apt-get update && apt-get -y install openjdk-8-jdkRUN apt-get update && apt-get -y install sshRUN apt-get update && apt-get -y install vimCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Combineeverythingintoonelayer.

FROM ubuntuRUN apt-get update && \

apt-get -y install openjdk-8-jdk ssh vimCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]


Caching

• Changingthefileitselfwillinvalidatethatlayeranddown• Oldversionsofimagesmaystillexistinthecacheandcanbereused

• OnlyCopywhat’sneeded• Morefocusedcopieslimitthechancesofthecachebeinginvalidated• Alsocutsdownonthesizeoftheimage

• Theorderofexecutionmatters• Frequentlychangingitemsarebetteroffatthebottomofthefiles• Copyfilesinrightbeforetheyareneeded

• Bundlecommandstopreventusingoldfiles

Reducing Size


Reducing Size

• Dumpunnecessaryfiles&dependencies• Don’tcopy/addfilesthataren’tneeded• .dockerignore canalsohelpremoveunnecessaryfiles• Removedevelopmentdependenciesfromnon-developmentimages• Canalsoreducesecurityrisk

• Cleanupafterinstallations• Removezipfilesafterunzipping• Cleanupinstall/tempfiles• Removeapplicationsonlyneededforinstallation(e.g.curl)


Dockerfile Sizing ExampleNotneededforproduction.


Build Stats


apt-get -y install openjdk-8-jdk ssh vimCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



apt-get -y install openjdk-8-jdkCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]

Dockerfile Sizing Example


Build Stats




Build Stats

Onlyinstallrequiredcomponents.


Build Stats


apt-get -y install --no-install-recommends openjdk-8-jdk COPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Build StatsBuildTime(Initial) ~105SecondsBuildTime(JARchange) ~5SecondsImageSize 444MB

Removetempinstallfiles.


Build Stats


apt-get -y install --no-install-recommends \openjdk-8-jdk && \rm –rf /var/lib/apt/lists/*

COPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]

Parent Images


Parent Image

• UseTrustedImages• https://arstechnica.com/information-technology/2018/06/backdoored-images-downloaded-5-million-times-finally-removed-from-docker-hub/

• Don’treinventthewheel• Makesureyouselectaversion• ‘latest’mayworktoday,butmayrequirechangestomorrow

• FindSmallestImagethatdoeswhatyouneedtodo• QuickerDownloads• LessRisks



apt-get -y install --no-install-recommends \openjdk-8-jdk && \rm –rf /var/lib/apt/lists/*

COPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]

Dockerfile Parent Example


Build Stats

ThereisalreadyanofficialimageforOpenJDK.


FROM openjdkCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Build Stats

Usingtheofficialimage.Nomoreinstallsteps.


Build Stats


FROM openjdk:8COPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Build Stats

Useamorespecificversion.


Build Stats




Build Stats

UsingjusttheJREonasmallerbaseimage.


Build Stats

FROM openjdk:8-jre-alpineCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]

Compiling the Code


Current State

GOOD NEWS• Wehaveagoodimageforaruntime• Noadditionaldependencies• Buildsfast• Smallsize

BADNEWS• Wehavenoimageforcompilation• Buildingthejarisexternaltotheimage• Can’tutilizetheDockercache



docker run --rm \-w /home/gradle/project \-v $(pwd):/home/gradle/project \gradle:4.9.0-jdk8-alpine \gradle clean build

Compiling the Code Example

Newbaseimage.


FROM gradle:4.9.0-jdk8-alpineCOPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Newbaseimage.Needaworkdirectory.


Needaworkdirectory.


Needtocopyinthecode.

FROM gradle:4.9.0-jdk8-alpineWORKDIR /home/gradle/project COPY build/libs/docker-app-ds.jar /app/docker-app-ds.jarCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Runthebuild.


Needtocopyinthecode.

FROM gradle:4.9.0-jdk8-alpineWORKDIR /home/gradle/project COPY build.gradle .COPY src srcCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Runthebuild.


Adjustthejarlocation.

FROM gradle:4.9.0-jdk8-alpineWORKDIR /home/gradle/project COPY build.gradle .COPY src srcRUN gradle build --no-daemonCMD ["java", "-jar", "build/libs/docker-app-ds.jar"]




DOESN’TWORK!Gradlethrowsapermissionerror!

Createsthedirectoryasrootuser.FROM gradle:4.9.0-jdk8-alpine

WORKDIR /home/gradle/project COPY build.gradle .COPY src srcRUN gradle build --no-daemonCMD ["java", "-jar", "build/libs/docker-app-ds.jar"]


Compiling the Code ExampleMakesuregradleownseverything.FROM gradle:4.9.0-jdk8-alpine

RUN mkdir /home/gradle/projectWORKDIR /home/gradle/project COPY --chown=gradle:gradle build.gradle .COPY --chown=gradle:gradle src srcRUN gradle build --no-daemonCMD ["java", "-jar", "build/libs/docker-app-ds.jar"]


FROM gradle:4.9.0-jdk8-alpineRUN mkdir /home/gradle/projectWORKDIR /home/gradle/project COPY --chown=gradle:gradle build.gradle .COPY --chown=gradle:gradle src srcRUN gradle build --no-daemonCMD ["java", "-jar", "build/libs/docker-app-ds.jar"]



Build StatsBuildTime(Initial) ~105SecondsBuildTime(Codechange) ~75SecondsImageSize 222MB

Build Stats



PROBLEM:Theimageisbiggerthanitneedstobeandnowcontainsthesourcecodeandacompiler.

QUESTION:Howcanweaddthebuildstepswithoutincreasingthesizeoftheruntime?

ANSWER:Multi-StageBuild


Compiling the Code ExampleNotneededforthebuild.FROM gradle:4.9.0-jdk8-alpine

RUN mkdir /home/gradle/projectWORKDIR /home/gradle/project COPY --chown=gradle:gradle build.gradle .COPY --chown=gradle:gradle src srcRUN gradle build --no-daemonCMD ["java", "-jar", "build/libs/docker-app-ds.jar"]

FROM openjdk:8-jre-alpineCOPY build/libs/docker-app-ds.jar /appCMD ["java", "-jar", "/app/docker-app-ds.jar"]



Thisjarnolongerexists.

FROM gradle:4.9.0-jdk8-alpineRUN mkdir /home/gradle/projectWORKDIR /home/gradle/project COPY --chown=gradle:gradle build.gradle .COPY --chown=gradle:gradle src srcRUN gradle build --no-daemon




Copythejarfromthebuild.

FROM gradle:4.9.0-jdk8-alpine AS builderRUN mkdir /home/gradle/projectWORKDIR /home/gradle/project COPY --chown=gradle:gradle build.gradle .COPY --chown=gradle:gradle src srcRUN gradle build --no-daemon

FROM openjdk:8-jre-alpineCOPY --from=builder \/home/gradle/project/build/libs/docker-app-ds.jar /app/docker-app-ds.jar

CMD ["java", "-jar", "/app/docker-app-ds.jar"]


BuildTime(Initial) ~105SecondsBuildTime(Codechange) ~75SecondsImageSize 222MB

Build Stats





BuildTime(Initial) ~120SecondsBuildTime(Codechange) ~75SecondsImageSize 116MB

Build Stats

NOCODECHANGE…NOREBUILD!

Security


Security Example

OpenJDKrunsas“root”bydefault.





Security Example

Createanewusertorunas.


FROM openjdk:8-jre-alpineRUN addgroup -g 1001 -S appuser && adduser -u -S appuser -G appuserUSER appuserCOPY --from=builder \/home/gradle/project/build/libs/docker-app-ds.jar /app/docker-app-ds.jar



Security Considerations

• Runastheleastprivilegeduser• Usetrusted,minimalimagesasabase• Patchyourimagesforsecurityfixes• Containersdon’tgetpatched,imagesdo

• Don’tputsecureinformationinDockerfiles and/orENVvariables• Addadditionalsecuritycheckstoyourpipeline:• ImageScans• ImageSigning

Multi-Stage Builds


Multi-Stage Builds

• Wearealreadyusingamultistagebuild• Mostcommonuseofmulti-stagebuild

• Youcanhavemorethantwostages• Someexamples:• Pullingartifactsfromdifferentimages• Buildingdifferentversionsofyourimage

• Differentbaseimage(differentflavorsoflinux:alpine,centos,ubuntu)• Differentimagesfordifferentuses(productionvsdevtesting)


FROM node:8.11.3-alpine AS fe-buildRUN apk add --no-cache --virtual .gyp python make g++RUN npm install –g @angular/cli && npm install –g tslint && \

npm install –g concurrently && npm installWORKDIR /app COPY docker-app-ui/ .RUN npm run build:prod

FROM microsoft/dotnet:2.1-sdk-alpine AS be-buildWORKDIR /app COPY docker-app-api/ .RUN dotnet restore && publish –c Release –o out

FROM microsoft/dotnet:2.1-aspnetcore-runtimeWORKDIR /appCOPY --from=be-build /app/out ./COPY --from=fe-build /app/dist wwwrootENTRYPOINT ["dotnet", "docker-app-api.dll"]

Multi-Stage Builds



FROM ubuntu AS devRUN apt-get update && \

apt-get -y install openjdk-8-jdk ssh vimCOPY --from=builder \/home/gradle/project/build/libs/docker-app-ds.jar /app/docker-app-ds.jar


FROM openjdk:8-jre-alpine AS prodRUN addgroup -g 1001 -S appuser && adduser -u -S appuser -G appuserUSER appuserCOPY --from=builder \/home/gradle/project/build/libs/docker-app-ds.jar /app/docker-app-ds.jar


Multi-Stage Buildsdocker build \--target prod \-t docker-app-api:1.0 .

Determineswheretostop

BuildKit


BuildKit

DockerBuildenhancementsfor18.09releaseintroducesamuch-neededoverhaulofthebuildarchitecture.ByintegratingBuildKit,usersshouldseeanimprovementonperformance,storagemanagement,featurefunctionality,andsecurity.

https://docs.docker.com/develop/develop-images/build_enhancements/


Enabling BuildKit

• Enableonaperbuildbasis• Useanenvironmentvariable(DOCKER_BUILDKIT)

DOCKER_BUILDKIT=1 docker build .

• Enableforallbuilds• Updatethedaemon.json orDockerDesktopSettings

{ "features": { "buildkit": true } }






Build Concurrency

fe-build

be-build

Runtime

fe-build be-build

Runtime






Build Concurrency

BuildTime(Initial) ~400SecondsBuildTime(Codechange) VariesImageSize 263MB

Build StatsBuildTime(Initial) ~290SecondsBuildTime(Codechange) VariesImageSize 263MB

Build Stats








Build Concurrency

proddev

buildjar








Build Concurrency

BuildTimeProd(Initial) ~370SecondsBuildTime(Codechange) ~75SecondsImageSizeProd 116MB

Build StatsBuildTimeProd(Initial) ~120SecondsBuildTime(Codechange) ~75SecondsImageSizeProd 116MB

Build Stats


Experimental Dockerfile Options

• Toenable,startthefilewiththis:# syntax=docker/dockerfile:experimental

• Enables:• --mountsyntax• --securitysyntax• --networksyntax

• AllexecutedviaRUNcommands


Security Syntax

• Allowscommandstorunininsecuremode• Equivalenttodocker run --privileged• Optionsareinsecure&sandbox• Requiresthesecurity.insecure entitlement• ExecutedPerRUN command

# syntax = docker/dockerfile:experimentalFROM ubuntu RUN --security=insecure cat /proc/self/status | grep CapEff


Network Syntax

• Allowscommandstorunwithdifferentnetworkoptions• Optionsarenone,host&default• Requiresthenetwork.host entitlement• ExecutedPerRUN command

# syntax = docker/dockerfile:experimentalFROM python:3.6 ADD mypackage.tgz wheels/ RUN --network=none pip install \

--find-links wheels mypackage


Mount Syntaxes

Numerousmountoptionsareavailable:• RUN --mount=type=bind

• Let’syoubinddirectoriesinsteadofcopying(stillneedstobeinthecontext)• RUN --mount=type=cache

• Cachedirectoriesforcompilersandpackages• RUN --mount=type=tmpfs

• Mountatemporaryfilesystem• RUN --mount=type=secret

• Mountasecretthatcanbeused,butnotstoredintheimage.• RUN --mount=type=ssh

• AccessfilesviaSSH(forinstance,logintoGitusingyourexistingSSHkey)


Cache Mount


Build Stats

Mountsasroot


Build Stats

# syntax = docker/dockerfile:experimentalFROM gradle:4.9.0-jdk8-alpine AS builderUSER rootWORKDIR /home/gradle/project COPY build.gradle .COPY src srcRUN --mount=type=cache,target=/home/gradle/.gradle gradle build --no-daemon




Secret Mount

MatchedbyID

# syntax = docker/dockerfile:experimentalFROM ubuntu RUN --mount=type=secret,target=.,id=secret_script \

bash -c my_script.sh

docker build \-t secret_image--secret id=secret_script,src=my_script.sh \.

Questions?

Nick Schultz

Principal Consultant

(920) 210-1429

Documents

DockerfileBest Practices - NVISIA · 2019-09-19 · NVISIA -Confidential and Proprietary Example Dockerfile FROMubuntu:18.04 COPY. /app RUNmake /app CMDpython /app/app.py What does