Upload
stefan-schimanski
View
349
Download
1
Embed Size (px)
Citation preview
Nune Isabekyan,https://x-team.com/blog/introduction-kubernetes-architecture/
Disclaimer:Kubernetes happens to be able to launch pods.It‘s even quite good atit.
We willnotlaunch pods today.
RestfulhttpAPI
//version
/api/api/v1/pods/api/v1/pods/status
/apis/apis/batch/apis/batch/v2alpha1/apis/batch/v2alpha1/jobs/apis/batch/v2alpha1/cronjobs/apis/batch/v1/apis/batch/v1/jobs
kube-apiserver
kubeletproxy
Node:
$kubectl create -ffoo.yaml
User:
scheduler controllermanager
apiserveringresscontroller
Master:
cloud nativeapps
Pods:
$kube-apiserver--secure-port 0--etcd-servershttp://127.0.0.1:2379--service-cluster-ip-range10.0.0.0/16--storage-backendetcd2--storage-media-typeapplication/json
$ etcd
$ kubectl config set-clusterlocal --server=http://127.0.0.1:8080$kubectl config set-context local --cluster=local$kubectl config use-context local
$ kubectl get namespaces –v=7$ kubectl get namespace default -ojson
$ kubectl annotate namespace default meetup=hello
$curl http://127.0.0.1:8080/api/v1/namespaces/default
$ etcdctl get /--recursive$ etcdctl get /registry/namespaces/default$ etcdctl -oextended get /registry/namespaces/default
$httpGET http://127.0.0.1:8080/api/v1/namespaces/default{
"apiVersion":"v1","kind": "Namespace", "metadata": {
"annotations": { "meetup": “hallo"
}, "creationTimestamp":"2017-03-10T07:51:39Z","name": "default", "resourceVersion": “73",
}, "spec": { "finalizers": ["kubernetes“]},"status": { "phase": "Active“ }
}
$httpGET http://127.0.0.1:8080/api/v1/namespaces/default |jq ".metadata.annotations[\"meetup\"]= \"$(date)\""|httpPUT http://127.0.0.1:8080/api/v1/namespaces/default
HTTP/1.1200OKContent-Length:341Content-Type:application/jsonDate:Fri,10Mar201708:28:01GMT
$while true;dohttpGEThttp://127.0.0.1:8080/api/v1/namespaces/default |
jq ".metadata.annotations[\"meetup\"]=\"$(date)\""|http--check-statusPUThttp://127.0.0.1:8080/api/v1/namespaces/default ||break
done
HTTP/1.1409ConflictContent-Length:310Content-Type:application/jsonDate:Fri,10Mar201708:27:58GMT{
"apiVersion":"v1","code":409,"details":{
"kind":"namespaces","name":"default“
},"kind":"Status","message":"Operationcannot be fulfilled onnamespaces \"default\":the object has been
modified;please apply your changes to the latest version and try again","metadata":{},"reason":"Conflict","status":"Failure“
}
/apis/batch/v2alpha1/jobs
Apigroup VersionResource
HTTPpaths:
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{"apiVersion":"v1","kind":"Status","metadata":{},"code":409,"message":“...","status":"Failure“}/apis/batch/v2alpha1/jobs
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
Resource vs.Kind
httppath vs.logical object
/apis/extensions/v1alpha1/jobs
/apis/batch/v2alpha1/jobs
/apis/batch/v1/jobs {“apiVersion“:“v1alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
/apis/extensions/v1alpha1/jobs
/apis/batch/v2alpha1/jobs
/apis/batch/v1/jobs {“apiVersion“:“v1alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
/apis/extensions/v1alpha1/jobs/nightly
/apis/batch/v2alpha1/jobs/nightly
/apis/batch/v1/jobs/nightly
{“apiVersion“:“v1alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{...}}
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v1beta1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
*Iomitted the namespace in/apis/batch/v1/jobs/namespaces/default/nightly
{“apiVersion“:“v1alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v2alpha1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
{“apiVersion“:“v1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
etcd
{“apiVersion“:“v1“,“kind“:“Job“,“metadata“:{
“name“:“nightly“},“spec“:{ ...}}
Protobufor
JSON
storageversion
encoding
Iwant my own kindsand store them inthe apiserver and use kubectl.
Discoveryhow kubectl knows which kinds/resources exist
$http127.0.0.1:8080/apis/{
"groups":[{"name": "batch","preferredVersion": {"groupVersion": "batch/v1", "version": "v1“},"versions": [{"groupVersion": "batch/v1","version":"v1"}]
},...]}
$http127.0.0.1:8080/apis/batch/v1{
"apiVersion":"v1","groupVersion":"batch/v1","kind":"APIResourceList","resources":[{
"kind":"Job","name":"jobs","namespaced":true,"verbs":["create","delete","deletecollection",
"get","list","patch","update","watch“]
},...]}
resource name⇒ /apis/batch/v1/jobs
ThirdPartyResources
ThirdPartyResources
*they are marked as beta inthe API,butbecame beta before we even had alpha.
apiVersion:extensions/v1beta1kind:ThirdPartyResourcemetadata:
name:databases.example.comdescription:"Aspecification of aSQLdatabase.“versions:
- name:v1
$kubectl create –fdatabases-tpr.yaml
apiVersion:extensions/v1beta1kind:ThirdPartyResourcemetadata:
name:databases.example.comdescription:"Aspecification of aSQLdatabase.“versions:
- name:v1
$kubectl create –fdatabases-tpr.yaml $kubectl create –fwordpress-databases.yaml
apiVersion:example.com/v1kind:Databasesmetadata:
name:wordpressspec:
user:wppassword:secretencoding:unicode
apiVersion:extensions/v1beta1kind:ThirdPartyResourcemetadata:
name:database.example.comdescription:"Aspecification of aSQLdatabase.“versions:
- name:v1
$kubectl create –fdatabases-tpr.yaml $kubectl create –fwordpress-database.yaml
apiVersion:example.com/v1kind:Databasemetadata:
name:wordpressspec:
user:wppassword:secretencoding:unicode
$ kubectl get databases –w --no-headerswordpress <none>{"apiVersion":"example.com/v1","kind":"Databases",...wordpress <none>{"apiVersion":"example.com/v1","kind":"Databases",...
https://github.com/kubernetes/features/issues/95
ThirdPartyResources are limited
• no version conversion• no defaulting• no validation• no subresources (scale,status)• no admission• alpha⇒ APImight change
• but: demand is high,expect improvements in1.7+
• Today‘s users of TPRs:https://gist.github.com/philips/a97a143546c87b86b870a82a753db14c
Controllerswhere is the business logic?
$while true;dohttpGEThttp://127.0.0.1:8080/api/v1/namespaces/default |
jq ".metadata.annotations[\"meetup\"]=\"$(date)\""|http--check-statusPUThttp://127.0.0.1:8080/api/v1/namespaces/default ||break
done⟲
$kubectl get namespaces -w --no-headers|
while read NSSTATUSTIME;do#dowhatever you likehere,e.g.change the namespaceecho"$NSchanged“
done⟲
$curl -f'http://127.0.0.1:8080/api/v1/namespaces?watch=true&resourceVersion=4711‘
{"type":"ADDED","object":{"kind":"Namespace","apiVersion":"v1","metadata":{"name ...{"type":“MODIFIED","object":{"kind":"Namespace","apiVersion":"v1","metadata":{"name ...{"type":“DELETED","object":{"kind":"Namespace","apiVersion":"v1","metadata":{"name ...
APIAggregation
Why
• ThirdPartyResources are limited• no version conversion• no defaulting• no validation• no subresources (scale,status)• no admission
• Some things need full powerof Go• Servicecatalog• OpenShift PaaS• other powerfulAPIs
≫ Goal:allow powerfulextensions without modifying Kubernetes itself
Alphainv1.6:k8s.io/apiserver
• generic apiserver library inGo• today used inside
• kube-apiserver• federation apiserver• service catalog
• allows creation of custom apiservers inacouple hundred lines of code
• each custom apiserver is its own process,communicating viaHTTPS• delegates authentication/authorization to kube-apiserver• uses etcd storage (possibly shared with kube)
Nune Isabekyan,https://x-team.com/blog/introduction-kubernetes-architecture/
kube-apiserver service catalog PaaS
kube-aggregator
kube-apiserver service catalog PaaS
kube-aggregator
API API API
API
Pods Jobs ... announcement ... build test project
kube-apiserver kube-apiserver kube-apiserver
federationapiserver
API API API
API
Pods replicasets Pods replicasets Pods replicasets
deployment service
Europe US Asia
Notthis:
federationcontrollers⟲
federated resources:
availability zones +regions:
kube-apiserver service catalog PaaS
kube-aggregator
discovery
discovery
kube-apiserver service catalog PaaS
kube-aggregator
GET
GET/apis/servicecatalog/subscription/database-prod-wordpress
kube-apiserver service catalog PaaS
kube-aggregator
GET/apis/servicecatalog/subscription/database-prod-wordpress
GET
RBAC Namespace
kube-apiserver service catalog PaaS
kube-aggregator
GET
Vision:$ helm install service-catalog$ kubectl create service-announcement ....
GET/apis/servicecatalog/subscription/database-prod-wordpress
Status
• willbe part of Kubernetes 1.6as analpha• https://github.com/kubernetes/sample-apiserver• potentially kube-aggregator integrated into kube-apiserver in1.7
Links
• https://docs.google.com/document/d/1y16jKL2hMjQO0trYBJJSczPAWj8vAgNFrdTZeCincmI/Two Ways to Extend the K8sAPI - Addresources to aKubernetes APIwith TPRor AA• https://github.com/kubernetes/community/blob/master/contributors/design-proposals/aggregated-api-servers.md• https://gist.github.com/philips/a97a143546c87b86b870a82a753db14c - Kubernetes Third-PartyResource Users
https://github.com/kubernetes/community/blob/master/contributors/devel/client-libraries.md
Backup
RestfulhttpAPI
//version
/api/api/v1/pods/api/v1/pods/status
/apis/apis/batch/apis/batch/v2alpha1/apis/batch/v2alpha1/jobs/apis/batch/v2alpha1/cronjobs/apis/batch/v1beta1/apis/batch/v1beta1/jobs
/apis/batch/v2alpha1/jobs
GroupVersionResourceHTTPpaths:
InGo: gvk :=schema.GroupVersionKind{Group:“batch“,Version:“v2alpha1“,Kind:“Job“}obj :=api.Scheme.New(gvk)codec :=api.Codecs.LegacyCodec(gvk.GroupVersion())codec.Decode(reqBody,gvk,obj)
typeJob struct {metav1.TypeMetametav1.ObjectMetaSpec JobSpecStatusJobStatus
}pkg/apis/batch/v2alpha1/types.go
typeTypeMeta struct {KindstringAPIVersion string
}
typeObjectMeta struct {Namestring...
}
RestfulhttpAPI
//version
/api/api/v1/pods/api/v1/pods/status
/apis/apis/batch/apis/batch/v2alpha1/apis/batch/v2alpha1/jobs/apis/batch/v2alpha1/cronjobs/apis/batch/v1beta1/apis/batch/v1beta1/jobs
MaxInFlightLimit
Timeo
utForNon
LongRu
nningReq
uests
PanicRe
covery
CORS
Authen
tication
Audit
Impe
rson
ation
Authoriza
tion
k8s.io/apiserver/pkg/server.DefaultBuildHandlerChain
„Filters“
k8s.io/apiserver/pkg/server/routes/index.go – /k8s.io/apiserver/pkg/server/routes/version.go – /versionk8s.io/apiserver/pkg/server/routes/swagger.go – /swaggerapik8s.io/apiserver/pkg/server/routes/openapi.go – /swagger.json
„Routes“
mux
k8s.io/apiserver/pkg/endpoints.APIGroupVersion.InstallREST
AddSupportedResourcesWebService – /apis/batch/v2alpha1
k8s.io/apiserver/pkg/endpoints.APIInstaller.Install
/apis/batch/v2alpha1/jobs/apis/batch/v2alpha1/cronjobs...
With
Requ
estIn
fo
ctx.RequestInfo
RestfulhttpAPI
mux
k8s.io/apiserver/pkg/endpoints.APIGroupVersion.InstallREST
AddSupportedResourcesWebService – /apis/batch/v2alpha1
k8s.io/apiserver/pkg/endpoints.APIInstaller.Install/apis/batch/v2alpha1/jobs/apis/batch/v2alpha1/cronjobs...
pkg/apis/batchtype Jobsstruct
pkg/apis/batch/v2alpha1type Jobsstruct
api.Schemek8s.io/apiserver
pkg/api api.Scheme.Convert(&internalJob,&v2alohaJob)
/apis/batch/v2alpha1/jobsGETPUTPOSTDELETE...
/status/scale/proxy...
subresources
RestfulhttpAPI
mux
pkg/apis/batchtype Jobsstruct
pkg/apis/batch/v2alpha1type Jobsstruct
api.Scheme api.Scheme.Convert(&job,&v1job)
POST/apis/batch/v2alpha1/jobs
k8s.io/apiserverpkg/endpoints/handlers.CreateNamedResource
binaryJSON
payload
Gostructv2alpha1.Job
HTTPRequest
Gostructinternal.Job
Storek8s.io/apiserver
pkg/registry/generic
Storagek8s.io/apiserverpkg/storage/etcd3
ProtoBufJob
Gostructv2alpha1.Job
etcd
type Scheme struct• AddKnownTypes(gv, obj Object) • Default(src Object)• Copy(src Object) Object• Convert(in, out interface{})• New(gvk) Object
ApiGrouppkg/apis/batchpkg/apis/batch/v1pkg/apis/batch/v2alpha1pkg/apis/batch/register.gopgk/apis/batch/install
GroupVersionKindResource
type Object interface• GetObjectKind() string
client-go/pkg/api.Schemeclient-go/pkg/api.Codecs
Discoverytype APIGroupList structtype APIVersions structtype APIResourceList struct
GroupVersionKind „gvk“GroupVersionResource
Unversioned typesUnstructuredList
Registry / Storagetype Storage interfacetype Lister interfacetype Updater interfacetype Getter interfacetype Deleter interface....
deepcopy-genconversion-gendefaulting-gen
Code Generation
type OwnerReference structtype ObjectReference structtype TypeMeta structtype ObjectMeta struct
Meta
api.Schemeapi.Codecsapi.Registryapi.GroupFactoryRegistry
Globals