adding historical notes for 2014 contributor summit
This commit is contained in:
parispittman
parent
f8c08b18ff
commit
6f726048ae
|
|
@ -0,0 +1,200 @@
|
||||||
|
# Kubernetes Contributor Conference, 2014-12-03 to 12-05
|
||||||
|
**Full notes:** (Has pictures; Shared with k-dev mailing list) (https://docs.google.com/document/d/1cQLY9yeFgxlr_SRgaBZYGcJ4UtNhLAjJNwJa8424JMA/edit?usp=sharing)
|
||||||
|
**Organizers:** thockin and bburns
|
||||||
|
**26 Attendees from:** Google, Red Hat, CoreOS, Box
|
||||||
|
**This is a historical document. No typo or grammar correction PRs needed.**
|
||||||
|
|
||||||
|
Last modified: Dec. 8. 2014
|
||||||
|
|
||||||
|
# Clustering and Cluster Formation
|
||||||
|
Goal: Decide how clusters should be formed and resized over time
|
||||||
|
Models for building clusters
|
||||||
|
* Master in charge - asset DB
|
||||||
|
Dynamic join - ask to join
|
||||||
|
* How Kelsey Hightower has seen this done on bare metal
|
||||||
|
Use Fleet as a machine database
|
||||||
|
A Fleet agent is run on each node
|
||||||
|
Each node registers its information in etcd when it comes up
|
||||||
|
Only security is that etcd expects the node to have a cert signed by a specific CA
|
||||||
|
Run an etcd proxy on each node
|
||||||
|
Don't run any salt scripts, everything is declarative
|
||||||
|
Just put a daemon (kube-register) on a machine to become part of the cluster
|
||||||
|
brendanburns: basically using Fleet as the cloud provider
|
||||||
|
* Puppet model - whitelist some cert and/or subnet that you want to trust everything in
|
||||||
|
One problem - if CA leaks, have to replace certs on all nodes
|
||||||
|
* briangrant: we may want to support adding nodes that aren't trusted, only scheduling work from the nodes' owner on them
|
||||||
|
* lavalamp: we need to differentiate between node states:
|
||||||
|
In the cluster
|
||||||
|
Ready to accept work
|
||||||
|
Trusted to accept work
|
||||||
|
* Proposal:
|
||||||
|
New nodes initiate contact with the master
|
||||||
|
Allow multiple config options for how trust can be established - IP, cert, etc.
|
||||||
|
Each new node only needs one piece of information - how to find the master
|
||||||
|
Can support many different auth modes - let anyone in, whitelist IPs, a particular signed cert, queue up requests for an admin to approve, etc.
|
||||||
|
Default should be auto-register with no auth/approval needed
|
||||||
|
Auth-ing is separate from registering
|
||||||
|
Supporting switching between permissive and strict auth modes:
|
||||||
|
Each node should register a public key such that if the auth mode is changed to require a cert upon registration, old nodes won't break
|
||||||
|
kelseyhightower: let the minion do the same thing that kube-register currently does
|
||||||
|
Separate adding a node to the cluster from declaring it as schedulable
|
||||||
|
* Use cases:
|
||||||
|
Kick the tires, everything should be automagic
|
||||||
|
Professional that needs security
|
||||||
|
* Working group for later: Joe, Kelsey, Quintin, Eric Paris
|
||||||
|
# Usability
|
||||||
|
* Getting started
|
||||||
|
Want easy entry for Docker users
|
||||||
|
Library/registry of pod templates
|
||||||
|
* GUI - visualization of relationships and dependencies, workflows, dashboards, ways to learn, first impressions
|
||||||
|
Will be easiest to start with a read-only UI before worrying about read-write workflows
|
||||||
|
* Docs
|
||||||
|
Need to refactor getting started guides so that there's one common guide
|
||||||
|
Each cloud provider will just have its own short guide on how to create a cluster
|
||||||
|
Need a simple test that can verify whether your cluster is healthy or diagnose why it isn't
|
||||||
|
Make it easier to get to architecture/design doc from front page of github project
|
||||||
|
Table of contents for docs?
|
||||||
|
Realistic examples
|
||||||
|
Kelsey has found that doing a tutorial of deploying with a canary helped make the value of labels clear
|
||||||
|
* CLI
|
||||||
|
Annoying when local auth files and config get overwritten when trying to work with multiple clusters
|
||||||
|
Like when running e2e tests
|
||||||
|
* Common friction points
|
||||||
|
External IPs
|
||||||
|
Image registry
|
||||||
|
Secrets
|
||||||
|
Deployment
|
||||||
|
Stateful services
|
||||||
|
Scheduling
|
||||||
|
Events/status
|
||||||
|
Log access
|
||||||
|
* Working groups
|
||||||
|
GUI - Jordan, Brian, Max, Satnam
|
||||||
|
CLI - Jeff, Sam, Derek
|
||||||
|
Docs - Proppy, Kelsey, TJ, Satnam, Jeff
|
||||||
|
Features/Experience - Dawn, Rohit, Kelsey, Proppy, Clayton: https://docs.google.com/document/d/1hqn6FtBNMe0sThbciq2PbE_P5BONBgCzHST4gz2zj50/edit
|
||||||
|
|
||||||
|
|
||||||
|
# v1beta3 discussion
|
||||||
|
|
||||||
|
12-04-2014
|
||||||
|
Network -- breakout
|
||||||
|
* Dynamic IP
|
||||||
|
Once we support live migration, IP assigned for each POD has to move together, which might be broken the underneath.
|
||||||
|
We don’t have introspection, which makes supporting various network topology harder.
|
||||||
|
External IP is an important part.
|
||||||
|
There’s a kick-the-tires mode and full-on mode (for GCE, AWS - fully featured).
|
||||||
|
How do we select kick-the-tires ? Weave, Flannel, Calico: pick one.
|
||||||
|
Someone does a comparison. thockin@ would like help in evaluating these tech against some benchmarks. Eric Paris can help - has a bare-metal setup. We’ll have a benchmark setup for evaluation.
|
||||||
|
We need to have two real use-cases at least - a webserver example; can 10 pods find each other. lavalamp@ working on a test.
|
||||||
|
If docker picks up a plugin model, we can use that.
|
||||||
|
Cluster will be dynamically change, we need to design a flexible network plugin API to accomplish this.
|
||||||
|
Flannel two things: network allocation through etcd and traffic routing w/ overlays. Also programs underlay networks (like GCE). Flannel will do IP allocation, not hard-coded.
|
||||||
|
One special use case: per node, there are only 20 ips could be allocated. Scheduler might need to know the limitation: OUT-OF-IP(?)
|
||||||
|
Different cloud providers, but OVS is a common mechanism
|
||||||
|
We might need Network Grids at the end
|
||||||
|
ACTIONS: better doc, test.
|
||||||
|
* Public Services
|
||||||
|
Hard problem: Have to scale to GCE, GCE load balancer cannot target to arbitrary IP, only can target to a VM for now.
|
||||||
|
Until we have an external IP, you cannot build a HA public service.
|
||||||
|
We can run Digital Ocean on top of kubernetes
|
||||||
|
Issue: When starting a public service, there is internal IP assigned. It is accessable from node within cluster, but not from outside. Now we have a 3-tier services, how to access one service from outside The issue is how to take this internal accessible service externalized. General solution: forwarding the traffic outside to the internal IP. First action, teach kubernetes mapping.
|
||||||
|
We need a registry of those public IPs. All traffic comes to that IP will be forwarded to proper IP internally.
|
||||||
|
public service can register with DNS, and do a intermiddle load balancing outside cluster / kubernetes. Label query to tell the endpoint.
|
||||||
|
K8s proxy can be L3 LB, and listen to the external IPs, it also talk to k8s service DB and find internal services; then goes to L7 LB, which could be HAP proxy, scheduled as a pod, it talks to Pods DB, find a cluster of pods to forward the traffic.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Two types of services: mapping external IPs and L3 LB to map to pods. L7 LB can access the IPs assigned to pods.
|
||||||
|
Policy: Add more nodes, more external IPs can be used.
|
||||||
|
Issue1: how to take external IP to map to a list of pods, L3 LB part.
|
||||||
|
Issue2: how to slice those external IPs: general pool vs. private pools.
|
||||||
|
* IP-per-service, visibility, segmenting
|
||||||
|
* Scale
|
||||||
|
* MAC
|
||||||
|
|
||||||
|
# Roadmap
|
||||||
|
|
||||||
|
* Should be driven by scenarios / use cases -- breakout
|
||||||
|
* Storage / stateful services -- breakout
|
||||||
|
Clustered databases / kv stores
|
||||||
|
Mongo
|
||||||
|
MySQL master/slave
|
||||||
|
Cassandra
|
||||||
|
etcd
|
||||||
|
zookeeper
|
||||||
|
redis
|
||||||
|
ldap
|
||||||
|
Alternatives
|
||||||
|
local storage
|
||||||
|
durable volumes
|
||||||
|
identity associated with volumes
|
||||||
|
lifecycle management
|
||||||
|
network storage (ceph, nfs, gluster, hdfs)
|
||||||
|
volume plugin
|
||||||
|
flocker - volume migration
|
||||||
|
“durable” data (as reliable as host)
|
||||||
|
* Upgrading Kubernetes
|
||||||
|
master components
|
||||||
|
kubelets
|
||||||
|
OS + kernel + Docker
|
||||||
|
* Usability
|
||||||
|
Easy cluster startup
|
||||||
|
Minion registration
|
||||||
|
Configuring k8s
|
||||||
|
move away from flags in master
|
||||||
|
node config distribution
|
||||||
|
kubelet config
|
||||||
|
dockercfg
|
||||||
|
Cluster scaling
|
||||||
|
CLI + config + deployment / rolling updates
|
||||||
|
Selected workloads
|
||||||
|
* Networking
|
||||||
|
External IPs
|
||||||
|
DNS
|
||||||
|
Kick-the-tires networking implementation
|
||||||
|
* Admission control not required for 1.0
|
||||||
|
* v1 API + deprecation policy
|
||||||
|
* Kubelet API well defined and versioned
|
||||||
|
* Basic resource-aware scheduling -- breakout
|
||||||
|
require limits?
|
||||||
|
auto-sizing
|
||||||
|
* Registry
|
||||||
|
Predictable deployment (config-time image resolution)
|
||||||
|
Easy code->k8s
|
||||||
|
Simple out-of-the box setup
|
||||||
|
One or many?
|
||||||
|
Proxy?
|
||||||
|
Service?
|
||||||
|
Configurable .dockercfg
|
||||||
|
* Productionization
|
||||||
|
Scalability
|
||||||
|
100 for 1.0
|
||||||
|
1000 by summer 2015
|
||||||
|
HA master -- not gating 1.0
|
||||||
|
Master election
|
||||||
|
Eliminate global in-memory state
|
||||||
|
IP allocator
|
||||||
|
Operations
|
||||||
|
Sharding
|
||||||
|
Pod getter
|
||||||
|
Kubelets need to coast when master down
|
||||||
|
Don’t blow away pods when master is down
|
||||||
|
Testing
|
||||||
|
More/better/easier E2E
|
||||||
|
E2E integration testing w/ OpenShift
|
||||||
|
More non-E2E integration tests
|
||||||
|
Long-term soaking / stress test
|
||||||
|
Backward compatibility
|
||||||
|
Release cadence and artifacts
|
||||||
|
Export monitoring metrics (instrumentation)
|
||||||
|
Bounded disk space on master and kubelets
|
||||||
|
GC of unused images
|
||||||
|
* Docs
|
||||||
|
Reference architecture
|
||||||
|
* Auth[nz]
|
||||||
|
plugins + policy
|
||||||
|
admin
|
||||||
|
user->master
|
||||||
|
master component->component: localhost in 1.0
|
||||||
|
kubelet->master
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
events/elections/2017/
|
events/elections/2017/
|
||||||
vendor/
|
vendor/
|
||||||
sig-contributor-experience/contribex-survey-2018.csv
|
sig-contributor-experience/contribex-survey-2018.csv
|
||||||
|
events/2014
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue