Kubernetes ApiServer三大server權(quán)限與數(shù)據(jù)存儲(chǔ)解析
確立目標(biāo)
- 理解啟動(dòng)kube-apiserver的權(quán)限相關(guān)的三個(gè)核心概念
Authentication/Authorization/Admission分別是認(rèn)證,授權(quán),準(zhǔn)入 - 理解kube-apiserver是中的管理核心資源的
KubeAPIServer是怎么啟動(dòng)的 - 理解Pod發(fā)送到
kube-apiserver后是怎么保存的
Run
kube-apiserver的啟動(dòng) 代碼在cmd/kube-apiserver
// 類似kubectl的源代碼,kube-apiserver的命令行工具也使用了cobra,我們很快就能找到啟動(dòng)的入口
RunE: func(cmd *cobra.Command, args []string) error {
// 這里包含2個(gè)參數(shù),前者是參數(shù)completedOptions,后者是一個(gè)stopCh <-chan struct{}
return Run(completedOptions, genericapiserver.SetupSignalHandler())
}
/*
在這里,我們可以和kubectl結(jié)合起來思考:
kubectl是一個(gè)命令行工具,執(zhí)行完命令就退出;kube-apiserver是一個(gè)常駐的服務(wù)器進(jìn)程,監(jiān)聽端口
這里引入了一個(gè)stopCh <-chan struct{},可以在啟動(dòng)后,用一個(gè) <-stopCh 作為阻塞,使程序不退出
用channel阻塞進(jìn)程退出,對(duì)比傳統(tǒng)的方法 - 用一個(gè)永不退出的for循環(huán),是一個(gè)很優(yōu)雅的實(shí)現(xiàn)
*/
func Run(completeOptions completedServerRunOptions, stopCh <-chan struct{}) error {
// 這里進(jìn)行創(chuàng)建服務(wù)鏈
server, err := CreateServerChain(completeOptions)
if err != nil {
return err
}
prepared, err := server.PrepareRun()
if err != nil {
return err
}
return prepared.Run(stopCh)
}
Three Servers
// 在CreateServerChain這個(gè)函數(shù)下,創(chuàng)建了3個(gè)server
func CreateServerChain(){
// API擴(kuò)展服務(wù),主要針對(duì)CRD
createAPIExtensionsServer(){}
// API核心服務(wù),包括常見的Pod/Deployment/Service,我們今天的重點(diǎn)聚焦在這里
// 我會(huì)跳過很多非核心的配置參數(shù),一開始就去研究細(xì)節(jié),很影響整體代碼的閱讀效率
CreateKubeAPIServer(){}
// API聚合服務(wù),主要針對(duì)metrics
createAggregatorServer(){}
//細(xì)節(jié)是第二個(gè)ApiServer需要第一個(gè)server的配置,第三個(gè)server會(huì)要第二個(gè)server的配置,最后返回的是聚合server
// 這些server的config都是由一個(gè)GenericConfig和一個(gè)ExtraConfig組成 有自己的特點(diǎn)和鏈上的
return aggregatorServer, nil
}
KubeAPIServer
// 創(chuàng)建配置的流程
func CreateKubeAPIServerConfig(){
// 創(chuàng)建通用配置genericConfig
genericConfig, versionedInformers, insecureServingInfo, serviceResolver, pluginInitializers, admissionPostStartHook, storageFactory, err := buildGenericConfig(s.ServerRunOptions, proxyTransport)
}
GenericConfig
// 通用配置的創(chuàng)建
func buildGenericConfig(s *options.ServerRunOptions,proxyTransport *http.Transport){
// Insecure對(duì)應(yīng)的非安全的通信,也就是HTTP
if lastErr = s.InsecureServing...
// Secure對(duì)應(yīng)的就是HTTPS
if lastErr = s.SecureServing...
// OpenAPIConfig是對(duì)外提供的API文檔
genericConfig.OpenAPIConfig = genericapiserver.DefaultOpenAPIConfig()
// 這一塊是storageFactory的實(shí)例化,可以看到采用的是etcd作為存儲(chǔ)方案
storageFactoryConfig := kubeapiserver.NewStorageFactoryConfig()
storageFactoryConfig.APIResourceConfig = genericConfig.MergedResourceConfig
completedStorageFactoryConfig, err := storageFactoryConfig.Complete(s.Etcd)
storageFactory, lastErr = completedStorageFactoryConfig.New()
// Authentication 認(rèn)證相關(guān)
if lastErr = s.Authentication.ApplyTo()...
// Authorization 授權(quán)相關(guān)
genericConfig.Authorization.Authorizer, genericConfig.RuleResolver, err = BuildAuthorizer()
// Admission 準(zhǔn)入機(jī)制
err = s.Admission.ApplyTo()
}
Authentication
func (o *BuiltInAuthenticationOptions) ApplyTo(){
// 前面都是對(duì)認(rèn)證config進(jìn)行參數(shù)設(shè)置,這里才是真正的實(shí)例化
authInfo.Authenticator, openAPIConfig.SecurityDefinitions, err = authenticatorConfig.New()
}
// New這塊的代碼,我們要抓住核心變量authenticators和tokenAuthenticators,也就是各種認(rèn)證方法
func (config Config) New() (authenticator.Request, *spec.SecurityDefinitions, error) {
// 核心變量authenticators和tokenAuthenticators
var authenticators []authenticator.Request
var tokenAuthenticators []authenticator.Token
if config.RequestHeaderConfig != nil {
// 1. 添加requestHeader
authenticators = append(authenticators, authenticator.WrapAudienceAgnosticRequest(config.APIAudiences, requestHeaderAuthenticator))
}
if config.ClientCAContentProvider != nil {
// 2. 添加ClientCA
authenticators = append(authenticators, certAuth)
}
if len(config.TokenAuthFile) > 0 {
// 3. token 添加tokenfile
tokenAuthenticators = append(tokenAuthenticators, authenticator.WrapAudienceAgnosticToken(config.APIAudiences, tokenAuth))
}
// 4. token 添加 service account,分兩種來源
if len(config.ServiceAccountKeyFiles) > 0 {
tokenAuthenticators = append(tokenAuthenticators, serviceAccountAuth)
}
if utilfeature.DefaultFeatureGate.Enabled(features.TokenRequest) && config.ServiceAccountIssuer != "" {
tokenAuthenticators = append(tokenAuthenticators, serviceAccountAuth)
}
if config.BootstrapToken {
if config.BootstrapTokenAuthenticator != nil {
// 5. token 添加 bootstrap
tokenAuthenticators = append(tokenAuthenticators, authenticator.WrapAudienceAgnosticToken(config.APIAudiences, config.BootstrapTokenAuthenticator))
}
}
if len(config.OIDCIssuerURL) > 0 && len(config.OIDCClientID) > 0 {
// 6. token 添加 oidc
Authenticators = append(tokenAuthenticators, authenticator.WrapAudienceAgnosticToken(config.APIAudiences, oidcAuth))
}
if len(config.WebhookTokenAuthnConfigFile) > 0 {
// 7. token 添加 webhook
tokenAuthenticators = append(tokenAuthenticators, webhookTokenAuth)
}
// 8. 組合tokenAuthenticators到tokenAuthenticators中
if len(tokenAuthenticators) > 0 {
tokenAuth := tokenunion.New(tokenAuthenticators...)
if config.TokenSuccessCacheTTL > 0 || config.TokenFailureCacheTTL > 0 {
tokenAuth = tokencache.New(tokenAuth, true, config.TokenSuccessCacheTTL, config.TokenFailureCacheTTL)
}
authenticators = append(authenticators, bearertoken.New(tokenAuth), websocket.NewProtocolAuthenticator(tokenAuth))
}
// 9. 沒有任何認(rèn)證方式且啟用了Anonymous
if len(authenticators) == 0 {
if config.Anonymous {
return anonymous.NewAuthenticator(), &securityDefinitions, nil
}
return nil, &securityDefinitions, nil
}
// 10. 組合authenticators
authenticator := union.New(authenticators...)
return authenticator, &securityDefinitions, nil
}
復(fù)雜的Authentication模塊的初始化順序我們看完了,有初步的了解即可,沒必要去強(qiáng)制記憶其中的加載順序。
Authorization
func BuildAuthorizer(){
// 與上面一致,實(shí)例化是在這個(gè)New中
return authorizationConfig.New()
}
// 不得不說,Authorizer這塊的閱讀體驗(yàn)更好
func (config Config) New() (authorizer.Authorizer, authorizer.RuleResolver, error) {
// 必須傳入一個(gè)Authorizer機(jī)制
if len(config.AuthorizationModes) == 0 {
return nil, nil, fmt.Errorf("at least one authorization mode must be passed")
}
var (
authorizers []authorizer.Authorizer
ruleResolvers []authorizer.RuleResolver
)
for _, authorizationMode := range config.AuthorizationModes {
// 具體的mode定義,可以跳轉(zhuǎn)到對(duì)應(yīng)的鏈接去看,不細(xì)講
switch authorizationMode {
case modes.ModeNode:
authorizers = append(authorizers, nodeAuthorizer)
ruleResolvers = append(ruleResolvers, nodeAuthorizer)
case modes.ModeAlwaysAllow:
authorizers = append(authorizers, alwaysAllowAuthorizer)
ruleResolvers = append(ruleResolvers, alwaysAllowAuthorizer)
case modes.ModeAlwaysDeny:
authorizers = append(authorizers, alwaysDenyAuthorizer)
ruleResolvers = append(ruleResolvers, alwaysDenyAuthorizer)
case modes.ModeABAC:
authorizers = append(authorizers, abacAuthorizer)
ruleResolvers = append(ruleResolvers, abacAuthorizer)
case modes.ModeWebhook:
authorizers = append(authorizers, webhookAuthorizer)
ruleResolvers = append(ruleResolvers, webhookAuthorizer)
case modes.ModeRBAC:
authorizers = append(authorizers, rbacAuthorizer)
ruleResolvers = append(ruleResolvers, rbacAuthorizer)
default:
return nil, nil, fmt.Errorf("unknown authorization mode %s specified", authorizationMode)
}
}
return union.New(authorizers...), union.NewRuleResolvers(ruleResolvers...), nil
}
const (
// ModeAlwaysAllow is the mode to set all requests as authorized
ModeAlwaysAllow string = "AlwaysAllow"
// ModeAlwaysDeny is the mode to set no requests as authorized
ModeAlwaysDeny string = "AlwaysDeny"
// ModeABAC is the mode to use Attribute Based Access Control to authorize
ModeABAC string = "ABAC"
// ModeWebhook is the mode to make an external webhook call to authorize
ModeWebhook string = "Webhook"
// ModeRBAC is the mode to use Role Based Access Control to authorize
ModeRBAC string = "RBAC"
// ModeNode is an authorization mode that authorizes API requests made by kubelets.
ModeNode string = "Node"
)
Admission
// 查看定義
err = s.Admission.ApplyTo()
func (a *AdmissionOptions) ApplyTo(){
return a.GenericAdmission.ApplyTo()
}
func (ps *Plugins) NewFromPlugins(){
for _, pluginName := range pluginNames {
// InitPlugin 為初始化的工作
plugin, err := ps.InitPlugin(pluginName, pluginConfig, pluginInitializer)
if err != nil {
return nil, err
}
}
}
func (ps *Plugins) InitPlugin(name string, config io.Reader, pluginInitializer PluginInitializer) (Interface, error){
// 獲取plugin
plugin, found, err := ps.getPlugin(name, config)
}
// 查看一下Interface的定義,就是對(duì)準(zhǔn)入機(jī)制的控制 抽象化的插件化的接口 服務(wù)于Admission Control
// Interface is an abstract, pluggable interface for Admission Control decisions.
type Interface interface {
Handles(operation Operation) bool
}
// 再去看看獲取plugin的地方
func (ps *Plugins) getPlugin(name string, config io.Reader) (Interface, bool, error) {
ps.lock.Lock()
defer ps.lock.Unlock()
// 我們?cè)偃パ芯縫s.registry這個(gè)參數(shù)是在哪里被初始化的
f, found := ps.registry[name]
}
// 接下來,我們從kube-apiserver啟動(dòng)過程,逐步找到Admission被初始化的地方
// 啟動(dòng)命令
command := app.NewAPIServerCommand()
// server配置
s := options.NewServerRunOptions()
// admission選項(xiàng)
Admission: kubeoptions.NewAdmissionOptions()
// 注冊(cè)準(zhǔn)入機(jī)制
RegisterAllAdmissionPlugins(options.Plugins)
// 準(zhǔn)入機(jī)制的所有內(nèi)容
func RegisterAllAdmissionPlugins(plugins *admission.Plugins){
// 這里有很多plugin的注冊(cè)
}
// 往上翻,我們能找到所有plugin,也就是準(zhǔn)入機(jī)制的定義 有三十幾種 已經(jīng)進(jìn)行了排序的
var AllOrderedPlugins = []string{
admit.PluginName, // AlwaysAdmit
autoprovision.PluginName, // NamespaceAutoProvision
lifecycle.PluginName, // NamespaceLifecycle
exists.PluginName, // NamespaceExists
scdeny.PluginName, // SecurityContextDeny
antiaffinity.PluginName, // LimitPodHardAntiAffinityTopology
limitranger.PluginName, // LimitRanger
serviceaccount.PluginName, // ServiceAccount
noderestriction.PluginName, // NodeRestriction
nodetaint.PluginName, // TaintNodesByCondition
alwayspullimages.PluginName, // AlwaysPullImages
imagepolicy.PluginName, // ImagePolicyWebhook
podsecurity.PluginName, // PodSecurity
podnodeselector.PluginName, // PodNodeSelector
podpriority.PluginName, // Priority
defaulttolerationseconds.PluginName, // DefaultTolerationSeconds
podtolerationrestriction.PluginName, // PodTolerationRestriction
eventratelimit.PluginName, // EventRateLimit
extendedresourcetoleration.PluginName, // ExtendedResourceToleration
label.PluginName, // PersistentVolumeLabel
setdefault.PluginName, // DefaultStorageClass
storageobjectinuseprotection.PluginName, // StorageObjectInUseProtection
gc.PluginName, // OwnerReferencesPermissionEnforcement
resize.PluginName, // PersistentVolumeClaimResize
runtimeclass.PluginName, // RuntimeClass
certapproval.PluginName, // CertificateApproval
certsigning.PluginName, // CertificateSigning
certsubjectrestriction.PluginName, // CertificateSubjectRestriction
defaultingressclass.PluginName, // DefaultIngressClass
denyserviceexternalips.PluginName, // DenyServiceExternalIPs
// new admission plugins should generally be inserted above here
// webhook, resourcequota, and deny plugins must go at the end
mutatingwebhook.PluginName, // MutatingAdmissionWebhook
validatingwebhook.PluginName, // ValidatingAdmissionWebhook
resourcequota.PluginName, // ResourceQuota
deny.PluginName, // AlwaysDeny
}
GenericAPIServer的初始化
理解kube-apiserver是中的管理核心資源的KubeAPIServer是怎么啟動(dòng)的
New
// 先對(duì)配置進(jìn)行complete補(bǔ)全再進(jìn)行new
func CreateKubeAPIServer(kubeAPIServerConfig *controlplane.Config, delegateAPIServer genericapiserver.DelegationTarget) (*controlplane.Instance, error) {
kubeAPIServer, err := kubeAPIServerConfig.Complete().New(delegateAPIServer)
if err != nil {
return nil, err
}
return kubeAPIServer, nil
}
GenericServer
// 在APIExtensionsServer、KubeAPIServer和AggregatorServer三種Server啟動(dòng)時(shí),我們都能發(fā)現(xiàn)這么一個(gè)函數(shù)
// APIExtensionsServer
genericServer, err := c.GenericConfig.New("apiextensions-apiserver", delegationTarget)
// KubeAPIServer
s, err := c.GenericConfig.New("kube-apiserver", delegationTarget)
// AggregatorServer
genericServer, err := c.GenericConfig.New("kube-aggregator", delegationTarget)
// 都通過GenericConfig創(chuàng)建了genericServer,我們先大致瀏覽下
func (c completedConfig) New(name string, delegationTarget DelegationTarget) (*GenericAPIServer, error) {
// 新建Handler
apiServerHandler := NewAPIServerHandler(name, c.Serializer, handlerChainBuilder, delegationTarget.UnprotectedHandler())
// 實(shí)例化一個(gè)Server
s := &GenericAPIServer{
...
}
// 處理鉤子hook操作
for k, v := range delegationTarget.PostStartHooks() {
s.postStartHooks[k] = v
}
for k, v := range delegationTarget.PreShutdownHooks() {
s.preShutdownHooks[k] = v
}
// 健康監(jiān)測(cè)
for _, delegateCheck := range delegationTarget.HealthzChecks() {
skip := false
for _, existingCheck := range c.HealthzChecks {
if existingCheck.Name() == delegateCheck.Name() {
skip = true
break
}
}
if skip {
continue
}
s.AddHealthChecks(delegateCheck)
}
// 安裝API相關(guān)參數(shù),這個(gè)是重點(diǎn)
installAPI(s, c.Config)
return s, nil
}
NewAPIServerHandler
func NewAPIServerHandler(name string, s runtime.NegotiatedSerializer, handlerChainBuilder HandlerChainBuilderFn, notFoundHandler http.Handler) *APIServerHandler {
// 采用了 github.com/emicklei/go-restful 這個(gè)庫(kù)作為 RESTful 接口的設(shè)計(jì),目前了解即可
gorestfulContainer := restful.NewContainer()
}
installAPI
一些通用的
func installAPI(s *GenericAPIServer, c *Config) {
// 添加 /index.html 路由規(guī)則
if c.EnableIndex {
routes.Index{}.Install(s.listedPathProvider, s.Handler.NonGoRestfulMux)
}
// 添加go語言 /pprof 的路由規(guī)則,常用于性能分析
if c.EnableProfiling {
routes.Profiling{}.Install(s.Handler.NonGoRestfulMux)
if c.EnableContentionProfiling {
goruntime.SetBlockProfileRate(1)
}
routes.DebugFlags{}.Install(s.Handler.NonGoRestfulMux, "v", routes.StringFlagPutHandler(logs.GlogSetter))
}
// 添加監(jiān)控相關(guān)的 /metrics 的指標(biāo)路由規(guī)則
if c.EnableMetrics {
if c.EnableProfiling {
routes.MetricsWithReset{}.Install(s.Handler.NonGoRestfulMux)
} else {
routes.DefaultMetrics{}.Install(s.Handler.NonGoRestfulMux)
}
}
// 添加版本 /version 的路由規(guī)則
routes.Version{Version: c.Version}.Install(s.Handler.GoRestfulContainer)
// 開啟服務(wù)發(fā)現(xiàn)
if c.EnableDiscovery {
s.Handler.GoRestfulContainer.Add(s.DiscoveryGroupManager.WebService())
}
if feature.DefaultFeatureGate.Enabled(features.APIPriorityAndFairness) {
c.FlowControl.Install(s.Handler.NonGoRestfulMux)
}
}
Apiserver
func (c completedConfig) New(delegationTarget genericapiserver.DelegationTarget) (*Master, error) {
// genericServer的初始化
s, err := c.GenericConfig.New("kube-apiserver", delegationTarget)
// 核心KubeAPIServer的實(shí)例化
m := &Master{
GenericAPIServer: s,
ClusterAuthenticationInfo: c.ExtraConfig.ClusterAuthenticationInfo,
}
// 注冊(cè)Legacy API的注冊(cè)
if c.ExtraConfig.APIResourceConfigSource.VersionEnabled(apiv1.SchemeGroupVersion) {
legacyRESTStorageProvider := corerest.LegacyRESTStorageProvider{}
if err := m.InstallLegacyAPI(&c, c.GenericConfig.RESTOptionsGetter, legacyRESTStorageProvider); err != nil {
return nil, err
}
}
// REST接口的存儲(chǔ)定義,可以看到很多k8s上的常見定義,比如node節(jié)點(diǎn)/storage存儲(chǔ)/event事件等等
restStorageProviders := []RESTStorageProvider{
authenticationrest.RESTStorageProvider{Authenticator: c.GenericConfig.Authentication.Authenticator, APIAudiences: c.GenericConfig.Authentication.APIAudiences},
authorizationrest.RESTStorageProvider{Authorizer: c.GenericConfig.Authorization.Authorizer, RuleResolver: c.GenericConfig.RuleResolver},
autoscalingrest.RESTStorageProvider{},
batchrest.RESTStorageProvider{},
certificatesrest.RESTStorageProvider{},
coordinationrest.RESTStorageProvider{},
discoveryrest.StorageProvider{},
extensionsrest.RESTStorageProvider{},
networkingrest.RESTStorageProvider{},
noderest.RESTStorageProvider{},
policyrest.RESTStorageProvider{},
rbacrest.RESTStorageProvider{Authorizer: c.GenericConfig.Authorization.Authorizer},
schedulingrest.RESTStorageProvider{},
settingsrest.RESTStorageProvider{},
storagerest.RESTStorageProvider{},
flowcontrolrest.RESTStorageProvider{},
// keep apps after extensions so legacy clients resolve the extensions versions of shared resource names.
// See https://github.com/kubernetes/kubernetes/issues/42392
appsrest.StorageProvider{},
admissionregistrationrest.RESTStorageProvider{},
eventsrest.RESTStorageProvider{TTL: c.ExtraConfig.EventTTL},
}
// 注冊(cè)API
if err := m.InstallAPIs(c.ExtraConfig.APIResourceConfigSource, c.GenericConfig.RESTOptionsGetter, restStorageProviders...); err != nil {
return nil, err
}
// 添加Hook
m.GenericAPIServer.AddPostStartHookOrDie("start-cluster-authentication-info-controller", func(hookContext genericapiserver.PostStartHookContext) error {
})
return m, nil
}
注冊(cè)API的關(guān)鍵在InstallLegacyAPI和InstallAPIs,如果你對(duì)kubernetes的資源有一定的了解,會(huì)知道核心資源都放在Legacy中如pod(如果不了解的話,點(diǎn)擊函數(shù)看一下,就能有所有了解)
InstallLegacyAPI
// 定義了legacy和非legacy資源的路由前綴
const (
// DefaultLegacyAPIPrefix is where the legacy APIs will be located.
DefaultLegacyAPIPrefix="/api"
// APTGroupPrefix is where non-legacy API group will be located.
APIGroupPrefix ="/apis"
)
func (m *Master) InstallLegacyAPI(c *completedConfig, restOptionsGetter generic.RESTOptionsGetter, legacyRESTStorageProvider corerest.LegacyRESTStorageProvider) error {
// RESTStorage的初始化
legacyRESTStorage, apiGroupInfo, err := legacyRESTStorageProvider.NewLegacyRESTStorage(restOptionsGetter)
// 前綴為 /api,注冊(cè)上對(duì)應(yīng)的Version和Resource
// Pod作為核心資源,沒有Group的概念
if err := m.GenericAPIServer.InstallLegacyAPIGroup(genericapiserver.DefaultLegacyAPIPrefix, &apiGroupInfo); err != nil {
return fmt.Errorf("error in registering group versions: %v", err)
}
return nil
}
// 我們?cè)偌?xì)看這個(gè)RESTStorage的初始化
func (c LegacyRESTStorageProvider) NewLegacyRESTStorage(restOptionsGetter generic.RESTOptionsGetter) (LegacyRESTStorage, genericapiserver.APIGroupInfo, error) {
// pod 模板
podTemplateStorage, err := podtemplatestore.NewREST(restOptionsGetter)
// event事件
eventStorage, err := eventstore.NewREST(restOptionsGetter, uint64(c.EventTTL.Seconds()))
// limitRange資源限制
limitRangeStorage, err := limitrangestore.NewREST(restOptionsGetter)
// resourceQuota資源配額
resourceQuotaStorage, resourceQuotaStatusStorage, err := resourcequotastore.NewREST(restOptionsGetter)
// secret加密
secretStorage, err := secretstore.NewREST(restOptionsGetter)
// PV 存儲(chǔ)
persistentVolumeStorage, persistentVolumeStatusStorage, err := pvstore.NewREST(restOptionsGetter)
// PVC 存儲(chǔ)
persistentVolumeClaimStorage, persistentVolumeClaimStatusStorage, err := pvcstore.NewREST(restOptionsGetter)
// ConfigMap 配置
configMapStorage, err := configmapstore.NewREST(restOptionsGetter)
// 等等核心資源,暫不一一列舉
// pod模板,我們的示例nginx-pod屬于這個(gè)類型的資源
podStorage, err := podstore.NewStorage()
// 保存storage的對(duì)應(yīng)關(guān)系
restStorageMap := map[string]rest.Storage{
"pods": podStorage.Pod,
"pods/attach": podStorage.Attach,
"pods/status": podStorage.Status,
"pods/log": podStorage.Log,
"pods/exec": podStorage.Exec,
"pods/portforward": podStorage.PortForward,
"pods/proxy": podStorage.Proxy,
"pods/binding": podStorage.Binding,
"bindings": podStorage.LegacyBinding,
...
}
}
Create Pod
// 查看Pod初始化 上一步的podStorage
func NewStorage(optsGetter generic.RESTOptionsGetter, k client.ConnectionInfoGetter, proxyTransport http.RoundTripper, podDisruptionBudgetClient policyclient.PodDisruptionBudgetsGetter) (PodStorage, error) {
store := &genericregistry.Store{
NewFunc: func() runtime.Object { return &api.Pod{} },
NewListFunc: func() runtime.Object { return &api.PodList{} },
PredicateFunc: registrypod.MatchPod,
DefaultQualifiedResource: api.Resource("pods"),
// 增改刪的策略
CreateStrategy: registrypod.Strategy,
UpdateStrategy: registrypod.Strategy,
DeleteStrategy: registrypod.Strategy,
ReturnDeletedObject: true,
TableConvertor: printerstorage.TableConvertor{TableGenerator: printers.NewTableGenerator().With(printersinternal.AddHandlers)},
}
}
// 查看 Strategy 的初始化 是一個(gè)全局變量 進(jìn)行實(shí)例化 調(diào)用了Scheme,核心資源的schme,legacyscheme
var Strategy = podStrategy{legacyscheme.Scheme, names.SimpleNameGenerator}
// 又查詢到Scheme的初始化。Schema可以理解為Kubernetes的注冊(cè)表,即所有的資源類型必須先注冊(cè)進(jìn)Schema才可使用 注冊(cè)里有資源的增刪改的策略
var Scheme = runtime.NewScheme()
Pod數(shù)據(jù)的保存
理解Pod發(fā)送到kube-apiserver后是怎么保存的
RESTCreateStrategy
// podStrategy 是封裝了 Pod 的各類動(dòng)作,這里我們先關(guān)注create這個(gè)操作
type podStrategy struct {
runtime.ObjectTyper
names.NameGenerator
}
// podStrategy 的接口
type RESTCreateStrategy interface {
runtime.ObjectTyper
names.NameGenerator
// 是否屬于當(dāng)前的 namespace
NamespaceScoped() bool
// 準(zhǔn)備創(chuàng)建前的檢查
PrepareForCreate(ctx context.Context, obj runtime.Object)
// 驗(yàn)證資源對(duì)象
Validate(ctx context.Context, obj runtime.Object) field.ErrorList
// 規(guī)范化
Canonicalize(obj runtime.Object)
}
// 完成了檢查,我們就要保存數(shù)據(jù)了
Storage
// PodStorage 是 Pod 存儲(chǔ)的實(shí)現(xiàn),里面包含了多個(gè)存儲(chǔ)的定義
type PodStorage struct {
// REST implements a RESTStorage for pods
Pod *REST
// BindingREST implements the REST endpoint for binding pods to nodes when etcd is in use.
Binding *BindingREST
// LegacyBindingREST implements the REST endpoint for binding pods to nodes when etcd is in use.
LegacyBinding *LegacyBindingREST
Eviction *EvictionREST
// StatusREST implements the REST endpoint for changing the status of a pod.
Status *StatusREST
// EphemeralContainersREST implements the REST endpoint for adding EphemeralContainers
EphemeralContainers *EphemeralContainersREST
Log *podrest.LogREST
Proxy *podrest.ProxyREST
Exec *podrest.ExecREST
Attach *podrest.AttachREST
PortForward *podrest.PortForwardREST
}
/*
從上一節(jié)的map關(guān)系中,保存在REST中
restStorageMap := map[string]rest.Storage{
"pods": podStorage.Pod,
}
*/
type REST struct {
*genericregistry.Store
// 代理傳輸層 大概率是和網(wǎng)絡(luò)相關(guān)的先不看
proxyTransport http.RoundTripper
}
// Store是一個(gè)通用的數(shù)據(jù)結(jié)構(gòu)
type Store struct {
// Storage定義
...
Storage DryRunnableStorage
}
// DryRunnableStorage中的Storage是一個(gè)Interface
type DryRunnableStorage struct {
Storage storage.Interface
// 和編解碼相關(guān)的codec
Codec runtime.Codec
}
func (s *DryRunnableStorage) Create(ctx context.Context, key string, obj, out runtime.Object, ttl uint64, dryRun bool) error {
if dryRun {
if err := s.Storage.Get(ctx, key, storage.GetOptions{}, out); err == nil {
return storage.NewKeyExistsError(key, 0)
}
return s.copyInto(obj, out)
}
// 這里,就是Create的真正調(diào)用
return s.Storage.Create(ctx, key, obj, out, ttl)
}
Storage Implement
// Storage Interface 的定義,包括基本的增刪改查,以及watch等等進(jìn)階操作
type Interface interface {
Versioner() Versioner
Create(ctx context.Context, key string, obj, out runtime.Object, ttl uint64) error
Delete(ctx context.Context, key string, out runtime.Object, preconditions *Preconditions, validateDeletion ValidateObjectFunc) error
Watch(ctx context.Context, key string, opts ListOptions) (watch.Interface, error)
WatchList(ctx context.Context, key string, opts ListOptions) (watch.Interface, error)
Get(ctx context.Context, key string, opts GetOptions, objPtr runtime.Object) error
GetToList(ctx context.Context, key string, opts ListOptions, listObj runtime.Object) error
List(ctx context.Context, key string, opts ListOptions, listObj runtime.Object) error
GuaranteedUpdate(
ctx context.Context, key string, ptrToType runtime.Object, ignoreNotFound bool,
precondtions *Preconditions, tryUpdate UpdateFunc, suggestion ...runtime.Object) error
Count(key string) (int64, error)
}
// 去找Storage的初始化
func NewRawStorage(config *storagebackend.Config) (storage.Interface, factory.DestroyFunc, error) {
return factory.Create(*config)
}
func Create(c storagebackend.Config) (storage.Interface, DestroyFunc, error) {
switch c.Type {
// 已經(jīng)不支持etcd2
case "etcd2":
return nil, nil, fmt.Errorf("%v is no longer a supported storage backend", c.Type)
// 默認(rèn)為etcd3版本
case storagebackend.StorageTypeUnset, storagebackend.StorageTypeETCD3:
return newETCD3Storage(c)
default:
return nil, nil, fmt.Errorf("unknown storage type: %s", c.Type)
}
}
Summary
kube-apiserver包含三個(gè)apiserverAPIExtensionsServer、KubeAPIServer和AggregatorServer三個(gè)APIServer底層均依賴通用的GenericServer,使用go-restful對(duì)外提供RESTful風(fēng)格的API服務(wù),三個(gè)server,都有兩類配置一類是專有的一個(gè)通用的genericServer,通用的配置中有三種Authentication/Authorization/Admission,控制權(quán)限的方式,kube-apiserver對(duì)請(qǐng)求進(jìn)行Authentication、Authorization和Admission三層驗(yàn)證,Admission是插件化的,可以通過webhook來拓展- 完成驗(yàn)證后,請(qǐng)求會(huì)根據(jù)路由規(guī)則,觸發(fā)到對(duì)應(yīng)資源的handler,主要包括數(shù)據(jù)的
預(yù)處理和保存,pod的底層是podStorage的對(duì)象,使用到注冊(cè)表schme kube-apiserver的底層存儲(chǔ)為etcd v3,它被抽象為一種RESTStorage,使網(wǎng)絡(luò)請(qǐng)求和底層存儲(chǔ)操作一一對(duì)應(yīng)
以上就是Kubernetes ApiServer三大server權(quán)限與數(shù)據(jù)存儲(chǔ)解析的詳細(xì)內(nèi)容,更多關(guān)于Kubernetes ApiServer權(quán)限存儲(chǔ)的資料請(qǐng)關(guān)注腳本之家其它相關(guān)文章!
- 一文詳解基于Kubescape進(jìn)行Kubernetes安全加固
- kubernetes數(shù)據(jù)持久化PV?PVC深入分析詳解
- kubernetes數(shù)據(jù)持久化StorageClass動(dòng)態(tài)供給實(shí)現(xiàn)詳解
- Kubernetes?controller?manager運(yùn)行機(jī)制源碼解析
- Kubernetes Informer數(shù)據(jù)存儲(chǔ)Index與Pod分配流程解析
- Kubernetes scheduler啟動(dòng)監(jiān)控資源變化解析
- IoT邊緣集群Kubernetes?Events告警通知實(shí)現(xiàn)示例
相關(guān)文章
Kubernetes(K8S)容器集群管理環(huán)境完整部署詳細(xì)教程-下篇
本系列文章主要介紹了Kubernetes(K8S)容器集群管理環(huán)境完整部署的詳細(xì)教程,分為上中下三篇文章,此為中篇,主要講解了K8S部署metrics-server插件,K8S部署集群dashboard插件,K8S部署集群coredns插件,K8S部署kube-state-metrics插件,K8S部署harbor私有倉(cāng)庫(kù)2022-01-01
Kubernetes(K8S)入門基礎(chǔ)內(nèi)容介紹
這篇文章介紹了Kubernetes(K8S)的入門基礎(chǔ)內(nèi)容,對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2022-03-03
Kubernetes ApiServer三大server權(quán)限與數(shù)據(jù)存儲(chǔ)解析
這篇文章主要為大家介紹了Kubernetes ApiServer三大server權(quán)限與數(shù)據(jù)存儲(chǔ)解析,有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪2022-11-11
k8s集群部署時(shí)etcd容器不停重啟問題以及處理詳解
一次在k8s集群中創(chuàng)建實(shí)例發(fā)現(xiàn)etcd集群狀態(tài)出現(xiàn)連接失敗狀況,導(dǎo)致創(chuàng)建實(shí)例失敗,下面這篇文章主要給大家介紹了關(guān)于k8s集群部署時(shí)etcd容器不停重啟問題以及處理的相關(guān)資料,需要的朋友可以參考下2023-01-01
Docker與K8s關(guān)系介紹不會(huì)Docker也可以使用K8s
想學(xué)K8s,必須得先學(xué)會(huì)Docker嗎?這是很多網(wǎng)友在開始有想法想要學(xué)?K8s的時(shí)候都會(huì)冒出來的想法,要回答這個(gè)問題,我們需要先搞清楚?Docker?和?K8s?他們的角色是什么,相互之間是什么關(guān)系2022-06-06
Kubernetes(K8S)容器集群管理環(huán)境完整部署詳細(xì)教程-中篇
本系列文章主要介紹了Kubernetes(K8S)容器集群管理環(huán)境完整部署的詳細(xì)教程,分為上中下三篇文章,此為中篇,需要的朋友可以參考下2022-01-01
kubectl中g(shù)et命令及使用示例總結(jié)
這篇文章主要為大家介紹了kubectl中g(shù)et命令及使用示例的總結(jié),有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪2022-03-03

