Operator3-设计一个operator

Operator3-设计一个operator

背景：

创建一个自己的operator

goland 创建项目

命名规则

关于应用命名一下吧：就拿月份来作应用名称吧：应用1：Jan 应用2：feb 应用3：mar 应用4：apr 应用5：may 也没有想好具体的代表什么，下面了边作边看......

kubebuilder init

[zhangpeng@zhangpeng develop-operator]$ kubebuilder init --plugins go/v3 --domain zhangpeng.com --owner "zhang peng"

开启支持多接口组

[zhangpeng@zhangpeng develop-operator]$ kubebuilder edit --multigroup=true

[zhangpeng@zhangpeng develop-operator]$ kubebuilder create api --group jan --version v1 --kind Jan

[zhangpeng@zhangpeng develop-operator]$ kubebuilder create api --group feb --version v1 --kind Feb

[zhangpeng@zhangpeng develop-operator]$ kubebuilder create api --group mar --version v1 --kind Mar [zhangpeng@zhangpeng develop-operator]$ kubebuilder create api --group apr --version v1 --kind Apr [zhangpeng@zhangpeng develop-operator]$ kubebuilder create api --group may --version v1 --kind May

从Jan开始

jan应用为一个deployment应用，参照与deployment的区别！注意：以下代码都是抄写自阳明大佬，些许修改......，比如有个& 还有关于service的修改

定义jan_type

apis/jan/v1/jan_type.go

type JanSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file Size *int32 `json:"size"` Image string `json:"image"` Resources corev1.ResourceRequirements `json:"resources,omitempty"` Envs []corev1.EnvVar `json:"envs,omitempty"` Ports []corev1.ServicePort `json:"ports,omitempty"` } // JanStatus defines the observed state of Jan type JanStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file appsv1.DeploymentStatus `json:",inline"` }

make install

make install 失败，继续拆解命令：

[zhangpeng@zhangpeng develop-operator]$ ./bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

cat config/crd/bases/jan.zhangpeng.com_jans.yaml

--- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.8.0 creationTimestamp: null name: jans.jan.zhangpeng.com spec: group: jan.zhangpeng.com names: kind: Jan listKind: JanList plural: jans singular: jan scope: Namespaced versions: - name: v1 schema: openAPIV3Schema: description: Jan is the Schema for the jans API properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: JanSpec defines the desired state of Jan properties: envs: items: description: EnvVar represents an environment variable present in a Container. properties: name: description: Name of the environment variable. Must be a C_IDENTIFIER. type: string value: description: 'Variable references $(VAR_NAME) are expanded using the previously defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".' type: string valueFrom: description: Source for the environment variable's value. Cannot be used if value is not empty. properties: configMapKeyRef: description: Selects a key of a ConfigMap. properties: key: description: The key to select. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the ConfigMap or its key must be defined type: boolean required: - key type: object fieldRef: description: 'Selects a field of the pod: supports metadata.name, metadata.namespace, `metadata.labels['''']`, `metadata.annotations['''']`, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.' properties: apiVersion: description: Version of the schema the FieldPath is written in terms of, defaults to "v1". type: string fieldPath: description: Path of the field to select in the specified API version. type: string required: - fieldPath type: object resourceFieldRef: description: 'Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.' properties: containerName: description: 'Container name: required for volumes, optional for env vars' type: string divisor: anyOf: - type: integer - type: string description: Specifies the output format of the exposed resources, defaults to "1" pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true resource: description: 'Required: resource to select' type: string required: - resource type: object secretKeyRef: description: Selects a key of a secret in the pod's namespace properties: key: description: The key of the secret to select from. Must be a valid secret key. type: string name: description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?' type: string optional: description: Specify whether the Secret or its key must be defined type: boolean required: - key type: object type: object required: - name type: object type: array image: type: string ports: items: description: ServicePort contains information on service's port. properties: appProtocol: description: The application protocol for this port. This field follows standard Kubernetes label syntax. Un-prefixed names are reserved for IANA standard service names (as per RFC-6335 and Non-standard protocols should use prefixed names such as mycompany.com/my-custom-protocol. type: string name: description: The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service. type: string nodePort: description: 'The port on each node on which this service is exposed when type is NodePort or LoadBalancer. Usually assigned by the system. If a value is specified, in-range, and not in use it will be used, otherwise the operation will fail. If not specified, a port will be allocated if this Service requires one. If this field is specified when creating a Service which does not need it, creation will fail. This field will be wiped when updating a Service to no longer need it (e.g. changing type from NodePort to ClusterIP). More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport' format: int32 type: integer port: description: The port that will be exposed by this service. format: int32 type: integer protocol: default: TCP description: The IP protocol for this port. Supports "TCP", "UDP", and "SCTP". Default is TCP. type: string targetPort: anyOf: - type: integer - type: string description: 'Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod''s container ports. If this is not specified, the value of the ''port'' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the ''port'' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service' x-kubernetes-int-or-string: true required: - port type: object type: array resources: description: ResourceRequirements describes the compute resource requirements. properties: limits: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object size: format: int32 maximum: 5 minimum: 1 type: integer type: object status: description: JanStatus defines the observed state of Jan properties: availableReplicas: description: Total number of available pods (ready for at least minReadySeconds) targeted by this deployment. format: int32 type: integer collisionCount: description: Count of hash collisions for the Deployment. The Deployment controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ReplicaSet. format: int32 type: integer conditions: description: Represents the latest available observations of a deployment's current state. items: description: DeploymentCondition describes the state of a deployment at a certain point. properties: lastTransitionTime: description: Last time the condition transitioned from one status to another. format: date-time type: string lastUpdateTime: description: The last time this condition was updated. format: date-time type: string message: description: A human readable message indicating details about the transition. type: string reason: description: The reason for the condition's last transition. type: string status: description: Status of the condition, one of True, False, Unknown. type: string type: description: Type of deployment condition. type: string required: - status - type type: object type: array observedGeneration: description: The generation observed by the deployment controller. format: int64 type: integer readyReplicas: description: readyReplicas is the number of pods targeted by this Deployment with a Ready Condition. format: int32 type: integer replicas: description: Total number of non-terminated pods targeted by this deployment (their labels match the selector). format: int32 type: integer unavailableReplicas: description: Total number of unavailable pods targeted by this deployment. This is the total number of pods that are still required for the deployment to have 100% available capacity. They may either be pods that are running but not yet available or pods that still have not been created. format: int32 type: integer updatedReplicas: description: Total number of non-terminated pods targeted by this deployment that have the desired template spec. format: int32 type: integer type: object type: object served: true storage: true subresources: status: {} status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: []

继续发布到集群：

[zhangpeng@zhangpeng develop-operator]$ kustomize build config/crd | kubectl apply -f -

[zhangpeng@zhangpeng develop-operator]$ kubectl describe crd jans.jan.zhangpeng.com

创建deployment pod service的方法

package jan import ( janv1 "develop-operator/apis/jan/v1" appv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" ) func NewJan(app *janv1.Jan) *appv1.Deployment { labels := map[string]string{"app": app.Name} selector := &metav1.LabelSelector{MatchLabels: labels} return &appv1.Deployment{ TypeMeta: metav1.TypeMeta{ Kind: "apps/v1", APIVersion: "Deployment", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: appv1.DeploymentSpec{ Replicas: app.Spec.Size, Selector: selector, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{Labels: labels}, Spec: corev1.PodSpec{Containers: newContainers(app)}, }, MinReadySeconds: 0, }, } } func newContainers(app *janv1.Jan) []corev1.Container { containerPorts := []corev1.ContainerPort{} for _, svcPort := range app.Spec.Ports { cport := corev1.ContainerPort{} cport.ContainerPort = svcPort.TargetPort.IntVal containerPorts = append(containerPorts, cport) } return []corev1.Container{ { Name: app.Name, Image: app.Spec.Image, Resources: app.Spec.Resources, Ports: containerPorts, ImagePullPolicy: corev1.PullIfNotPresent, Env: app.Spec.Envs, }, } } func NewService(app *janv1.Jan) *corev1.Service { return &corev1.Service{ TypeMeta: metav1.TypeMeta{ Kind: "Service", APIVersion: "v1", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: corev1.ServiceSpec{ Type: corev1.ServiceTypeNodePort, Ports: app.Spec.Ports, Selector: map[string]string{ "app": app.Name, }, }, } }

jan_controller.go Reconcile

基本阳明大佬的博客抄来的,Reconcile调谐函数：

/* Copyright 2022 zhang peng. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package jan import ( "context" "encoding/json" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" "reflect" "sigs.k8s.io/controller-runtime/pkg/reconcile" "k8s.io/apimachinery/pkg/runtime" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log" janv1 "develop-operator/apis/jan/v1" ) // JanReconciler reconciles a Jan object type JanReconciler struct { client.Client Scheme *runtime.Scheme } //+kubebuilder:rbac:groups=jan.zhangpeng.com,resources=jans,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=jan.zhangpeng.com,resources=jans/status,verbs=get;update;patch //+kubebuilder:rbac:groups=jan.zhangpeng.com,resources=jans/finalizers,verbs=update // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. // TODO(user): Modify the Reconcile function to compare the state specified by // the Jan object against the actual cluster state, and then // perform operations to make the cluster state reflect the state specified by // the user. // // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile func (r *JanReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { _ = log.FromContext(ctx) instance := &janv1.Jan{} err := r.Client.Get(context.TODO(), req.NamespacedName, instance) if err != nil { if errors.IsNotFound(err) { // Request object not found, could have been deleted after reconcile request. // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. // Return and don't requeue return reconcile.Result{}, nil } // Error reading the object - requeue the request. return reconcile.Result{}, err } if instance.DeletionTimestamp != nil { return reconcile.Result{}, err } // 如果不存在，则创建关联资源 // 如果存在，判断是否需要更新 // 如果需要更新，则直接更新 // 如果不需要更新，则正常返回 deploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, deploy); err != nil && errors.IsNotFound(err) { // 创建关联资源 // 1. 创建 Deploy deploy := NewJan(instance) if err := r.Client.Create(context.TODO(), deploy); err != nil { return reconcile.Result{}, err } // 2. 创建 Service service := NewService(instance) if err := r.Client.Create(context.TODO(), service); err != nil { return reconcile.Result{}, err } // 3. 关联 Annotations data, _ := json.Marshal(instance.Spec) if instance.Annotations != nil { instance.Annotations["spec"] = string(data) } else { instance.Annotations = map[string]string{"spec": string(data)} } if err := r.Client.Update(context.TODO(), instance); err != nil { return reconcile.Result{}, nil } return reconcile.Result{}, nil } oldspec := janv1.JanSpec{} if err := json.Unmarshal([]byte(instance.Annotations["spec"]), &oldspec); err != nil { return reconcile.Result{}, err } if !reflect.DeepEqual(instance.Spec, oldspec) { // 更新关联资源 newDeploy := NewJan(instance) oldDeploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldDeploy); err != nil { return reconcile.Result{}, err } oldDeploy.Spec = newDeploy.Spec if err := r.Client.Update(context.TODO(), oldDeploy); err != nil { return reconcile.Result{}, err } newService := NewService(instance) oldService := &corev1.Service{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldService); err != nil { return reconcile.Result{}, err } oldService.Spec = newService.Spec if err := r.Client.Update(context.TODO(), oldService); err != nil { return reconcile.Result{}, err } return reconcile.Result{}, nil } return reconcile.Result{}, nil } // SetupWithManager sets up the controller with the Manager. func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&janv1.Jan{}). Complete(r) }

强调一下：

make run and test

apiVersion: jan.zhangpeng.com/v1 kind: Jan metadata: name: jan-sample spec: size: 2 image: nginx:1.7.9 ports: - port: 80 targetPort: 80 nodePort: 30002

[zhangpeng@zhangpeng develop-operator]$ kubectl apply -f config/samples/jan_v1_jan.yaml jan.jan.zhangpeng.com/jan-sample created

继续改造

修改Service Type

[zhangpeng@zhangpeng develop-operator]$ make install [zhangpeng@zhangpeng develop-operator]$ ./bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases [zhangpeng@zhangpeng develop-operator]$ kustomize build config/crd | kubectl apply -f -

apiVersion: jan.zhangpeng.com/v1 kind: Jan metadata: name: jan-sample spec: size: 3 image: nginx:1.7.9 ports: - port: 80 targetPort: 80 // nodePort: 30002 type: ClusterIP

[zhangpeng@zhangpeng develop-operator]$ kubectl delete -f config/samples/jan_v1_jan.yaml jan.jan.zhangpeng.com "jan-sample" deleted [zhangpeng@zhangpeng develop-operator]$ kubectl apply -f config/samples/jan_v1_jan.yaml jan.jan.zhangpeng.com/jan-sample created [zhangpeng@zhangpeng develop-operator]$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE jan-sample ClusterIP 10.99.27.123 80/TCP 1s kubernetes ClusterIP 10.96.0.1 443/TCP 19d [zhangpeng@zhangpeng develop-operator]$ kubectl get jan NAME AGE jan-sample 5m21s

继续模仿

kubebuilder create api --group common --version v1 --kind Common

/* Copyright 2022 zhang peng. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package v1 import ( corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. // JanSpec defines the desired state of Jan type JanSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file //+kubebuilder:default:=1 //+kubebuilder:validation:Minimum:=1 Replicas *int32 `json:"replicas,omitempty" protobuf:"varint,1,opt,name=replicas"` Image string `json:"image"` Resources corev1.ResourceRequirements `json:"resources,omitempty"` Envs []corev1.EnvVar `json:"envs,omitempty"` Ports []corev1.ServicePort `json:"ports,omitempty"` Type corev1.ServiceType `json:"type,omitempty"` } const ( Running = "Running" Pending = "Pending" NotReady = "NotReady" Failed = "Failed" ) // JanStatus defines the observed state of Jan type JanStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file // Phase is the phase of guestbook Phase string `json:"phase,omitempty"` // replicas is the number of Pods created by the StatefulSet controller. Replicas int32 `json:"replicas"` // readyReplicas is the number of Pods created by the StatefulSet controller that have a Ready Condition. ReadyReplicas int32 `json:"readyReplicas"` // LabelSelector is label selectors for query over pods that should match the replica count used by HPA. LabelSelector string `json:"labelSelector,omitempty"` } //+kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas,selectorpath=.status.labelSelector //+kubebuilder:printcolumn:name="Phase",type="string",JSONPath=".status.phase",description="The phase of game." //+kubebuilder:printcolumn:name="DESIRED",type="integer",JSONPath=".spec.replicas",description="The desired number of pods." //+kubebuilder:printcolumn:name="CURRENT",type="integer",JSONPath=".status.replicas",description="The number of currently all pods." //+kubebuilder:printcolumn:name="READY",type="integer",JSONPath=".status.readyReplicas",description="The number of pods ready." //+kubebuilder:printcolumn:name="AGE",type="date",JSONPath=".metadata.creationTimestamp",description="CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC." //+kubebuilder:object:root=true //+kubebuilder:subresource:status // Jan is the Schema for the jans API type Jan struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` Spec JanSpec `json:"spec,omitempty"` Status JanStatus `json:"status,omitempty"` } //+kubebuilder:object:root=true // JanList contains a list of Jan type JanList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` Items []Jan `json:"items"` } func init() { SchemeBuilder.Register(&Jan{}, &JanList{}) }

package jan import ( janv1 "develop-operator/apis/jan/v1" appv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" ) func NewJan(app *janv1.Jan) *appv1.Deployment { labels := map[string]string{"app": app.Name} selector := &metav1.LabelSelector{MatchLabels: labels} return &appv1.Deployment{ TypeMeta: metav1.TypeMeta{ Kind: "apps/v1", APIVersion: "Deployment", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: appv1.DeploymentSpec{ Replicas: app.Spec.Replicas, Selector: selector, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{Labels: labels}, Spec: corev1.PodSpec{Containers: newContainers(app)}, }, MinReadySeconds: 0, }, Status: appv1.DeploymentStatus{}, } } func newContainers(app *janv1.Jan) []corev1.Container { containerPorts := []corev1.ContainerPort{} for _, svcPort := range app.Spec.Ports { cport := corev1.ContainerPort{} cport.ContainerPort = svcPort.TargetPort.IntVal containerPorts = append(containerPorts, cport) } return []corev1.Container{ { Name: app.Name, Image: app.Spec.Image, Resources: app.Spec.Resources, Ports: containerPorts, ImagePullPolicy: corev1.PullIfNotPresent, Env: app.Spec.Envs, }, } } func NewService(app *janv1.Jan) *corev1.Service { return &corev1.Service{ TypeMeta: metav1.TypeMeta{ Kind: "Service", APIVersion: "v1", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: corev1.ServiceSpec{ Type: app.Spec.Type, Ports: app.Spec.Ports, Selector: map[string]string{ "app": app.Name, }, }, } }

jan_controller.go

/* Copyright 2022 zhang peng. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package jan import ( "context" "encoding/json" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" "reflect" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/reconcile" janv1 "develop-operator/apis/jan/v1" ) // JanReconciler reconciles a Jan object type JanReconciler struct { client.Client Scheme *runtime.Scheme } //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/status,verbs=get;update;patch //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/finalizers,verbs=update //+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=apps,resources=deployments/status,verbs=get;update;patch //+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=networking,resources=ingresses,verbs=get;list;watch;create;update;patch;delete // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. // TODO(user): Modify the Reconcile function to compare the state specified by // the Jan object against the actual cluster state, and then // perform operations to make the cluster state reflect the state specified by // the user. // // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile func (r *JanReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { defer utilruntime.HandleCrash() _ = log.FromContext(ctx) instance := &janv1.Jan{} err := r.Client.Get(context.TODO(), req.NamespacedName, instance) if err != nil { if errors.IsNotFound(err) { // Request object not found, could have been deleted after reconcile request. // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. // Return and don't requeue return reconcile.Result{}, nil } // Error reading the object - requeue the request. return reconcile.Result{}, err } if instance.DeletionTimestamp != nil { return reconcile.Result{}, err } // 如果不存在，则创建关联资源 // 如果存在，判断是否需要更新 // 如果需要更新，则直接更新 // 如果不需要更新，则正常返回 deploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, deploy); err != nil && errors.IsNotFound(err) { // 创建关联资源 // 1. 创建 Deploy deploy := NewJan(instance) if err := r.Client.Create(context.TODO(), deploy); err != nil { return reconcile.Result{}, err } // 2. 创建 Service service := NewService(instance) if err := r.Client.Create(context.TODO(), service); err != nil { return reconcile.Result{}, err } // 3. 关联 Annotations data, _ := json.Marshal(instance.Spec) if instance.Annotations != nil { instance.Annotations["spec"] = string(data) } else { instance.Annotations = map[string]string{"spec": string(data)} } if err := r.Client.Update(context.TODO(), instance); err != nil { return reconcile.Result{}, nil } return reconcile.Result{}, nil } oldspec := janv1.JanSpec{} if err := json.Unmarshal([]byte(instance.Annotations["spec"]), &oldspec); err != nil { return reconcile.Result{}, err } if !reflect.DeepEqual(instance.Spec, oldspec) { // 更新关联资源 newDeploy := NewJan(instance) oldDeploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldDeploy); err != nil { return reconcile.Result{}, err } oldDeploy.Spec = newDeploy.Spec if err := r.Client.Update(context.TODO(), oldDeploy); err != nil { return reconcile.Result{}, err } newService := NewService(instance) oldService := &corev1.Service{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldService); err != nil { return reconcile.Result{}, err } oldService.Spec = newService.Spec if err := r.Client.Update(context.TODO(), oldService); err != nil { return reconcile.Result{}, err } return reconcile.Result{}, nil } newStatus := janv1.JanStatus{ Replicas: *instance.Spec.Replicas, ReadyReplicas: instance.Status.Replicas, } if newStatus.Replicas == newStatus.ReadyReplicas { newStatus.Phase = janv1.Running } else { newStatus.Phase = janv1.NotReady } if !reflect.DeepEqual(instance.Status, newStatus) { instance.Status = newStatus log.FromContext(ctx).Info("update game status", "name", instance.Name) err := r.Client.Status().Update(ctx, instance) return reconcile.Result{}, err } return reconcile.Result{}, nil } // SetupWithManager sets up the controller with the Manager. func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&janv1.Jan{}). Complete(r) }

清空原来的Jan应用

[zhangpeng@zhangpeng develop-operator]$ kubectl delete jan jan-sample jan.jan.zhangpeng.com "jan-sample" deleted

apiVersion: jan.zhangpeng.com/v1 kind: Jan metadata: name: jan-sample spec: replicas: 2 image: nginx:1.7.9 ports: - port: 80 targetPort: 80 type: ClusterIP

[zhangpeng@zhangpeng develop-operator]$ kubectl apply -f config/samples/jan_v1_jan.yaml jan.jan.zhangpeng.com/jan-sample created [zhangpeng@zhangpeng develop-operator]$ kubectl get jan NAME PHASE DESIRED CURRENT READY AGE jan-sample Running 2 2 2 1s

问题又来了：

偷懒秘籍：

最终代码

上面感觉还是缺少点东西，什么呢？ingress我是否也可以封过来？

增加ingress相关字段

/* Copyright 2022 zhang peng. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package v1 import ( corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. // JanSpec defines the desired state of Jan type JanSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file //+kubebuilder:default:=1 //+kubebuilder:validation:Minimum:=1 Replicas *int32 `json:"replicas,omitempty" protobuf:"varint,1,opt,name=replicas"` Image string `json:"image"` Resources corev1.ResourceRequirements `json:"resources,omitempty"` Envs []corev1.EnvVar `json:"envs,omitempty"` Ports []corev1.ServicePort `json:"ports,omitempty"` Type corev1.ServiceType `json:"type,omitempty"` Host string `json:"host,omitempty"` } const ( Running = "Running" Pending = "Pending" NotReady = "NotReady" Failed = "Failed" ) // JanStatus defines the observed state of Jan type JanStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file // Phase is the phase of guestbook Phase string `json:"phase,omitempty"` // replicas is the number of Pods created by the StatefulSet controller. Replicas int32 `json:"replicas"` // readyReplicas is the number of Pods created by the StatefulSet controller that have a Ready Condition. ReadyReplicas int32 `json:"readyReplicas"` // LabelSelector is label selectors for query over pods that should match the replica count used by HPA. LabelSelector string `json:"labelSelector,omitempty"` } //+kubebuilder:printcolumn:name="Host",type="string",JSONPath=".spec.host",description="The host address." //+kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas,selectorpath=.status.labelSelector //+kubebuilder:printcolumn:name="Phase",type="string",JSONPath=".status.phase",description="The phase of game." //+kubebuilder:printcolumn:name="DESIRED",type="integer",JSONPath=".spec.replicas",description="The desired number of pods." //+kubebuilder:printcolumn:name="CURRENT",type="integer",JSONPath=".status.replicas",description="The number of currently all pods." //+kubebuilder:printcolumn:name="READY",type="integer",JSONPath=".status.readyReplicas",description="The number of pods ready." //+kubebuilder:printcolumn:name="AGE",type="date",JSONPath=".metadata.creationTimestamp",description="CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC." //+kubebuilder:object:root=true //+kubebuilder:subresource:status // Jan is the Schema for the jans API type Jan struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` Spec JanSpec `json:"spec,omitempty"` Status JanStatus `json:"status,omitempty"` } //+kubebuilder:object:root=true // JanList contains a list of Jan type JanList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` Items []Jan `json:"items"` } func init() { SchemeBuilder.Register(&Jan{}, &JanList{}) }

创建NewIngress方法

package jan import ( janv1 "develop-operator/apis/jan/v1" appv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" v1 "k8s.io/api/networking/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" ) func NewJan(app *janv1.Jan) *appv1.Deployment { labels := map[string]string{"app": app.Name} selector := &metav1.LabelSelector{MatchLabels: labels} return &appv1.Deployment{ TypeMeta: metav1.TypeMeta{ Kind: "apps/v1", APIVersion: "Deployment", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: appv1.DeploymentSpec{ Replicas: app.Spec.Replicas, Selector: selector, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{Labels: labels}, Spec: corev1.PodSpec{Containers: newContainers(app)}, }, MinReadySeconds: 0, }, Status: appv1.DeploymentStatus{}, } } func newContainers(app *janv1.Jan) []corev1.Container { containerPorts := []corev1.ContainerPort{} for _, svcPort := range app.Spec.Ports { cport := corev1.ContainerPort{} cport.ContainerPort = svcPort.TargetPort.IntVal containerPorts = append(containerPorts, cport) } return []corev1.Container{ { Name: app.Name, Image: app.Spec.Image, Resources: app.Spec.Resources, Ports: containerPorts, ImagePullPolicy: corev1.PullIfNotPresent, Env: app.Spec.Envs, }, } } func NewService(app *janv1.Jan) *corev1.Service { return &corev1.Service{ TypeMeta: metav1.TypeMeta{ Kind: "Service", APIVersion: "v1", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, OwnerReferences: []metav1.OwnerReference{ *metav1.NewControllerRef(app, schema.GroupVersionKind{ Group: janv1.GroupVersion.Group, Version: janv1.GroupVersion.Version, Kind: "Jan", }), }, }, Spec: corev1.ServiceSpec{ Type: app.Spec.Type, Ports: app.Spec.Ports, Selector: map[string]string{ "app": app.Name, }, }, } } const ( port = 80 ) func NewIngress(app *janv1.Jan) *v1.Ingress { pathType := v1.PathTypePrefix return &v1.Ingress{ TypeMeta: metav1.TypeMeta{ Kind: "Ingress", APIVersion: "v1", }, ObjectMeta: metav1.ObjectMeta{ Name: app.Name, Namespace: app.Namespace, }, Spec: v1.IngressSpec{ IngressClassName: nil, Rules: []v1.IngressRule{ { Host: app.Spec.Host, IngressRuleValue: v1.IngressRuleValue{ HTTP: &v1.HTTPIngressRuleValue{ Paths: []v1.HTTPIngressPath{{ Path: "/", PathType: &pathType, Backend: v1.IngressBackend{ Service: &v1.IngressServiceBackend{ Name: app.Name, Port: v1.ServiceBackendPort{ Number: int32(port), }, }, Resource: nil, }, }, }}}, }, }, }, } }

jan_controller.go中增加ingress相关

/* Copyright 2022 zhang peng. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package jan import ( "context" "encoding/json" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" "reflect" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/reconcile" janv1 "develop-operator/apis/jan/v1" ) // JanReconciler reconciles a Jan object type JanReconciler struct { client.Client Scheme *runtime.Scheme } //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/status,verbs=get;update;patch //+kubebuilder:rbac:groups=mar.zhangpeng.com,resources=jan/finalizers,verbs=update //+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=apps,resources=deployments/status,verbs=get;update;patch //+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=networking,resources=ingresses,verbs=get;list;watch;create;update;patch;delete // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. // TODO(user): Modify the Reconcile function to compare the state specified by // the Jan object against the actual cluster state, and then // perform operations to make the cluster state reflect the state specified by // the user. // // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.11.2/pkg/reconcile func (r *JanReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { defer utilruntime.HandleCrash() _ = log.FromContext(ctx) instance := &janv1.Jan{} err := r.Client.Get(context.TODO(), req.NamespacedName, instance) if err != nil { if errors.IsNotFound(err) { // Request object not found, could have been deleted after reconcile request. // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. // Return and don't requeue return reconcile.Result{}, nil } // Error reading the object - requeue the request. return reconcile.Result{}, err } if instance.DeletionTimestamp != nil { return reconcile.Result{}, err } // 如果不存在，则创建关联资源 // 如果存在，判断是否需要更新 // 如果需要更新，则直接更新 // 如果不需要更新，则正常返回 deploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, deploy); err != nil && errors.IsNotFound(err) { // 创建关联资源 // 1. 创建 Deploy deploy := NewJan(instance) if err := r.Client.Create(context.TODO(), deploy); err != nil { return reconcile.Result{}, err } // 2. 创建 Service service := NewService(instance) if err := r.Client.Create(context.TODO(), service); err != nil { return reconcile.Result{}, err } // 3. 创建 Ingress ingress := NewIngress(instance) if err := r.Client.Create(context.TODO(), ingress); err != nil { return reconcile.Result{}, err } // 4. 关联 Annotations data, _ := json.Marshal(instance.Spec) if instance.Annotations != nil { instance.Annotations["spec"] = string(data) } else { instance.Annotations = map[string]string{"spec": string(data)} } if err := r.Client.Update(context.TODO(), instance); err != nil { return reconcile.Result{}, nil } return reconcile.Result{}, nil } oldspec := janv1.JanSpec{} if err := json.Unmarshal([]byte(instance.Annotations["spec"]), &oldspec); err != nil { return reconcile.Result{}, err } if !reflect.DeepEqual(instance.Spec, oldspec) { data, _ := json.Marshal(instance.Spec) if instance.Annotations != nil { instance.Annotations["spec"] = string(data) } else { instance.Annotations = map[string]string{"spec": string(data)} } if err := r.Client.Update(context.TODO(), instance); err != nil { return reconcile.Result{}, nil } // 更新关联资源 newDeploy := NewJan(instance) oldDeploy := &appsv1.Deployment{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldDeploy); err != nil { return reconcile.Result{}, err } oldDeploy.Spec = newDeploy.Spec if err := r.Client.Update(context.TODO(), oldDeploy); err != nil { return reconcile.Result{}, err } newService := NewService(instance) oldService := &corev1.Service{} if err := r.Client.Get(context.TODO(), req.NamespacedName, oldService); err != nil { return reconcile.Result{}, err } oldService.Spec = newService.Spec if err := r.Client.Update(context.TODO(), oldService); err != nil { return reconcile.Result{}, err } return reconcile.Result{}, nil } newStatus := janv1.JanStatus{ Replicas: *instance.Spec.Replicas, ReadyReplicas: instance.Status.Replicas, } if newStatus.Replicas == newStatus.ReadyReplicas { newStatus.Phase = janv1.Running } else { newStatus.Phase = janv1.NotReady } if !reflect.DeepEqual(instance.Status, newStatus) { instance.Status = newStatus log.FromContext(ctx).Info("update game status", "name", instance.Name) err = r.Client.Status().Update(ctx, instance) if err != nil { return reconcile.Result{}, err } } return reconcile.Result{}, nil } // SetupWithManager sets up the controller with the Manager. func (r *JanReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&janv1.Jan{}). Complete(r) }

实验一下：

[zhangpeng@zhangpeng develop-operator]$ kubectl delete -f config/samples/jan_v1_jan.yaml jan.jan.zhangpeng.com "jan-sample" deleted

修改config/samples/jan_v1_jan.yaml 如下：

apiVersion: jan.zhangpeng.com/v1 kind: Jan metadata: name: jan-sample spec: replicas: 3 image: nginx:1.17.6 host: zhangpeng.com ports: - port: 80 targetPort: 80 type: ClusterIP

apiVersion: jan.zhangpeng.com/v1 kind: Jan metadata: name: jan-sample spec: replicas: 2 image: nginx:1.17.6 host: www1.zhangpeng.com ports: - port: 80 targetPort: 80 type: ClusterIP

总结一下：

operator要解决的是什么 自己还是没有搞明确，也没有想好怎么去设计一个operator。只是简单的实现了一些基本的功能，还没有体会到更多的便利性。 本来想照着eck写，但是对我这种初学者还是有点难，一步一步 去完善写吧......

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。