前言
在Kubernetes中部署spring boot应用整体上来说是一件比较繁琐的事情,而Spring Boot Operator则能带给你更清爽简单的体验。
Spring Boot Operator基于Kubernetes的custom resource definitions (CRDs)扩展API进行的开发。
打包Docker镜像
在讲部署之前我们需要先将我们的SpringBoot应用打包成标准的DockerImage。
Spring Boot 基础就不介绍了,推荐下这个实战教程: https://www.javastack.cn/categories/Spring-Boot/
java项目打包镜像用maven/gradle插件比较多,我的另一篇文章构建SpringBoot的Docker镜像,这里在介绍一个新的google开源的插件Jib,该插件使用起来比较方便。
注意:jib打包的镜像会导致java应用的pid=1,在使用SpringBootOperator进行发布时候,Operator会设置kubernetes的ShareProcessNamespace参数为true(v1.10+版本都可使用)来解决该问题。
下面就来演示一下我们通过https://start.spring.io生成一个标准的SpringBoot项目operator-demo,然后使用jib插件进行镜像打包
scriptmvncom.google.cloud.tools:jib-maven-plugin:build\-Djib.to.auth.username=${{secrets.MY_USERNAME}}\-Djib.to.auth.password=${{secrets.MY_PASSWORD}}\-Djib.container.jvmFlags=--add-opens,java.base/sun.nio.ch=ALL-UNNAMED\-Djib.from.image=freemanliu/oprenjre:11.0.5\-Dimage=registry.cn-shanghai.aliyuncs.com/qingmuio/operator-demo/operator-demo:v1.0.0
执行上面的命令之后我们将得到一个标准的docker镜像,该镜像会被推送到远程仓库。
Operator快速体验
完成了镜像的构建之后,我们紧接着来安装我们的Operator到kubernetes集群,当然了首先你需要一套集群,可以参考我之前一篇文章部署高可用kubernetes,虽然版本比较老,但是新版本其实也差不多的一个思路。
快速安装
此处快速安装只是为了快速体验demo
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml
apply成功之后控制台输出
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated
稍等片刻查看是否已经安装成功
scriptkubectlgetpo-nspring-boot-operator-system
成功如下输出
NAMEREADYSTATUSRESTARTSAGEspring-boot-operator-controller-manager-7f498596bb-wcwtn2/2Running02m15s
部署OperatorDemo应用
完成了Operator的部署之后,我们来部署我们第一个应用,这里我们就发布上面我们编写的springboot应用opreator-demo。 首先我们需要先编写一个Spring Boot Application 的CRD部署yaml,如下
#Demo.yamlapiVersion:springboot.qingmu.io/v1alpha1kind:SpringBootApplicationmetadata:name:operator-demospec:springBoot:version:v1.0.0#image:registry.cn-shanghai.aliyuncs.com/qingmuio/operator-demo/operator-demo:v1.0.0
细心的同学可能发现了,为啥连Image
都没有?这怎么发布,就name,version,就能完成发布?是的没错!就能完成发布,后面我讲详细讲到他是如何完成的。 接着我们apply一下
scriptkubectlapply-fDemo.yaml
看到console输出
springbootapplication.springboot.qingmu.io/operator-democreated
验证
表示创建成功了,接着我们来看下我们部署的第一个应用,这里我们直接用上面的yaml中的name过滤即可。 查看pod
script~#kubectlgetpo|grepoperator-demooperator-demo-7574f4789c-mg58m1/1Running076soperator-demo-7574f4789c-ssr8v1/1Running076soperator-demo-7574f4789c-sznww1/1Running076s
查看下我们的pid不等于1的设置是否生效,根据下面的结果可以看到通过设置ShareProcessNamespace该参数我们可以在Kubernetes层面来解决这个pid=1的问题。
scriptkubectlexec-itoperator-demo-7574f4789c-mg58mbashbash-5.0#ps-efUIDPIDPPIDCSTIMETTYTIMECMDroot10002:06?00:00:00/pauseroot602602:06?00:00:09java--add-opensjava.base/sun.nio.ch=ALL-UNNAMED-cp/app/resources:/app/classes:/app/libs/*io.qingmu.operator.operatordemo.Oper...root380002:07pts/000:00:00bashroot4438002:07pts/000:00:00ps-ef
查看svc
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml0
我们来访问一下试试。
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml1
我们来试着缩减他的副本数到1个 编辑我们的Demo.yaml,加入一个新的属性replicas
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml2
应用一下
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml3
再次查看pod,你会发现我们的pod已经缩放为一个副本了
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml4
清理operator-demo
要删除该pod 我们只需要执行delete即可
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml5
再次查看pod,已经没了
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml6
部署自己的应用
部署自己私有仓库的应用需要需要先创建secret(如果已经创建跳过即可) 创建docker-registry的secret
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml7
自己应用的crd Yaml
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml8
一个完整的Spring Boot Application Yaml
下面是一个完整的yaml属性结构,大部分属性我们都可以用默认配置的即可。 不设置属性,默认使用Operator中设置的通用值详见后面的自定义安装Operator。
Spring Boot 基础就不介绍了,推荐下这个实战教程: https://www.javastack.cn/categories/Spring-Boot/
scriptkubectlapply-fhttps://raw.githubusercontent.com/goudai/spring-boot-operator/master/manifests/deployment.yaml9
优雅停机的路径
由于优雅停机默认是关闭的并且并不支持Get请求所以我们需要开启和搭个桥 首先在application.yml
中启用
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated0
然后桥接一个Get方法
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated1
node亲和的使用
举一个列子 我们有一个springboot应用 user-service 希望他能分布到3个可用区的6个节点上: 首先我们把机器划分多个可用区
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated2
现在我们有三个可以区 每个区有2台workload,一共6台。然后我们需要给这些机器分别打上label。 将全部的i区机器标注为cn-i
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated3
同理将h区的标注为h,g区同理
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated4
现在准备工作我们就绪了,现在我们来设置让它达到我们的调度效果,像如下编写即可。
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated5
Operator 自定义安装
上面我们快速的安装了好了,接着我们来讲解下如何自定义安装,以及有哪些自定义的参数,可以个性化的参数我们用环境变量的方式注入。 下面来修改Deployment
完成自己个性化的配置部署,从我提供的部署yaml中拉倒最后,找到name是spring-boot-operator-controller-manager
的Deployment,我们将修改它。
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated6
自定义安装之后部署
yaml可以简化为如下。
namespace/spring-boot-operator-systemcreatedcustomresourcedefinition.apiextensions.k8s.io/springbootapplications.springboot.qingmu.iocreatedrole.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-manager-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolecreatedclusterrole.rbac.authorization.k8s.io/spring-boot-operator-metrics-readercreatedrolebinding.rbac.authorization.k8s.io/spring-boot-operator-leader-election-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-manager-rolebindingcreatedclusterrolebinding.rbac.authorization.k8s.io/spring-boot-operator-proxy-rolebindingcreatedservice/spring-boot-operator-controller-manager-metrics-servicecreateddeployment.apps/spring-boot-operator-controller-managercreated7
附录
环境变量表格
来源:https://qingmu.io/2020/04/08/Spring-Boot-Operator-User-Guide/