前言

容器化,云原生越演越烈,新概念非常之多。信息爆炸的同时,带来层层迷雾。我尝试从扩容出发理解其脉路,经过实践探索,整理形成一个入门教程,包括下面四篇文章。

这是第四篇,istio管理应用。

istio

服务网格(Service Mesh)这个术语通常用于描述构成这些应用程序的微服务网络以及应用之间的交互。随着规模和复杂性的增长,服务网格越来越难以理解和管理。它的需求包括服务发现、负载均衡、故障恢复、指标收集和监控以及通常更加复杂的运维需求,例如 A/B 测试、金丝雀发布、限流、访问控制和端到端认证等。

Istio 提供了一个完整的解决方案,通过为整个服务网格提供行为洞察和操作控制来满足微服务应用程序的多样化需求。

安装istio

istio1.1支持kubernetes1.11,1.12,1.13,Docker for mac的stable版本太低,需要切换Docker for mac 到 *edge*版本,kubernetes会升级到1.13。使用 kubectl version 确认kubernetes版本:

1
2
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T21:04:45Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}

按照安装指南安装istio到k8s。 kubectl get svc -n istio-system 查看isto服务(svc是service的速记别名)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
NAME                     TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                                                                                      AGE
grafana                  ClusterIP      10.96.238.84     <none>        3000/TCP                                                                                                                                     19s
istio-citadel            ClusterIP      10.110.233.235   <none>        8060/TCP,15014/TCP                                                                                                                           19s
istio-egressgateway      ClusterIP      10.107.81.184    <none>        80/TCP,443/TCP,15443/TCP                                                                                                                     20s
istio-galley             ClusterIP      10.108.48.185    <none>        443/TCP,15014/TCP,9901/TCP                                                                                                                   20s
istio-ingressgateway     LoadBalancer   10.99.64.197     localhost     15020:31681/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:32145/TCP,15030:31376/TCP,15031:30092/TCP,15032:31491/TCP,15443:32689/TCP   19s
istio-pilot              ClusterIP      10.101.105.130   <none>        15010/TCP,15011/TCP,8080/TCP,15014/TCP                                                                                                       19s
istio-policy             ClusterIP      10.107.208.244   <none>        9091/TCP,15004/TCP,15014/TCP                                                                                                                 19s
istio-sidecar-injector   ClusterIP      10.104.152.19    <none>        443/TCP                                                                                                                                      19s
istio-telemetry          ClusterIP      10.102.233.77    <none>        9091/TCP,15004/TCP,15014/TCP,42422/TCP                                                                                                       19s
jaeger-agent             ClusterIP      None             <none>        5775/UDP,6831/UDP,6832/UDP                                                                                                                   18s
jaeger-collector         ClusterIP      10.100.208.250   <none>        14267/TCP,14268/TCP                                                                                                                          18s
jaeger-query             ClusterIP      10.104.246.180   <none>        16686/TCP                                                                                                                                    18s
kiali                    ClusterIP      10.100.34.9      <none>        20001/TCP                                                                                                                                    19s
prometheus               ClusterIP      10.103.238.247   <none>        9090/TCP                                                                                                                                     19s
tracing                  ClusterIP      10.103.49.38     <none>        80/TCP                                                                                                                                       18s
zipkin                   ClusterIP      10.105.126.70    <none>        9411/TCP

查看isto的pod是否全部正常启动:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
➜  kubectl get pods -n istio-system
NAME                                      READY   STATUS      RESTARTS   AGE
grafana-749c78bcc5-jczt2                  1/1     Running     1          20m
istio-citadel-899dfb67c-r6cmx             1/1     Running     0          20m
istio-cleanup-secrets-1.1.3-d42jx         0/1     Completed   0          20m
istio-egressgateway-748d5fd794-zkcjs      1/1     Running     1          20m
istio-galley-555dd7c7d7-t9fhz             1/1     Running     0          20m
istio-grafana-post-install-1.1.3-2qzg5    0/1     Completed   0          20m
istio-ingressgateway-55dd86767f-n5v7m     1/1     Running     1          20m
istio-pilot-7979d58649-lg6vq              2/2     Running     0          20m
istio-policy-f89c945dc-8d6mn              2/2     Running     0          55s
istio-security-post-install-1.1.3-4ggpf   0/1     Completed   0          20m
istio-sidecar-injector-998dd6cbb-fbdgw    1/1     Running     0          20m
istio-telemetry-7d9d866c65-2pngr          2/2     Running     0          16s
istio-tracing-595796cf54-mp6tp            1/1     Running     1          20m
kiali-5df77dc9b6-jq26m                    1/1     Running     0          20m
prometheus-7f87866f5f-p2568               1/1     Running     0          20m

部署应用

调整pods符合istio规范

部署引用前,需要按照Istio对Pod和服务的要求,进行微调。

微调后的istio\flaskapp.yaml如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
apiVersion: v1
kind: Service
metadata:
  name: flaskapp
spec:
  ports:
    - port: 5000
      name: http
  selector:
    app: business
---
apiVersion: extensions/v1beta1 
kind: Deployment
metadata:
  name: flaskapp
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: business
        version: v1
    spec:
      containers:
      - image: flaskapp:0.0.2
        name: flaskapp
        ports:
        - containerPort: 5000

微调后的istio\redis.yaml如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  ports:
    - port: 6379
      name: redis
  selector:
    app: redis
---
apiVersion: extensions/v1beta1 
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
        version: v1
    spec:
      containers:
      - image: redis:4-alpine3.8
        name: redisdb
        ports:
        - containerPort: 6379

因为istio附带ingress,所以我们取消了前置的nginx负载,直接使用ingress。

启动istio的sidecar自动注入

istio通过在pod中输入sidecar,用来管理流量,设置default名称空间下默认注入:

1
2
➜  docker2istio kubectl label namespace default istio-injection=enabled
namespace/default labeled

启动服务

启动服务方式和k8s没有区别

1
2
3
4
5
➜  docker2istio kubectl apply -f k8s/redis.yaml -f k8s/flaskapp.yaml
service/redis created
deployment.extensions/redis created
service/flaskapp created
deployment.extensions/flaskapp created

使用kubectl describe pod/flaskapp-757cd47df4-zkzv2 确认istio正常管理app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
Name:               flaskapp-757cd47df4-zkzv2
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               docker-desktop/192.168.65.3
Start Time:         Wed, 24 Apr 2019 18:43:16 +0800
Labels:             app=business
                    pod-template-hash=757cd47df4
                    version=v1
Annotations:        sidecar.istio.io/status:
                      {"version":"3420543c87a5049f8ec099530c3992a5c1f06bf54fa56d7a3877e7ffc658ea8d","initContainers":["istio-init"],"containers":["istio-proxy"]...
Status:             Running
IP:                 10.1.0.45
Controlled By:      ReplicaSet/flaskapp-757cd47df4
Init Containers:
  istio-init:
    Container ID:  docker://0bac5e70832c287a1c446d38034c75ebe5b8e66b9e2fb6d5a1fc3bcb8a9f644c
    Image:         docker.io/istio/proxy_init:1.1.3
    Image ID:      docker-pullable://istio/proxy_init@sha256:000d022d27c198faa6cc9b03d806482d08071e146423d6e9f81aa135499c4ed3
    Port:          <none>
    Host Port:     <none>
    Args:
      
      ...
      
Containers:
  flaskapp:
    Container ID:   docker://9cdf76e12c710f1d50c2b3a2ac349909f481975f220af5b31e1b76717bcabd95
    Image:          flaskapp:0.0.2
    Image ID:       docker://sha256:d52877069956696bb9009c0af43a0d339d2d253e67f5fb09c9f1f052a35528de
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Wed, 24 Apr 2019 18:43:19 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-ndv6q (ro)
  istio-proxy:
    Container ID:  docker://84eecd89d7c203b7b97749d2a02c63cb8b4c57cfda96f0c7585547a8918344c1
    Image:         docker.io/istio/proxyv2:1.1.3
    Image ID:      docker-pullable://istio/proxyv2@sha256:b682918f2f8fcca14b3a61bbd58f4118311eebc20799f24b72ceddc5cd749306
    Port:          15090/TCP
    Host Port:     0/TCP
    Args:
    
      ...
    
    State:          Running
      Started:      Wed, 24 Apr 2019 18:43:19 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  128Mi
    Requests:
      cpu:      10m
      memory:   40Mi
    Readiness:  http-get http://:15020/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30
    Environment:
      POD_NAME:                      flaskapp-757cd47df4-zkzv2 (v1:metadata.name)
      POD_NAMESPACE:                 default (v1:metadata.namespace)
      INSTANCE_IP:                    (v1:status.podIP)
      ISTIO_META_POD_NAME:           flaskapp-757cd47df4-zkzv2 (v1:metadata.name)
      ISTIO_META_CONFIG_NAMESPACE:   default (v1:metadata.namespace)
      ISTIO_META_INTERCEPTION_MODE:  REDIRECT
      ISTIO_METAJSON_LABELS:         {"app":"business","pod-template-hash":"757cd47df4","version":"v1"}

    Mounts:
      /etc/certs/ from istio-certs (ro)
      /etc/istio/proxy from istio-envoy (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-ndv6q (ro)

...

!!!注意 这里的pod,会被自动注入名为istio-init的initContainer和名为**istio-proxy的container。具有这2个container,标志pod接受istio流量管理。

确认服务正常启动

1
2
3
4
5
➜  docker2istio kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
flaskapp     ClusterIP   10.107.67.44     <none>        5000/TCP       4m31s
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        36m
redis        ClusterIP   10.100.210.72    <none>        6379/TCP       4m31s

确认pod正常启动

1
2
3
4
➜  docker2istio kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
flaskapp-77ddb9698c-xb2gm   1/1     Running   0          4m2s
redis-c5dd6fcfc-8tqwm       1/1     Running   0          4m2s

部署istio的gateway

因为取消了前置的nginx负载,需要设置gateway,集群外部才能够访问服务。istio\gateway.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: flaskapp-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - "*"
  gateways:
  - flaskapp-gateway
  http:
  - match:
    - uri:
        exact: /
    route:
    - destination:
        host: flaskapp
        port:
          number: 5000

kubectl apply -f istio/gateway.yaml提交后,查看gameway的端口:

1
2
3
4
➜  docker2istio kubectl get svc istio-ingressgateway -n istio-system

NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                                                                                                                                      AGE
istio-ingressgateway   LoadBalancer   10.99.64.197   localhost     15020:31681/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:32145/TCP,15030:31376/TCP,15031:30092/TCP,15032:31491/TCP,15443:32689/TCP   179m

这里的服务类型为LoadBalancer,可以直接使用80端口访问服务:

1
2
➜  docker2istio curl http://localhost
Hello World by 10.1.0.40 from 10.1.0.30 ! 该页面已被访问 1 次。

金丝雀发布

istio的流量管理提供了非常便捷的金丝雀发布功能。

首先,我们调整app/flaskapp.py,将域名获取调整成为主机名称:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
#-*- coding:utf-8 -*-
import socket
from flask import Flask
from redis import Redis


app = Flask(__name__)
redis = Redis(host='redis', port=6379)

def get_host_ip():
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        s.connect(('8.8.8.8', 80))
        ip = s.getsockname()[0]
    except:
        ip = '127.0.0.1'
    finally:
        s.close()
    return ip

def get_hostname():
    return socket.gethostname()

@app.route('/')
def hello():
    app.logger.debug('hello in')
    from flask import request
    count = redis.incr('hits')
    # host_ip = get_host_ip()
    host_id = get_hostname()
    client_ip= request.headers['X-Real-Ip'] if 'X-Real-Ip' in request.headers else request.remote_addr 
    app.logger.debug("Hello out {} {} {}:".format(host_id,client_ip,count))
    return 'Hello World by {} from {} ! 该页面已被访问 {} 次。\n'.format(host_id,client_ip,count)


if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True)

使用 docker build -f app/Dockerfile -t flaskapp:0.0.3 app 编译新的镜像。

制作新的deployment,使其使用新的镜像:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
apiVersion: extensions/v1beta1 
kind: Deployment
metadata:
  name: flaskapp-v2
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: business
        version: v2
    spec:
      containers:
      - image: flaskapp:0.0.3
        name: flaskapp
        ports:
        - containerPort: 5000

创建并确认新的部署生效:

1
2
3
4
5
6
7
➜  docker2istio kubectl apply -f istio/flaskappv2.yaml
deployment.extensions/flaskapp-v2 created
➜  docker2istio kubectl get deployment
NAME          READY   UP-TO-DATE   AVAILABLE   AGE
flaskapp      3/3     3            3           72m
flaskapp-v2   1/1     1            1           7s
redis         1/1     1            1           129m

调整VirtualService,控制流量分发策略:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flaskapp
spec:
  hosts:
  - "*"
  gateways:
  - flaskapp-gateway
  http:
  - match:
    - uri:
        exact: /
    route:
    - destination:
        host: flaskapp
        port:
          number: 5000
  - route:
    - destination:
        host: flaskapp
        subset: v1
      weight: 80
    - destination:
        host: flaskapp
        subset: v2
      weight: 20

这里重点是指定规则,按照8:2,将flaskapp的访问分配到deployment v1 和 deployment v2 上。

kubectl apply -f istio/gateway2.yaml 应用新的流量分配策略:

访问观察服务:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 7 次。
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 8 次。
➜  docker2istio curl http://localhost
Hello World by flaskapp-v2-ff9f8cfdb-vh788 from 127.0.0.1 ! 该页面已被访问 9 次。
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 10 次。
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 11 次。
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 12 次。
➜  docker2istio curl http://localhost
Hello World by 10.1.0.45 from 127.0.0.1 ! 该页面已被访问 13 次。
➜  docker2istio curl http://localhost
Hello World by flaskapp-v2-ff9f8cfdb-vh788 from 127.0.0.1 ! 该页面已被访问 14 次。

可以看到,访问被按比例分配到v1和v2两个版本上。

总结

从上面实践过程空可见,istio完善了k8s提供的编排功能,在纵向扩容的基础上可以实现横向扩展。