一区二区国产日产,欧美激情精品,免费一级欧美在线大片

kubernetes導出有狀態服務

kubernetes導出有狀態服務

(金慶的專欄 2018.7)

網游服務器中的房間服務器是有狀態服務器，可以用 kubernetes statefulset 開啟多個實例。

為了讓客戶端能夠直連房間服務器，除了 statefulset 要求的 headless 服務，
還須為每個實例創建 NodePort 類型的服務, 并且選擇Pod和禁止轉發。

下面 bootcamp.yml 先創建了 bootcamp headless 服務(clusterIP: None),
又創建了 bootcamp StatefulSet, 實例個數為 2.
然后創建 bootcamp-0,1,2 服務，分別對應 bootcamp-0,1,2 pod.

服務個數大于實例個數，是想測試下服務沒有對應的實例時的表現。

網游中的匹配服務器將分配一個房間給客戶端，列舉 bootcamp-0,1,2 pod 所在節點的外網 IP,
連同對應服務的端口，發送給客戶端，讓客戶端直連。

[jinqing@host-10-240-79-10 statefulset]$ cat bootcamp.yml
apiVersion: v1
kind: Service
metadata:
name: bootcamp
namespace: jq
labels:
    name: bootcamp
spec:
ports:
    - port: 8080
clusterIP: None # StatefulSet要求Headless服務
selector:
    app: bootcamp # 選擇 bootcamp 應用

---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: bootcamp
namespace: jq
spec:
serviceName: bootcamp # 上面的 Headless 服務名
replicas: 2
template:
    metadata:
      labels:
        app: bootcamp # 應用名，與服務中的 selector 對應
    spec:
      containers:
      - name: bootcamp
        image: docker.io/jocatalin/kubernetes-bootcamp:v1

---
kind: Service
apiVersion: v1
metadata:
name: bootcamp-0
namespace: jq
spec:
type: NodePort # 對外服務
externalTrafficPolicy: Local # 不要轉發到其他節點
selector:
    app: bootcamp
    statefulset.kubernetes.io/pod-name: bootcamp-0 # 選擇 pod
ports:
- protocol: TCP
    nodePort: 30880 # 對外端口
    port: 8080
---
kind: Service
apiVersion: v1
metadata:
name: bootcamp-1
namespace: jq
spec:
type: NodePort
externalTrafficPolicy: Local
selector:
    app: bootcamp
    statefulset.kubernetes.io/pod-name: bootcamp-1
ports:
- protocol: TCP
    nodePort: 30881
    port: 8080
---
kind: Service
apiVersion: v1
metadata:
name: bootcamp-2
namespace: jq
spec:
type: NodePort
externalTrafficPolicy: Local
selector:
    app: bootcamp
    statefulset.kubernetes.io/pod-name: bootcamp-2
ports:
- protocol: TCP
    nodePort: 30882
    port: 8080

因為 statefulset 的每個實例有不同的標簽，所以可以為服務選擇一個實例。

利用 externalTrafficPolicy: Local 設置來禁止轉發。
參考 service.spec.externalTrafficPolicy 的說明：

https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-type-nodeport

Setting service.spec.externalTrafficPolicy to the value Local will only proxy requests to local endpoints, never forwarding traffic to other nodes and thereby preserving the original source IP address. If there are no local endpoints, packets sent to the node are dropped, ...

因為有可能多個Pod開在同一節點上，所以對外端口設成了不同的 30880-30882。
如果限制每個節點只開一個實例，則對外端口可以設成同一個。

創建服務：

[jinqing@host-10-240-79-10 statefulset]$ kubectl apply -f bootcamp.yml
service "bootcamp" created
statefulset.apps "bootcamp" created
service "bootcamp-0" created
service "bootcamp-1" created
service "bootcamp-2" created

服務如下：

[jinqing@host-10-240-79-10 statefulset]$ kubectl get service -n jq
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
bootcamp     ClusterIP   None             <none>        8080/TCP         3m
bootcamp-0   NodePort    10.96.128.137    <none>        8080:30880/TCP   3m
bootcamp-1   NodePort    10.109.2.56      <none>        8080:30881/TCP   3m
bootcamp-2   NodePort    10.102.181.193   <none>        8080:30882/TCP   3m

2個實例：

[jinqing@host-10-240-79-10 statefulset]$ kubectl get pod -n jq -o wide
NAME         READY     STATUS    RESTARTS   AGE       IP            NODE
bootcamp-0   1/1       Running   0          4m        10.244.0.42   host-10-240-79-10
bootcamp-1   1/1       Running   0          4m        10.244.1.63   host-10-240-79-11

訪問服務必須指定節點，不會自動轉發：

[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.10:30880
Hello Kubernetes bootcamp! | Running on: bootcamp-0 | v=1
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.10:30881
curl: (7) Failed connect to 10.240.79.10:30881; Connection timed out
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30880
curl: (7) Failed connect to 10.240.79.11:30880; Connection timed out
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1

沒有負載均衡，30881端口總是訪問 bootcamp-1：

[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30881
Hello Kubernetes bootcamp! | Running on: bootcamp-1 | v=1

也可以從外網訪問.

30882 端口無法連接：

[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.11:30882
curl: (7) Failed connect to 10.240.79.11:30882; Connection refused
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.10:30882
curl: (7) Failed connect to 10.240.79.10:30882; Connection refused

3個端口都有監聽：

[root@host-10-240-79-11 tmp]# netstat -ntl | grep 3088
tcp6       0      0 :::30881                :::*                    LISTEN
tcp6       0      0 :::30882                :::*                    LISTEN
tcp6       0      0 :::30880                :::*                    LISTEN

iptables-save 輸出如下, 其中 10.244是Pod的網段。

沒有實例運行的節點上，會丟棄請求：

-A KUBE-NODEPORTS -s 127.0.0.0/8 -p tcp -m comment --comment "jq/bootcamp-1:" -m tcp --dport 30881 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "jq/bootcamp-1:" -m tcp --dport 30881 -j KUBE-XLB-LJXDQ4W47M42IZBH
-A KUBE-NODEPORTS -s 127.0.0.0/8 -p tcp -m comment --comment "jq/bootcamp-0:" -m tcp --dport 30880 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "jq/bootcamp-0:" -m tcp --dport 30880 -j KUBE-XLB-U5NEOQT6R5WSBVOH

-A KUBE-XLB-LJXDQ4W47M42IZBH -s 10.244.0.0/16 -m comment --comment "Redirect pods trying to reach external loadbalancer VIP to clusterIP" -j KUBE-SVC-LJXDQ4W47M42IZBH
-A KUBE-XLB-LJXDQ4W47M42IZBH -m comment --comment "jq/bootcamp-1: has no local endpoints" -j KUBE-MARK-DROP
-A KUBE-XLB-U5NEOQT6R5WSBVOH -s 10.244.0.0/16 -m comment --comment "Redirect pods trying to reach external loadbalancer VIP to clusterIP" -j KUBE-SVC-U5NEOQT6R5WSBVOH
-A KUBE-XLB-U5NEOQT6R5WSBVOH -m comment --comment "jq/bootcamp-0: has no local endpoints" -j KUBE-MARK-DROP

有實例運行的節點上會轉發給 Pod 8080：

-A KUBE-NODEPORTS -s 127.0.0.0/8 -p tcp -m comment --comment "jq/bootcamp-0:" -m tcp --dport 30880 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "jq/bootcamp-0:" -m tcp --dport 30880 -j KUBE-XLB-U5NEOQT6R5WSBVOH
-A KUBE-NODEPORTS -s 127.0.0.0/8 -p tcp -m comment --comment "jq/bootcamp-1:" -m tcp --dport 30881 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "jq/bootcamp-1:" -m tcp --dport 30881 -j KUBE-XLB-LJXDQ4W47M42IZBH

-A KUBE-XLB-LJXDQ4W47M42IZBH -s 10.244.0.0/16 -m comment --comment "Redirect pods trying to reach external loadbalancer VIP to clusterIP" -j KUBE-SVC-LJXDQ4W47M42IZBH
-A KUBE-XLB-LJXDQ4W47M42IZBH -m comment --comment "Balancing rule 0 for jq/bootcamp-1:" -j KUBE-SEP-LJQA4WUIKJUQ5ALU
-A KUBE-XLB-U5NEOQT6R5WSBVOH -s 10.244.0.0/16 -m comment --comment "Redirect pods trying to reach external loadbalancer VIP to clusterIP" -j KUBE-SVC-U5NEOQT6R5WSBVOH
-A KUBE-XLB-U5NEOQT6R5WSBVOH -m comment --comment "jq/bootcamp-0: has no local endpoints" -j KUBE-MARK-DROP

-A KUBE-SEP-LJQA4WUIKJUQ5ALU -s 10.244.1.63/32 -m comment --comment "jq/bootcamp-1:" -j KUBE-MARK-MASQ
-A KUBE-SEP-LJQA4WUIKJUQ5ALU -p tcp -m comment --comment "jq/bootcamp-1:" -m tcp -j DNAT --to-destination 10.244.1.63:8080

30882 端口無法連接
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "jq/bootcamp-2: has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 30882 -j REJECT --reject-with icmp-port-unreachable

測試下擴容：

[jinqing@host-10-240-79-10 statefulset]$ kubectl get statefulset -n jq
NAME       DESIRED   CURRENT   AGE
bootcamp   2         2         45m
[jinqing@host-10-240-79-10 statefulset]$ kubectl scale --replicas=3 statefulset/bootcamp -n jq
statefulset.apps "bootcamp" scaled
[jinqing@host-10-240-79-10 statefulset]$ kubectl get statefulset -n jq
NAME       DESIRED   CURRENT   AGE
bootcamp   3         3         47m
[jinqing@host-10-240-79-10 statefulset]$ kubectl get pod -n jq -o wide
NAME         READY     STATUS    RESTARTS   AGE       IP            NODE
bootcamp-0   1/1       Running   0          48m       10.244.0.42   host-10-240-79-10
bootcamp-1   1/1       Running   0          48m       10.244.1.63   host-10-240-79-11
bootcamp-2   1/1       Running   0          45s       10.244.2.60   host-10-240-79-12
[jinqing@host-10-240-79-10 statefulset]$ curl 10.240.79.12:30882
Hello Kubernetes bootcamp! | Running on: bootcamp-2 | v=1

posted on 2018-07-14 11:43 金慶閱讀(891) 評論(0) 編輯收藏引用所屬分類: 9. 其它、2. 網游開發

只有注冊用戶登錄后才能發表評論。
【推薦】100%開源！大型工業跨平臺軟件C++源碼提供，建模，組態！

相關文章: TortoiseGit is OK but GitExtensions fails DeathVoteExpirationTimeout in Orleans How to delete local branches of GitExtension Clustering provider in Orleans Why Orleans' actor is virutal What comes after microservice? Rust Deref coercion example Rust Error Return Check Policy Rust visibility Why does Rust check borrow even in single thread

網站導航: 博客園 IT新聞 BlogJava 博問 Chat2DB 管理

金慶的專欄

公告

留言簿(12)

隨筆分類(502)

隨筆檔案(423)

相冊

搜索

積分與排名

最新評論

閱讀排行榜

評論排行榜