kube-registry-proxy不暴露任何節點上的端口5000

Question

我在我的kubernetes集羣中使用private Docker registry addon，並且我希望在每個節點上公開5000端口以便輕鬆地從localhost：5000中提取圖像。因此，我在每個節點上放置了一個pod清單文件/etc/kubernetes/manifests/kube-registry-proxy.manifest以啓動端口5000的本地代理。它在幾個月前在裸機ubuntu上手動部署kubernetes時起作用，但在嘗試kargo時，端口5000沒有偵聽時失敗。

我使用kargo用白布網絡插件，泊塢窗註冊表的配置是：

apiVersion: v1 
kind: Pod 
metadata: 
    name: kube-registry-proxy 
    namespace: kube-system 
spec: 
    containers: 
    - name: kube-registry-proxy 
    image: gcr.io/google_containers/kube-registry-proxy:0.3 
    resources: 
     limits: 
     cpu: 100m 
     memory: 50Mi 
    env: 
    - name: REGISTRY_HOST 
     value: kube-registry.kube-system.svc.cluster.local 
    - name: REGISTRY_PORT 
     value: "5000" 
    - name: FORWARD_PORT 
     value: "5000" 
    ports: 
    - name: registry 
     containerPort: 5000 
     hostPort: 5000 

KUBE-登記處送交：

kind: PersistentVolume 
apiVersion: v1 
metadata: 
    name: kube-system-kube-registry-pv 
    labels: 
    kubernetes.io/cluster-service: "true" 
spec: 
    capacity: 
    storage: 500Gi 
    accessModes: 
    - ReadWriteMany 
    persistentVolumeReclaimPolicy: Retain 
    hostPath: 
    path: /registry 

kind: PersistentVolumeClaim 
apiVersion: v1 
metadata: 
    name: kube-registry-pvc 
    namespace: kube-system 
    labels: 
    kubernetes.io/cluster-service: "true" 
spec: 
    accessModes: 
    - ReadWriteMany 
    resources: 
    requests: 
     storage: 500Gi 

apiVersion: v1 
kind: ReplicationController 
metadata: 
    name: kube-registry-v0 
    namespace: kube-system 
    labels: 
    k8s-app: kube-registry 
    version: v0 
    kubernetes.io/cluster-service: "true" 
spec: 
    replicas: 1 
    selector: 
    k8s-app: kube-registry 
    version: v0 
    template: 
    metadata: 
     labels: 
     k8s-app: kube-registry 
     version: v0 
     kubernetes.io/cluster-service: "true" 
    spec: 
     containers: 
     - name: registry 
     image: registry:2.5.1 
     resources: 
      # keep request = limit to keep this container in guaranteed class 
      limits: 
      cpu: 100m 
      memory: 100Mi 
      requests: 
      cpu: 100m 
      memory: 100Mi 
     env: 
     - name: REGISTRY_HTTP_ADDR 
      value: :5000 
     - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY 
      value: /var/lib/registry 
     volumeMounts: 
     - name: image-store 
      mountPath: /var/lib/registry 
     ports: 
     - containerPort: 5000 
      name: registry 
      protocol: TCP 
     volumes: 
     - name: image-store 
     persistentVolumeClaim: 
      claimName: kube-registry-pvc 

apiVersion: v1 
kind: Service 
metadata: 
    name: kube-registry 
    namespace: kube-system 
    labels: 
    k8s-app: kube-registry 
    kubernetes.io/cluster-service: "true" 
    kubernetes.io/name: "KubeRegistry" 
spec: 
    selector: 
    k8s-app: kube-registry 
    ports: 
    - name: registry 
    port: 5000 
    protocol: TCP 

我已經運行kargo之前創建一個吊艙清單文件/etc/kubernetes/manifests/kube-registry-proxy.manifest代理正在所有節點上運行，但沒有任何監聽端口5000.某些輸出：

ubuntu@k8s15m1:~$ kubectl get all --all-namespaces | grep registry-proxy 
kube-system po/kube-registry-proxy-k8s15m1    1/1  Running    1   1h 
kube-system po/kube-registry-proxy-k8s15m2    1/1  Running    0   1h 
kube-system po/kube-registry-proxy-k8s15s1    1/1  Running    0   1h 

ubuntu@k8s15m1:~$ docker ps | grep registry 
756fcf674288  gcr.io/google_containers/kube-registry-proxy:0.3  "/usr/bin/run_proxy"  19 minutes ago  Up 19 minutes       k8s_kube-registry-proxy.bebf6da1_kube-registry-proxy-k8s15m1_kube-system_a818b22dc7210ecd31414e328ae28e43_7221833c 

ubuntu@k8s15m1:~$ docker logs 756fcf674288 | tail 
waiting for kube-registry.kube-system.svc.cluster.local to come online 
starting proxy 

ubuntu@k8s15m1:~$ netstat -ltnp | grep 5000 

ubuntu@k8s15m1:~$ curl -v localhost:5000/v1/ 
* Trying 127.0.0.1... 
* connect to 127.0.0.1 port 5000 failed: Connection refused 
* Failed to connect to localhost port 5000: Connection refused 
* Closing connection 0 
curl: (7) Failed to connect to localhost port 5000: Connection refused 

ubuntu@k8s15m1:~$ kubectl get po kube-registry-proxy-k8s15m1 --namespace=kube-system -o wide 
NAME       READY  STATUS RESTARTS AGE  IP    NODE 
kube-registry-proxy-k8s15m1 1/1  Running 3   1h  10.233.69.64 k8s15m1 

ubuntu@k8s15m1:~$ curl -v 10.233.69.64:5000/v1/ 
* Trying 10.233.69.64... 
* Connected to 10.233.69.64 (10.233.69.64) port 5000 (#0) 
> GET /v1/ HTTP/1.1 
> Host: 10.233.69.64:5000 
> User-Agent: curl/7.47.0 
> Accept: */* 
> 
< HTTP/1.1 404 Not Found 
< Content-Type: text/plain; charset=utf-8 
< Docker-Distribution-Api-Version: registry/2.0 
< X-Content-Type-Options: nosniff 
< Date: Tue, 14 Mar 2017 16:41:56 GMT 
< Content-Length: 19 
< 
404 page not found 
* Connection #0 to host 10.233.69.64 left intact 

Answer 1

我認爲這裏有一些事情正在進行。

最重要的，要知道，Kubernetes Services有3種形式：ClusterIP（這是默認值），NodePort（這聽起來很像是你所期待發生），和LoadBalancer（我不會再提，但文檔）。

我希望，如果你更新你的Service明確請求type: NodePort，你會得到更接近你腦子裏想的是什麼（但要注意，除非你改變了它，NodePort ports are limited to 30000-32767

這樣：

apiVersion: v1 kind: Service metadata: name: kube-registry namespace: kube-system labels: k8s-app: kube-registry kubernetes.io/cluster-service: "true" kubernetes.io/name: "KubeRegistry" spec: type: NodePort # <--- updated line selector: k8s-app: kube-registry ports: - name: registry port: 5000 nodePort: 30500 # <-- you can specify, or omit this protocol: TCP

如果您對您希望服務偵聽的端口的意見，隨意指定，或者只是把它關閉，Kubernetes會選擇一個從可用空間。

爲了完整性，我將提及接下來的事情，但是我要說的是一個不好的做法，所以請不要這樣做。您也可以讓Pods直接監聽Node的實際TCP/IP堆棧，由specifying hostPort監聽;所以在你的情況下，它將是hostPort: 5000右下containerPort: 5000，導致Pod行爲就像一個正常的docker -p 5000:5000命令會。但是這樣做會讓Pods成爲一場噩夢，所以請不要這樣做。

其次，你的404 curl：

我要去承擔，根據你的curl命令的輸出10.233.69.x是您的服務CIDR，這就解釋了爲什麼5000端口與任何迴應。請求是正確的，但/v1/是一個錯誤的URI嘗試。 Docker Registry API docs包含約checking it is a V2 API instance部分。我最喜歡的curl註冊表是https://registry.example.com/v2/_catalog，因爲它會返回其中每個圖像的名稱，確保我的憑據正確，註冊表服務器正常運行，等等。

我知道這需要很多東西，所以如果你覺得我掩蓋了某些東西，請告訴我，我會盡力解決它。祝你好運！