9๋‹จ๊ณ„: Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ๋…ธ์ถœ

์ด ๋‹จ๊ณ„์—์„œ๋Š” Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด์— ๋Œ€ํ•ด Kubernetes ์„œ๋น„์Šค๋ฅผ ์„ค์ •ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค. Kubernetes ์„œ๋น„์Šค๋Š” ๋…ธ์ถœ๋  ์ˆ˜ ์žˆ๋Š” IP ์ฃผ์†Œ๋ฅผ ์–ป๊ธฐ ์œ„ํ•ด ์ธ๊ทธ๋ ˆ์Šค ๋ฐฐํฌ๋งˆ๋‹ค ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ํ”„๋ก์‹œ์— ๋Œ€ํ•œ ํด๋ผ์ด์–ธํŠธ ํ˜ธ์ถœ์€ ์ด IP ์ฃผ์†Œ๋กœ ํ™•์ธํ•  ํ˜ธ์ŠคํŠธ ์ด๋ฆ„์„ ํ˜ธ์ถœํ•ฉ๋‹ˆ๋‹ค.

Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด์˜ Kubernetes ์„œ๋น„์Šค ์˜ต์…˜

IP ์ฃผ์†Œ๋ฅผ ํ• ๋‹นํ•˜๊ธฐ ์œ„ํ•ด Kubernetes ์„œ๋น„์Šค๋ฅผ ์ œ๊ณตํ•˜๋Š” ๊ฒฝ์šฐ ๋‘ ๊ฐ€์ง€ ์˜ต์…˜์ด ์žˆ์Šต๋‹ˆ๋‹ค.

์ž์ฒด Kubernetes ์„œ๋น„์Šค ๋งŒ๋“ค๊ธฐ

ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์˜ ๊ฒฝ์šฐ Apigee์—์„œ๋Š” ๊ฐ ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด์— ๋Œ€ํ•ด ์ปค์Šคํ…€ Kubernetes ์„œ๋น„์Šค๋ฅผ ๋งŒ๋“œ๋Š” ๊ฒƒ์„ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.

  • Apigee๋Š” ๊ธฐ๋ณธ ์„œ๋น„์Šค๋ฅผ ์‚ญ์ œํ•˜์ง€๋งŒ ์‚ญ์ œ ์‹œ ์ปค์Šคํ…€ Kubernetes ์„œ๋น„์Šค๋ฅผ ์‚ญ์ œํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ Apigee Hybrid๋ฅผ ๋‹ค์‹œ ์„ค์น˜ํ•  ๋•Œ IP ์ฃผ์†Œ๊ฐ€ ํ•ด์ œ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
  • EKS, AKS, OpenShift์™€ ๊ฐ™์ด Google Cloud์— ์—†๋Š” ํ”Œ๋žซํผ์˜ ๊ฒฝ์šฐ Kubernetes ์„œ๋น„์Šค๊ฐ€ ํด๋ผ์šฐ๋“œ ์ œ๊ณต์—…์ฒด์™€ ํ˜‘๋ ฅํ•˜๋„๋ก ๋งž์ถค์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. Apigee๋Š” ๊ธฐ๋ณธ Kubernetes ์„œ๋น„์Šค์— ๋Œ€ํ•œ ๋ชจ๋“  ๋งž์ถค์„ค์ •์„ ์ง€์›ํ•˜์ง€ ์•Š์œผ๋ฏ€๋กœ ๊ธฐ๋ณธ ์„œ๋น„์Šค๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค ์ปค์Šคํ…€ Kubernetes ์„œ๋น„์Šค๋ฅผ ๋งŒ๋“œ๋Š” ๊ฒƒ์ด ๋” ์ข‹์Šต๋‹ˆ๋‹ค.

๋‹ค์Œ ๋‹จ๊ณ„๋ฅผ ๋”ฐ๋ผ ํŠธ๋ž˜ํ”ฝ์„ ์„ค์ •ํ•˜๊ณ  ์ƒˆ ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋กœ ๋ผ์šฐํŒ…ํ•ฉ๋‹ˆ๋‹ค.

  1. ํ•„์ˆ˜ ํฌ๋“œ ์„ ํƒ๊ธฐ ๋ผ๋ฒจ(app, ingress_name, org)์„ ์‚ฌ์šฉํ•˜์—ฌ Kubernetes ์„œ๋น„์Šค๋ฅผ ๋งŒ๋“ญ๋‹ˆ๋‹ค. ์ด ๋ผ๋ฒจ์€ ์ด๋ฏธ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ํฌ๋“œ์— ์žˆ์Šต๋‹ˆ๋‹ค. ๋‹ค์Œ์„ ์˜ˆ์‹œ๋กœ ์‚ฌ์šฉํ•˜์—ฌ ์„œ๋น„์Šค ํŒŒ์ผ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค.
    apiVersion: v1
    kind: Service
    metadata:
      name: SERVICE_NAME
      namespace: apigee
    spec:
      ports:
      - name: status-port
        port: 15021
        protocol: TCP
        targetPort: 15021
      - name: https
        port: 443
        protocol: TCP
        targetPort: 8443
      selector:
        app: apigee-ingressgateway #required
        ingress_name: INGRESS_NAME
        org: ORG_NAME
      type: LoadBalancer
      loadBalancerIP: LOAD_BALANCER_IP
    • SERVICE_NAME์€ ์ด ์„œ๋น„์Šค๋ฅผ ์‹๋ณ„ํ•˜๋Š” ๋ฐ ์‚ฌ์šฉ๋˜๋Š” ์ด๋ฆ„์ž…๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด apigee-prod-1์ž…๋‹ˆ๋‹ค.
    • INGRESS_NAME์€ ์ด Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ๊ฒŒ์ดํŠธ์›จ์ด์˜ ์ด๋ฆ„์ž…๋‹ˆ๋‹ค. overrides.yaml ํŒŒ์ผ์—์„œ ingressGateways.name์— ์ œ๊ณตํ•œ ์ด๋ฆ„๊ณผ ์ผ์น˜ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๊ตฌ์„ฑ ์†์„ฑ ์ฐธ์กฐ์˜ ingressGateways๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.
    • ORG_NAME์€ Apigee ์กฐ์ง์˜ ์ด๋ฆ„์ž…๋‹ˆ๋‹ค. overrides.yaml ํŒŒ์ผ์—์„œ org์— ์ œ๊ณตํ•œ ์ด๋ฆ„๊ณผ ์ผ์น˜ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๊ตฌ์„ฑ ์†์„ฑ ์ฐธ์กฐ์˜ org๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.
    • LOAD_BALANCER_IP๋Š” ๋ถ€ํ•˜ ๋ถ„์‚ฐ๊ธฐ์˜ IP ์ฃผ์†Œ์ž…๋‹ˆ๋‹ค.

    Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋Š” ๋‹ค์Œ ํฌํŠธ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

    ํฌํŠธ ์„ค๋ช…
    443 ๋Ÿฐํƒ€์ž„ ํŠธ๋ž˜ํ”ฝ
    15021 ์ƒํƒœ ์ ๊ฒ€ status-port๋Š” GKE ์ธ๊ทธ๋ ˆ์Šค ์ƒํƒœ ์ ๊ฒ€์— ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” /healthz/ready ์—”๋“œํฌ์ธํŠธ๋ฅผ ๋…ธ์ถœํ•ฉ๋‹ˆ๋‹ค.
  2. SERVICE_FILENAME.yaml์„ ์ ์šฉํ•˜์—ฌ ์„œ๋น„์Šค๋ฅผ ๋งŒ๋“ญ๋‹ˆ๋‹ค.
    kubectl apply -f SERVICE_FILENAME.yaml
  3. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด์˜ ์™ธ๋ถ€ IP๋ฅผ ์ฐพ์Šต๋‹ˆ๋‹ค.
    kubectl get svc -n apigee SERVICE_NAME

    ๋‹ค์Œ๊ณผ ๋น„์Šทํ•œ ๊ฒฐ๊ณผ๊ฐ€ ์ถœ๋ ฅ๋ฉ๋‹ˆ๋‹ค.

    NAME                                        TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                                      AGE
    apigee-ingressgateway-prod-hybrid-37a39bd   LoadBalancer   192.0.2.123   233.252.0.123   15021:32049/TCP,80:31624/TCP,443:30723/TCP   16h
  4. ๊ธฐ๋ณธ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ์„œ๋น„์Šค์— ๋Œ€ํ•ด ๋ถ€ํ•˜ ๋ถ„์‚ฐ๊ธฐ๋ฅผ ์‚ฌ์šฉ ์ค‘์ง€ํ•ฉ๋‹ˆ๋‹ค.
    1. ์žฌ์ •์˜ ํŒŒ์ผ์—์„œ ingressGateways[].svcType ์†์„ฑ์„ ClusterIP๋กœ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.
      ingressGateways:
        svcType: ClusterIP
    2. apigeectl apply --org๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ณ€๊ฒฝ์‚ฌํ•ญ์„ ์ ์šฉํ•ฉ๋‹ˆ๋‹ค.
      ${APIGEECTL_HOME}/apigeectl apply -f ${HYBRID_FILES}/overrides/overrides.yaml

    ์ž์„ธํ•œ ๋‚ด์šฉ์€ ๊ธฐ๋ณธ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ์„œ๋น„์Šค์— ๋Œ€ํ•ด loadbalancer ์‚ฌ์šฉ ์ค‘์ง€๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

  5. ์ƒํƒœ ์ ๊ฒ€ ํ˜ธ์ถœ์„ ์ˆ˜ํ–‰ํ•˜์—ฌ ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋ฅผ ํ…Œ์ŠคํŠธํ•ฉ๋‹ˆ๋‹ค.

    curl -H 'User-Agent: GoogleHC/' https://DOMAIN/healthz/ingress -k \
      --resolve "DOMAIN:443:INGRESS_IP_ADDRESS"

    ๊ฐ ํ•ญ๋ชฉ์˜ ์˜๋ฏธ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

    ์„ฑ๊ณตํ•˜๋ฉด ๋ช…๋ น์–ด์—์„œ ๋‹ค์Œ์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.

    Apigee Ingress is healthy
  6. ์ด IP ์ฃผ์†Œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋“ฑ๋ก๊ธฐ๊ด€ ๋˜๋Š” DNS ์ œ๊ณต์—…์ฒด์—์„œ DNS ๋ ˆ์ฝ”๋“œ(์ผ๋ฐ˜์ ์œผ๋กœ A ๋˜๋Š” CNAME ๋ ˆ์ฝ”๋“œ)๋ฅผ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.

๊ธฐ๋ณธ Kubernetes ์„œ๋น„์Šค ์‚ฌ์šฉ

๋น„ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์ด๋‚˜ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋ฅผ ํ†ตํ•ด ์ดˆ๊ธฐ ํŠธ๋ž˜ํ”ฝ์„ ํ…Œ์ŠคํŠธํ•˜๋ ค๋Š” ๊ฒฝ์šฐ์—๋Š” Apigee Hybrid๊ฐ€ ๊ฐ ์ธ๊ทธ๋ ˆ์Šค ๋ฐฐํฌ์— ๋Œ€ํ•ด ๊ธฐ๋ณธ Kubernetes ์„œ๋น„์Šค๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

overrides.yaml ํŒŒ์ผ์—์„œ ๊ธฐ๋ณธ ์„œ๋น„์Šค์— ๋Œ€ํ•œ ๊ตฌ์„ฑ์„ ์ œํ•œ์ ์œผ๋กœ ๋ณ€๊ฒฝํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๊ตฌ์„ฑ ์˜ต์…˜์€ Apigee ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด ๊ด€๋ฆฌ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”. ์˜ˆ๋ฅผ ๋“ค์–ด ์ฃผ์„์„ ์ถ”๊ฐ€ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์˜ ๊ฒฝ์šฐ ์ธ๊ทธ๋ ˆ์Šค์šฉ Kubernetes ์„œ๋น„์Šค๋ฅผ ์ œ๊ณตํ•˜๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค. ์ž์ฒด Kubernetes ์„œ๋น„์Šค ๋งŒ๋“ค๊ธฐ์˜ ๋‹จ๊ณ„๋ฅผ ๋”ฐ๋ฅด์„ธ์š”.

  1. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ๊ธฐ๋ณธ Apigee ์ธ๊ทธ๋ ˆ์Šค ์„œ๋น„์Šค์˜ ์™ธ๋ถ€ IP๋ฅผ ์ฐพ์Šต๋‹ˆ๋‹ค.
    kubectl get svc -n apigee -l app=apigee-ingressgateway

    ๋‹ค์Œ๊ณผ ๋น„์Šทํ•œ ๊ฒฐ๊ณผ๊ฐ€ ์ถœ๋ ฅ๋ฉ๋‹ˆ๋‹ค.

    NAME                                        TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                                      AGE
    apigee-ingressgateway-prod-hybrid-37a39bd   LoadBalancer   192.0.2.123   233.252.0.123   15021:32049/TCP,80:31624/TCP,443:30723/TCP   16h
  2. ์ƒํƒœ ์ ๊ฒ€ ํ˜ธ์ถœ์„ ์ˆ˜ํ–‰ํ•˜์—ฌ ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋ฅผ ํ…Œ์ŠคํŠธํ•ฉ๋‹ˆ๋‹ค.

    curl -H 'User-Agent: GoogleHC/' https://DOMAIN/healthz/ingress -k \
      --resolve "DOMAIN:443:INGRESS_IP_ADDRESS"

    ๊ฐ ํ•ญ๋ชฉ์˜ ์˜๋ฏธ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

    ์„ฑ๊ณตํ•˜๋ฉด ๋ช…๋ น์–ด์—์„œ ๋‹ค์Œ์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค.

    Apigee Ingress is healthy
  3. ์ด IP ์ฃผ์†Œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋“ฑ๋ก๊ธฐ๊ด€ ๋˜๋Š” DNS ์ œ๊ณต์—…์ฒด์—์„œ DNS ๋ ˆ์ฝ”๋“œ(์ผ๋ฐ˜์ ์œผ๋กœ A ๋˜๋Š” CNAME ๋ ˆ์ฝ”๋“œ)๋ฅผ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.
1 2 3 4 5 6 7 8 9 (๋‹ค์Œ) 10๋‹จ๊ณ„: ํ”„๋ก์‹œ ๋ฐฐํฌ