Apigee Hybrid๋ฅผ ๋ฒ„์ „ 1.3.6์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œ

๋ฒ„์ „ 1.3.6์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œ ๊ฐœ์š”

Apigee Hybrid ์—…๊ทธ๋ ˆ์ด๋“œ ์ ˆ์ฐจ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์„น์…˜์œผ๋กœ ์ •๋ฆฌ๋ฉ๋‹ˆ๋‹ค.

  1. ์ค€๋น„
    1. ์„œ๋น„์Šค ๊ณ„์ • ๋งŒ๋“ค๊ธฐ ๋ฐ ์—…๋ฐ์ดํŠธ
    2. ํ™˜๊ฒฝ ๊ทธ๋ฃน ๊ณ„ํš
    3. ์žฌ์ •์˜ ํŒŒ์ผ ๋ณต์‚ฌ ๋ฐ ์—…๋ฐ์ดํŠธ
  2. Istio ๋ฐ cert-manager ์—…๊ทธ๋ ˆ์ด๋“œ
  3. Hybrid ๋Ÿฐํƒ€์ž„ ๋ฒ„์ „ 1.3 ์„ค์น˜
  4. ์‚ญ์ œ

์„ ํ–‰ ์กฐ๊ฑด

์ค€๋น„

  1. (๊ถŒ์žฅ์‚ฌํ•ญ) ๋ฒ„์ „ 1.2 $APIGEECTL_HOME/ ๋””๋ ‰ํ„ฐ๋ฆฌ์˜ ๋ฐฑ์—… ์‚ฌ๋ณธ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
    tar -czvf $APIGEECTL_HOME/../apigeectl-v1.2-backup.tar.gz $APIGEECTL_HOME
  2. (๊ถŒ์žฅ์‚ฌํ•ญ) Cassandra ๋ฐฑ์—… ๋ฐ ๋ณต๊ตฌ์˜ ์•ˆ๋‚ด์— ๋”ฐ๋ผ Cassandra ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค๋ฅผ ๋ฐฑ์—…ํ•ฉ๋‹ˆ๋‹ค.
  3. Kubernetes ํ”Œ๋žซํผ์„ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์—…๊ทธ๋ ˆ์ด๋“œํ•ฉ๋‹ˆ๋‹ค. ๋„์›€์ด ํ•„์š”ํ•˜๋ฉด ํ”Œ๋žซํผ ๋ฌธ์„œ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.
    ํ”Œ๋žซํผ ์—…๊ทธ๋ ˆ์ด๋“œํ•  ๋ฒ„์ „
    GKE 1.15.x
    Anthos 1.5
    AKS 1.16.x(Anthos ์—ฐ๊ฒฐ ํด๋Ÿฌ์Šคํ„ฐ ์‚ฌ์šฉ)
  4. Hybrid ์„ค์น˜์—์„œ Apigee Connect๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š๋Š” ๊ฒฝ์šฐ Apigee Connect๋ฅผ ์‚ฌ์šฉ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
    1. Apigee Connect API๊ฐ€ ์‚ฌ์šฉ ์„ค์ •๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
      gcloud services list | grep apigeeconnect
      
      apigeeconnect.googleapis.com         Apigee Connect API
    2. ์‚ฌ์šฉ ์„ค์ •๋˜์ง€ ์•Š์•˜๋‹ค๋ฉด Apigee Connect API๋ฅผ ์‚ฌ์šฉ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
      gcloud services enable apigeeconnect.googleapis.com --project $PROJECT_ID

      ์—ฌ๊ธฐ์„œ $PROJECT_ID๋Š” Google Cloud ํ”„๋กœ์ ํŠธ ID์ž…๋‹ˆ๋‹ค.

    3. ๋‹ค์Œ ์˜ˆ์‹œ์™€ ๊ฐ™์ด ๋ช…๋ น์ค„์—์„œ gcloud ์ธ์ฆ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ๊ฐ€์ ธ์˜ต๋‹ˆ๋‹ค.

      TOKEN=$(gcloud auth print-access-token)

      ํ† ํฐ์ด ์ž…๋ ฅ๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•˜๋ ค๋ฉด ๋‹ค์Œ ์˜ˆ์‹œ์™€ ๊ฐ™์ด echo๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

      echo $TOKEN

      ๊ทธ๋Ÿฌ๋ฉด ํ† ํฐ์ด ์ธ์ฝ”๋”ฉ๋œ ๋ฌธ์ž์—ด๋กœ ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค.

      ์ž์„ธํ•œ ๋‚ด์šฉ์€ gcloud ๋ช…๋ น์ค„ ๋„๊ตฌ ๊ฐœ์š”๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

    4. ์กฐ์ง์—์„œ Apigee Connect๊ฐ€ ์‚ฌ์šฉ ์„ค์ •๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
      curl -H "Authorization: Bearer $TOKEN" \
        "https://apigee.googleapis.com/v1/organizations/$ORG_NAME"

      ์—ฌ๊ธฐ์„œ $ORG_NAME์€ ์กฐ์ง์˜ ID์ž…๋‹ˆ๋‹ค.

      ์ถœ๋ ฅ์— ๋‹ค์Œ์ด ํฌํ•จ๋˜์—ˆ๋‹ค๋ฉด

            "name" : "features.mart.connect.enabled",
            "value" : "true"

      Apigee Connect๊ฐ€ ์‚ฌ์šฉ ์„ค์ •๋œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.

    5. Apigee Connect๊ฐ€ ์‚ฌ์šฉ ์„ค์ •๋˜์ง€ ์•Š์€ ๊ฒฝ์šฐ Apigee Connect ์—์ด์ „ํŠธ ์—ญํ• ์„ MART ์„œ๋น„์Šค ๊ณ„์ •์— ํ• ๋‹นํ•ฉ๋‹ˆ๋‹ค.
      gcloud projects add-iam-policy-binding $PROJECT_ID \
        --member serviceAccount:apigee-mart@$PROJECT_ID.iam.gserviceaccount.com \
        --role roles/apigeeconnect.Agent
    6. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Apigee Connect๋ฅผ ์‚ฌ์šฉ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
      curl -H "Authorization: Bearer $TOKEN" -X PUT \
        -H "Content-Type: application/json" \
        -d '{
          "name" : "'"$ORG_NAME"'",
          "properties" : {
            "property" : [ {
              "name" : "features.hybrid.enabled",
              "value" : "true"
            }, {
              "name" : "features.mart.connect.enabled",
              "value" : "true"
            } ]
          }
        }' \
        "https://apigee.googleapis.com/v1/organizations/$ORG_NAME"
      

      ์ถœ๋ ฅ์— ๋‹ค์Œ ๋‘ ์†์„ฑ์ด ํฌํ•จ๋˜๋ฉด Apigee Connect๊ฐ€ ์„ฑ๊ณต์ ์œผ๋กœ ์‚ฌ์šฉ ์„ค์ •๋œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.

            {
              "name": "features.mart.connect.enabled",
              "value": "true"
            },
            {
              "name": "features.hybrid.enabled",
              "value": "true"
            }
      
  5. apigee-watcher ์„œ๋น„์Šค ๊ณ„์ •์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค. Apigee Watcher๋Š” v1.3์—์„œ ๋„์ž…๋œ ์ƒˆ๋กœ์šด ์„œ๋น„์Šค ๊ณ„์ •์ž…๋‹ˆ๋‹ค. ์กฐ์ง ์ˆ˜์ค€์˜ ๋ณ€๊ฒฝ์‚ฌํ•ญ์— ๋Œ€ํ•œ ๋™๊ธฐํ™” ๋‹ด๋‹น์ž๋ฅผ ์‚ดํ”ผ๊ณ  ์ด๋Ÿฌํ•œ ๋ณ€๊ฒฝ์‚ฌํ•ญ์„ ์ ์šฉํ•˜์—ฌ Istio ์ธ๊ทธ๋ ˆ์Šค๋ฅผ ๊ตฌ์„ฑํ•ฉ๋‹ˆ๋‹ค.

    ๊ธฐ๋ณธ Hybrid ๋””๋ ‰ํ„ฐ๋ฆฌ์—์„œ ๋‹ค์Œ์„ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.

    ./tools/create-service-account apigee-watcher ./service-accounts
  6. Apigee ๋Ÿฐํƒ€์ž„ ์—์ด์ „ํŠธ ์—ญํ• ์„ ๊ฐ์‹œ์ž ์„œ๋น„์Šค ๊ณ„์ •์— ํ• ๋‹นํ•ฉ๋‹ˆ๋‹ค.
    gcloud projects add-iam-policy-binding $PROJECT_ID \
      --member serviceAccount:apigee-watcher@$PROJECT_ID.iam.gserviceaccount.com \
      --role roles/apigee.runtimeAgent

    ์—ฌ๊ธฐ์„œ PROJECT_ID๋Š” Google Cloud ํ”„๋กœ์ ํŠธ ID์ž…๋‹ˆ๋‹ค. ์„œ๋น„์Šค ๊ณ„์ • ์ด๋ฉ”์ผ ์ฃผ์†Œ๊ฐ€ ์ด ํŒจํ„ด๊ณผ ๋‹ค๋ฅธ ๊ฒฝ์šฐ ์ ์ ˆํ•˜๊ฒŒ ๋ณ€๊ฒฝํ•ฉ๋‹ˆ๋‹ค.

    ์ถœ๋ ฅ์—๋Š” ๋‹ค์Œ์„ ํฌํ•จํ•œ ๋ชจ๋“  ์„œ๋น„์Šค ๊ณ„์ •๊ณผ ์—ญํ• ์˜ ๋ชฉ๋ก์ด ํฌํ•จ๋˜์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

      ...
    - members:
      - serviceAccount:apigee-watcher@hybrid13rc5.iam.gserviceaccount.com
      role: roles/apigee.runtimeAgent
      ...
  7. ๋ผ์šฐํŒ…ํ•  ํ™˜๊ฒฝ ๊ทธ๋ฃน์„ ๊ณ„ํšํ•ฉ๋‹ˆ๋‹ค. Apigee Hybrid 1.3์€ routingRules ๋Œ€์‹  ํ™˜๊ฒฝ ๊ทธ๋ฃน์„ ์‚ฌ์šฉํ•˜์—ฌ ๊ธฐ๋ณธ ๊ฒฝ๋กœ ๋ผ์šฐํŒ…์„ ๊ด€๋ฆฌํ•ฉ๋‹ˆ๋‹ค. Hybrid ๊ตฌ์„ฑ์—์„œ routingRules๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ ๋ผ์šฐํŒ…์„ ๋ณต์ œํ•˜๊ธฐ ์œ„ํ•ด ํ™˜๊ฒฝ ๊ทธ๋ฃน์„ ์„ค๊ณ„ํ•ฉ๋‹ˆ๋‹ค.

    ํ™˜๊ฒฝ ๊ทธ๋ฃน์„ ํ•˜๋‚˜ ์ด์ƒ ๋งŒ๋“ค์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

    ํ™˜๊ฒฝ ๊ทธ๋ฃน ์ •๋ณด๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

  8. ์žฌ์ •์˜ ํŒŒ์ผ์„ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.
    1. ์žฌ์ •์˜ ํŒŒ์ผ์˜ ์‚ฌ๋ณธ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค.
    2. gcp ๋ฐ k8sCluster ์Šคํƒ ์ž๋ฅผ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.

      ๋‹ค์Œ ๊ตฌ์„ฑ ์†์„ฑ์€ Hybrid ๋ฒ„์ „ 1.3์—์„œ ๊ต์ฒด๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

      • gcpRegion์ด gcp:region์œผ๋กœ ๋ณ€๊ฒฝ
      • gcpProjectID์ด gcp:projectID์œผ๋กœ ๋ณ€๊ฒฝ
      • gcpProjectIDRuntime์ด gcp:gcpProjectIDRuntime์œผ๋กœ ๋ณ€๊ฒฝ
      • k8sClusterName์ด k8s:clusterName์œผ๋กœ ๋ณ€๊ฒฝ
      • k8sClusterRegion์ด k8s:clusterRegion์œผ๋กœ ๋ณ€๊ฒฝ

      ์˜ˆ๋ฅผ ๋“ค์–ด ๋‹ค์Œ ๊ตฌ์กฐ๋ฅผ

      gcpRegion: gcp region
      gcpProjectID: gcp project ID
      gcpProjectIDRuntime: gcp project ID
      
      k8sClusterName: name
      k8sClusterRegion: region

      ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋ฐ”๊ฟ‰๋‹ˆ๋‹ค.

      gcp:
       projectID: gcp project ID
       region: gcp region
       gcpProjectIDRuntime: gcp project ID # optional. This is only required if you
                                             # want logger/metrics data to be sent in
                                             # different gcp project.
      
      k8sCluster:
       name: gcp project ID
       region: gcp region
      
    3. ์žฌ์ •์˜ ํŒŒ์ผ์— ์ด๋ฏธ ๊ณ ์œ ํ•œ ์ธ์Šคํ„ด์Šค ์‹๋ณ„์ž๊ฐ€ ์—†๋Š” ๊ฒฝ์šฐ ํ•˜๋‚˜๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      # unique identifier for this installation. 63 chars length limit
      instanceID: ID

      ์—ฌ๊ธฐ์—์„œ ID๋Š” 'my-hybrid-131-installation' ๋˜๋Š” 'acmecorp-hybrid-131'๊ณผ ๊ฐ™์ด Hybrid ์„ค์น˜์— ๋Œ€ํ•œ ๊ณ ์œ  ์‹๋ณ„์ž์ž…๋‹ˆ๋‹ค.

    4. ๊ฐ์‹œ์ž(apigee-watcher) ์„œ๋น„์Šค ๊ณ„์ •์„ ์žฌ์ •์˜ ํŒŒ์ผ์— ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      # Note: the SA should have the "Apigee Runtime Agent" role
      watcher:
       serviceAccountPath: "service account file"
    5. ์ธก์ •ํ•ญ๋ชฉ(apigee-metrics) ์„œ๋น„์Šค ๊ณ„์ •์„ ์žฌ์ •์˜ ํŒŒ์ผ์— ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      metrics:
       serviceAccountPath: "service account file"
    6. virtualhosts: ์Šคํƒ ์ž๋ฅผ ์—…๋ฐ์ดํŠธํ•˜์—ฌ routingRules๋ฅผ ํ™˜๊ฒฝ ๊ทธ๋ฃน์œผ๋กœ ๋ฐ”๊ฟ‰๋‹ˆ๋‹ค.
      1. -name: ์ด๋ฆ„์„ ํ™˜๊ฒฝ ๊ทธ๋ฃน์˜ ์ด๋ฆ„์œผ๋กœ ๋ฐ”๊ฟ‰๋‹ˆ๋‹ค. ๊ฐ ํ™˜๊ฒฝ ๊ทธ๋ฃน์— ํ•˜๋‚˜์”ฉ ์—ฌ๋Ÿฌ ๊ฐœ์˜ ์ด๋ฆ„ ํ•ญ๋ชฉ์„ ํฌํ•จํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
      2. hostAliases:[] ์ด ์ค„์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      3. sslCertPath: ๋ฐ sslKeyPath: ํ•ญ๋ชฉ์„ ์œ ์ง€ํ•˜๊ฑฐ๋‚˜ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      4. ๋ชจ๋“  routingRules ํ•ญ๋ชฉ์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.

      ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ์ด

      virtualhosts:
        - name: default
          hostAliases:
            - "*.acme.com"
          sslCertPath: ./certs/keystore.pem
          sslKeyPath: ./certs/keystore.key
          routingRules:
            - paths:
              - /foo
              - /bar
            - env: my-environment

      ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋ฐ”๋€๋‹ˆ๋‹ค.

      virtualhosts:
        - name: example-env-group
          sslCertPath: ./certs/keystore.pem
          sslKeyPath: ./certs/keystore.key
    7. mart ๋ฐ connectAgent: ์Šคํƒ ์ž๋ฅผ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค.
      1. mart: ์•„๋ž˜์˜ hostAlias:, sslCertPath:, sslKeyPath: ํ•ญ๋ชฉ์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      2. connectAgent: ์Šคํƒ ์ž๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      3. connectAgent: ์•„๋ž˜์— serviceAccountPath: ํ•ญ๋ชฉ์„ ์ถ”๊ฐ€ํ•˜๊ณ  Apigee Connect Agent ์—ญํ• ์ด ํ• ๋‹น๋œ ์„œ๋น„์Šค ๊ณ„์ • ํŒŒ์ผ์˜ ๊ฒฝ๋กœ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค(์ผ๋ฐ˜์ ์œผ๋กœ MART ์„œ๋น„์Šค ๊ณ„์ •).

      ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ์ด

      mart:
        hostAlias: "mart.apigee-hybrid-docs.net"
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json
        sslCertPath: ./certs/fullchain.pem
        sslKeyPath: ./certs/privkey.key

      ๋‹ค์Œ๊ณผ ๊ฐ™์ด ๋ฐ”๋€๋‹ˆ๋‹ค.

      mart:
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json
      
      connectAgent:
        serviceAccountPath: ./service-accounts/hybrid-project-apigee-mart.json

Istio ๋ฐ cert-manager ์—…๊ทธ๋ ˆ์ด๋“œ

Apigee Hybrid ๋ฒ„์ „ 1.3์—์„œ๋Š” cert-manager v0.14.2๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ธ์ฆ์„œ๋ฅผ ๊ด€๋ฆฌ ๋ฐ ํ™•์ธํ•˜๊ณ , Anthos Service Mesh(ASM) ๋ฒ„์ „ 1.5.7 ์ด์ƒ๊ณผ ํ•จ๊ป˜ ์ œ๊ณต๋œ Istio ๋ฐฐํฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋Ÿฐํƒ€์ž„ ์ธ๊ทธ๋ ˆ์Šค ๊ฒŒ์ดํŠธ์›จ์ด๋ฅผ ์ƒ์„ฑ ๋ฐ ๊ด€๋ฆฌํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

Istio 1.4.6์„ ASM 1.5.7 ์ด์ƒ์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œ

  1. ๋‹ค์šดํƒ€์ž„์„ ์ตœ์†Œํ™”ํ•˜๊ธฐ ์œ„ํ•ด Istio ๋ฐฐํฌ์™€ HPA์—๋Š” ๊ฐ๊ฐ ๋ณต์ œ๋ณธ์ด 2๊ฐœ ์ด์ƒ ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ๋ณต์ œ๋ณธ ์ˆ˜๋ฅผ ๊ฒฐ์ •ํ•ฉ๋‹ˆ๋‹ค.
    kubectl -n istio-system get deployments # list of deployments
    kubectl -n istio-system get hpa # list of hpa
  2. ๋ณต์ œ๋ณธ์ด ํ•˜๋‚˜๋งŒ ์žˆ๋Š” ๊ฐ ๋ฐฐํฌ๋ฅผ ์ˆ˜์ •ํ•˜๊ณ  replicas:๋ฅผ 2๊ฐœ ์ด์ƒ์œผ๋กœ ๋Š˜๋ฆฝ๋‹ˆ๋‹ค.
    kubectl -n istio-system edit deployment name

    ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

    spec:
      progressDeadlineSeconds: 600
      replicas: 2
  3. ๋ณต์ œ๋ณธ์ด ํ•˜๋‚˜๋งŒ ์žˆ๋Š” ๊ฐ HPA๋ฅผ ์ˆ˜์ •ํ•˜๊ณ  minReplicas:๋ฅผ 2๊ฐœ ์ด์ƒ์œผ๋กœ ๋Š˜๋ฆฝ๋‹ˆ๋‹ค.
    kubectl -n istio-system edit hpa name

    ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

    spec:
      maxReplicas: 5
      minReplicas: 2
    
  4. ASM ๋‹ค์šด๋กœ๋“œ ๋ฐ ์„ค์น˜์˜ ์„ค์น˜ ์•ˆ๋‚ด์— ๋”ฐ๋ผ ASM์„ ๋‹ค์šด๋กœ๋“œ ๋ฐ ์„ค์น˜ํ•ฉ๋‹ˆ๋‹ค.
  5. ์„ค์น˜ ํ›„์—๋Š” ๋ฒ„์ „ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ 1.5.x๊ฐ€ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ์„ค์น˜๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
    ./bin/istioctl version
    
    client version: 1.5.8-asm.0
    apigee-mart-ingressgateway version:
    citadel version: 1.4.6
    galley version: 1.4.6
    ingressgateway version: 1.5.8-asm.0
    pilot version: 1.4.6
    policy version: 1.4.6
    sidecar-injector version: 1.4.6
    telemetry version: 1.4.6
    pilot version: 1.5.8-asm.0
    data plane version: 1.4.6 (1 proxies), 1.5.8-asm.0 (2 proxies)

cert-manager ์—…๊ทธ๋ ˆ์ด๋“œ

  1. ํ˜„์žฌ cert-manager ๋ฐฐํฌ๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
    kubectl delete -n cert-manager deployment cert-manager cert-manager-cainjector cert-manager-webhook
  2. Kubernetes ๋ฒ„์ „์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
    kubectl version
  3. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ Jetstack์—์„œ cert-manager๋ฅผ ์„ค์น˜ํ•ฉ๋‹ˆ๋‹ค.
    kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.2/cert-manager.yaml 

Hybrid ๋Ÿฐํƒ€์ž„ ์„ค์น˜

  1. ๋ณ€์ˆ˜์— ์ตœ์‹  ๋ฒ„์ „ ๋ฒˆํ˜ธ๋ฅผ ์ €์žฅํ•ฉ๋‹ˆ๋‹ค.
    export VERSION=$(curl -s \
        https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/current-version.txt?ignoreCache=1)
  2. ๋ณ€์ˆ˜๊ฐ€ ๋ฒ„์ „ ๋ฒˆํ˜ธ๋กœ ์ฑ„์›Œ์ ธ ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. ๋‹ค๋ฅธ ๋ฒ„์ „์„ ์‚ฌ์šฉํ•˜๋ ค๋ฉด ํ•ด๋‹น ๋ฒ„์ „์„ ๋Œ€์‹  ํ™˜๊ฒฝ ๋ณ€์ˆ˜์— ์ €์žฅํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
    echo $VERSION
      1.3.6
  3. ์‚ฌ์šฉ ์ค‘์ธ ์šด์˜์ฒด์ œ์— ํ•ด๋‹นํ•˜๋Š” ์ถœ์‹œ ํŒจํ‚ค์ง€๋ฅผ ๋‹ค์šด๋กœ๋“œํ•ฉ๋‹ˆ๋‹ค.

    Mac 64๋น„ํŠธ:

    curl -LO \
        https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_mac_64.tar.gz

    Linux 64๋น„ํŠธ:

    curl -LO \
        https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_linux_64.tar.gz

    Mac 32๋น„ํŠธ:

    curl -LO \
        https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_mac_32.tar.gz

    Linux 32๋น„ํŠธ:

    curl -LO \
        https://storage.googleapis.com/apigee-public/apigee-hybrid-setup/$VERSION/apigeectl_linux_32.tar.gz
  4. ํ˜„์žฌ apigeectl/ ๋””๋ ‰ํ„ฐ๋ฆฌ์˜ ์ด๋ฆ„์„ ๋ฐฑ์—… ๋””๋ ‰ํ„ฐ๋ฆฌ ์ด๋ฆ„์œผ๋กœ ๋ฐ”๊ฟ‰๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
    mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.2/ 
  5. ๋‹ค์šด๋กœ๋“œํ•œ gzip ํŒŒ์ผ ์ฝ˜ํ…์ธ ๋ฅผ Hybrid ๊ธฐ๋ณธ ๋””๋ ‰ํ„ฐ๋ฆฌ์— ์ถ”์ถœํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

    tar xvzf filename.tar.gz -C hybrid-base-directory
  6. cd ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ธฐ๋ณธ ๋””๋ ‰ํ„ฐ๋ฆฌ๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.
  7. tar ์ฝ˜ํ…์ธ ๋Š” ๊ธฐ๋ณธ์ ์œผ๋กœ ์ด๋ฆ„์— ํ•ด๋‹น ๋ฒ„์ „๊ณผ ํ”Œ๋žซํผ์ด ํฌํ•จ๋œ ๋””๋ ‰ํ„ฐ๋ฆฌ๋กœ ํ™•์žฅ๋ฉ๋‹ˆ๋‹ค(์˜ˆ: ./apigeectl_1.0.0-f7b96a8_linux_64). ์ด ๋””๋ ‰ํ„ฐ๋ฆฌ์˜ ์ด๋ฆ„์„ apigeectl๋กœ ๋ฐ”๊ฟ‰๋‹ˆ๋‹ค.

    mv apigeectl_1.0.0-f7b96a8_linux_64 apigeectl
  8. apigee-system์—์„œ apigee-resources-install ์ž‘์—…์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
    kubectl -n apigee-system delete job apigee-resources-install
  9. ์ด์ „ CRD๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
    kubectl delete crd apigeetelemetries.apigee.cloud.google.com
  10. externalSeedHost ์†์„ฑ์„ ์‚ฌ์šฉํ•˜์—ฌ ์žฌ์ •์˜ ํŒŒ์ผ์˜ cassandra: ์Šคํƒ ์ž๋ฅผ ์—…๋ฐ์ดํŠธํ•ฉ๋‹ˆ๋‹ค. ์ด ์†์„ฑ์€ ์ƒˆ Hybrid ๋ฒ„์ „ 1.3.6 ์„ค์น˜์—์„œ ๋ฒ„์ „ 1.2 ์„ค์น˜์™€ ๋™์ผํ•œ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ์‚ฌ์šฉํ•˜๋„๋ก ๋„์™€์ค๋‹ˆ๋‹ค. ์ด ๋‹จ๊ณ„๋Š” ์ผํšŒ์„ฑ์œผ๋กœ Hybrid ๋ฒ„์ „ 1.2์—์„œ ๋ฒ„์ „ 1.3.6 ์ด์ƒ์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๋Š” ๊ฒฝ์šฐ์—๋งŒ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
    1. 1.2.0 ์„ค์น˜๋ฅผ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๋Š” ๋™์ผํ•œ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ์—์„œ ๊ธฐ์กด Cassandra์˜ IP ์ฃผ์†Œ ์ค‘ ํ•˜๋‚˜๋ฅผ ์ฐพ์Šต๋‹ˆ๋‹ค.
      kubectl -n namespace get pods -o wide

      ์—ฌ๊ธฐ์„œ namespace๋Š” Apigee Hybrid ๋„ค์ž„์ŠคํŽ˜์ด์Šค์ž…๋‹ˆ๋‹ค.

      Cassandra ๋…ธ๋“œ์˜ IP ์ฃผ์†Œ๋ฅผ ๊ธฐ๋กํ•ด ๋‘ก๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

      kubectl -n apigee get pods -o wide
      NAME                  READY   STATUS    RESTARTS   AGE   IP          NODE
      apigee-cassandra-0    1/1     Running   0          33d   10.68.8.24   gke-example-cluster-rc5-apigee-data-c8bf1234-09kc
      apigee-cassandra-1    1/1     Running   0          16d   10.68.8.33   gke-example-cluster-rc5-apigee-data-c9221ee7-10kc
      apigee-cassandra-2    1/1     Running   0          23h   10.68.9.11   gke-example-cluster-rc5-apigee-data-d123e456-11kc
    2. externalSeedHost ์†์„ฑ ๊ฐ’์„ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
      cassandra:
       externalSeedHost: Cassandra_node_IP

      ์—ฌ๊ธฐ์„œ Cassandra_node_IP๋Š” cassandra ๋…ธ๋“œ์˜ IP์ž…๋‹ˆ๋‹ค(์•ž์˜ ์˜ˆ์‹œ์—์„œ 10.68.8.24).

  11. ์ƒˆ apigeectl/ ๋””๋ ‰ํ„ฐ๋ฆฌ์—์„œ apigeectl init, apigeectl apply, apigeectl check-ready๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.
    1. Hybrid 1.3.6์„ ์ดˆ๊ธฐํ™”ํ•ฉ๋‹ˆ๋‹ค.
      apigeectl init -f overrides_1.3.yaml

      ์—ฌ๊ธฐ์„œ overrides_1.3.yaml์€ ํŽธ์ง‘๋œ override.yaml ํŒŒ์ผ์ž…๋‹ˆ๋‹ค.

    2. Hybrid ๋ฒ„์ „ 1.3์—์„œ --dry-run ํ”Œ๋ž˜๊ทธ์˜ ๊ตฌ๋ฌธ์€ ์‹คํ–‰ ์ค‘์ธ kubectl ๋ฒ„์ „์— ๋”ฐ๋ผ ๋‹ค๋ฆ…๋‹ˆ๋‹ค. kubectl์˜ ๋ฒ„์ „์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
      gcloud version
    3. ํ…Œ์ŠคํŠธ ์‹คํ–‰์œผ๋กœ ์˜ค๋ฅ˜๊ฐ€ ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

      kubectl ๋ฒ„์ „ 1.17 ์ดํ•˜:

      apigeectl apply -f overrides_1.3.yaml --dry-run=true

      kubectl ๋ฒ„์ „ 1.18 ์ด์ƒ:

      apigeectl apply -f overrides_1.3.yaml --dry-run=client
    4. ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•ฉ๋‹ˆ๋‹ค. ์„ค์น˜์— ๋”ฐ๋ผ ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ ๋˜๋Š” ๋ฐ๋ชจ/์‹คํ—˜์šฉ ํ™˜๊ฒฝ์˜ ์•ˆ๋‚ด๋ฅผ ์„ ํƒํ•˜์—ฌ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.

      ํ”„๋กœ๋•์…˜

      ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์˜ ๊ฒฝ์šฐ ๊ฐ ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ๊ตฌ์„ฑ์š”์†Œ๋ฅผ ๊ฐœ๋ณ„์ ์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๊ณ  ๋‹ค์Œ ๊ตฌ์„ฑ์š”์†Œ๋กœ ์ง„ํ–‰ํ•˜๊ธฐ ์ „์— ์—…๊ทธ๋ ˆ์ด๋“œ๋œ ๊ตฌ์„ฑ์š”์†Œ์˜ ์ƒํƒœ๋ฅผ ํ™•์ธํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

      1. ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜์—ฌ Cassandra๋ฅผ ์—…๊ทธ๋ ˆ์ด๋“œํ•ฉ๋‹ˆ๋‹ค.
        apigeectl apply -f overrides_1.3.yaml --datastore
      2. ์™„๋ฃŒ๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
        kubectl -n namespace get pods

        ์—ฌ๊ธฐ์„œ namespace๋Š” Apigee Hybrid ๋„ค์ž„์ŠคํŽ˜์ด์Šค์ž…๋‹ˆ๋‹ค.

        ํฌ๋“œ๊ฐ€ ์ค€๋น„๋œ ๊ฒฝ์šฐ์—๋งŒ ๋‹ค์Œ ๋‹จ๊ณ„๋กœ ์ง„ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

      3. ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜์—ฌ ์›๊ฒฉ ๋ถ„์„ ๊ตฌ์„ฑ์š”์†Œ๋ฅผ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๊ณ  ์™„๋ฃŒ๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
        apigeectl apply -f overrides_1.3.yaml --telemetry
        kubectl -n namespace get pods
      4. ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜์—ฌ ์กฐ์ง ์ˆ˜์ค€ ๊ตฌ์„ฑ์š”์†Œ(MART, Watcher, Apigee Connect)๋ฅผ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๊ณ  ์™„๋ฃŒ๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
        apigeectl apply -f overrides_1.3.yaml --org
        kubectl -n namespace get pods
      5. ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜์—ฌ ํ™˜๊ฒฝ์„ ์—…๊ทธ๋ ˆ์ด๋“œํ•ฉ๋‹ˆ๋‹ค. ๋‹ค์Œ ๋‘ ๊ฐ€์ง€ ์ค‘์—์„œ ์„ ํƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
        • ํ•œ ๋ฒˆ์— ํ•˜๋‚˜์˜ ํ™˜๊ฒฝ์— ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜๊ณ  ์™„๋ฃŒ๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. ํ™˜๊ฒฝ๋งˆ๋‹ค ์ด ๋‹จ๊ณ„๋ฅผ ๋ฐ˜๋ณตํ•ฉ๋‹ˆ๋‹ค.
          apigeectl apply -f overrides_1.3.yaml --env env_name
          kubectl -n namespace get pods

          ์—ฌ๊ธฐ์„œ env_name์€ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๋Š” ํ™˜๊ฒฝ์˜ ์ด๋ฆ„์ž…๋‹ˆ๋‹ค.

        • ํ•œ ๋ฒˆ์— ๋ชจ๋“  ํ™˜๊ฒฝ์— ์žฌ์ •์˜๋ฅผ ์ ์šฉํ•˜๊ณ  ์™„๋ฃŒ ์—ฌ๋ถ€๋ฅผ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.
          apigeectl apply -f overrides_1.3.yaml --all-envs
          kubectl -n namespace get pods

      ๋ฐ๋ชจ/์‹คํ—˜์šฉ

      ๋Œ€๋ถ€๋ถ„์˜ ๋ฐ๋ชจ ๋˜๋Š” ์‹คํ—˜์šฉ ํ™˜๊ฒฝ์—์„œ๋Š” ๋ชจ๋“  ๊ตฌ์„ฑ์š”์†Œ์— ์žฌ์ •์˜๋ฅผ ํ•œ ๋ฒˆ์— ์ ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ฐ๋ชจ/์‹คํ—˜์šฉ ํ™˜๊ฒฝ์˜ ๊ทœ๋ชจ๊ฐ€ ํฌ๊ณ  ๋ณต์žกํ•˜๊ฑฐ๋‚˜ ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์„ ๋น„์Šทํ•˜๊ฒŒ ๋ชจ๋ฐฉํ•˜๋Š” ๊ฒฝ์šฐ ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ ์—…๊ทธ๋ ˆ์ด๋“œ ์•ˆ๋‚ด๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

      1. apigeectl apply -f overrides_1.3.yaml
      2. ์ƒํƒœ๋ฅผ ํŒŒ์•…ํ•ฉ๋‹ˆ๋‹ค.
        apigeectl check-ready -f overrides_1.3.yaml

      ์ž์„ธํ•œ ๋‚ด์šฉ์€ GKE Hybrid ์„ค์ • - 5๋‹จ๊ณ„: GKE์— Hybrid ์„ค์น˜๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

    5. Hybrid 1.3์„ ์„ค์ •ํ•˜๊ณ  ์‹คํ–‰ํ•œ ํ›„์—๋Š” ๋ชจ๋“  Cassandra ๋…ธ๋“œ(์ด์ „ ๋…ธ๋“œ ๋ฐ ์ƒˆ ๋…ธ๋“œ)๊ฐ€ ๋™์ผํ•œ Cassandra ํด๋Ÿฌ์Šคํ„ฐ์— ํฌํ•จ๋˜๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. Cassandra ๋…ธ๋“œ ์ค‘ ํ•˜๋‚˜์—์„œ ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.
      kubectl -n namespace get pods
      kubectl -n namespace exec old Cassandra pod -- nodetool status

      ๋‹ค์Œ ์ถœ๋ ฅ ์˜ˆ์‹œ์—์„œ 10.68.8.24๋Š” ๋ฒ„์ „ 1.2.0์—์„œ ๋น„๋กฏ๋œ ๊ฒƒ์ด๋ฉฐ externalSeedHost๋กœ ์‚ฌ์šฉํ•œ ๋…ธ๋“œ IP์ž…๋‹ˆ๋‹ค. 10.68.7.11์€ ๋ฒ„์ „ 1.3.6์—์„œ ๋น„๋กฏ๋œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.

      Datacenter: dc-1
      ================
      Status=Up/Down
      |/ State=Normal/Leaving/Joining/Moving
      --  Address     Load        Tokens       Owns (effective)  Host ID                               Rack
      UN  10.68.8.24  379.41 KiB  256          50.8%             11bbd43b-af64-464b-a96d-0d6dd0521de1  ra-1
      UN  10.68.7.11  1.35 MiB    256          49.2%             0b4d9e08-f353-413a-b4a9-7d18a8d07e58  ra-1

      ๊ฐ™์€ ํด๋Ÿฌ์Šคํ„ฐ์— ์žˆ์ง€ ์•Š์€ ๊ฒฝ์šฐ externalSeedHost ๊ฐ’์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

    6. ๋ชจ๋“  ํฌ๋“œ๊ฐ€ ์‹คํ–‰๋˜๋ฉด ์žฌ์ •์˜ ํŒŒ์ผ์—์„œ externalSeedHost๋ฅผ ์ œ๊ฑฐํ•˜๊ณ  --datastore ์˜ต์…˜์„ ์‚ฌ์šฉํ•˜์—ฌ apigeectl apply๋ฅผ ๋‹ค์‹œ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.
      apigeectl apply --datastore -f overrides_1.3.6.yaml

    ์‚ญ์ œ

    ๋ชจ๋“  ํฌ๋“œ๊ฐ€ ์ •์ƒ์ด๊ณ  ASM ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์ƒˆ ์„ค์น˜์— ์œ ํšจํ•จ์„ ํ™•์ธํ•œ ํ›„ ์‚ญ์ œํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

    • Hybrid 1.2 ๋ฆฌ์†Œ์Šค
    • ์ด์ „ Cassandra ์ธ์Šคํ„ด์Šค
    • Istio 1.4.6 ๋ฆฌ์†Œ์Šค

    Hybrid 1.2.0 ๋ฆฌ์†Œ์Šค ์‚ญ์ œ

    1. 1.2.0 virtualHost ๋ผ์šฐํŒ… ์„ธ๋ถ€์ •๋ณด๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      $APIGEECTL_HOME-v1.2/apigeectl delete -s virtualhost -f 1.2.0_overrides.yaml

      ์—ฌ๊ธฐ์„œ $APIGEECTL_HOME-v1.2๋Š” apigeectl ๋ฒ„์ „ 1.2 ๋””๋ ‰ํ„ฐ๋ฆฌ๋ฅผ ๋ฐฑ์—…ํ•œ ๋””๋ ‰ํ„ฐ๋ฆฌ์ž…๋‹ˆ๋‹ค.

    2. ์—”๋“œํฌ์ธํŠธ๊ฐ€ ๊ณ„์† ์ •์ƒ์ ์œผ๋กœ ์ž‘๋™ํ•˜๊ณ  ์žˆ๊ณ  1.3.0 ๊ตฌ์„ฑ์š”์†Œ๊ฐ€ ๋ชจ๋‘ ์ž‘๋™ ์ค‘์ž„์„ ํ™•์ธํ•œ ๊ฒฝ์šฐ ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ Hybrid 1.2.0 ๋ฆฌ์†Œ์Šค๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      $APIGEECTL_HOME-v1.2/apigeectl delete -c "mart,connect-agent,synchronizer,runtime,udca,metrics,logger" \
        -f 1.2.0_overrides.yaml

    ์ด์ „ Cassandra ์ธ์Šคํ„ด์Šค ์‚ฌ์šฉ ์ค‘๋‹จ

    1. cd๋ฅผ ์‚ฌ์šฉํ•ด ์ƒˆ๋กœ ์„ค์น˜๋œ apigeectl ๋””๋ ‰ํ„ฐ๋ฆฌ๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.
    2. tools/cas_cleanup.sh ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.

      ์ด ์Šคํฌ๋ฆฝํŠธ๋Š” Cassandra ๋ง์—์„œ ์ด์ „ Cassandra ํฌ๋“œ๋ฅผ ์‚ฌ์šฉ ์ค‘์ง€ํ•˜๊ณ  ์ด์ „ STS๋ฅผ ์‚ญ์ œํ•˜๋ฉฐ PVC๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.

      bash cas_cleanup.sh Apigee namespace

    Istio ๋ฒ„์ „ 1.4.6 ๋ฆฌ์†Œ์Šค ์‚ญ์ œ

    1. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ๊ฐ€์žฅ ์ตœ๊ทผ์˜ Istio v.1.4.6 ๋ฆฌ์†Œ์Šค๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      kubectl delete all -n istio-system --selector \
        'app in (apigee-mart-istio-ingressgateway, galley, security, istio-nodeagent, istio-mixer, sidecarInjectorWebhook, istio-mixer)'
    2. ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ Istio 1.4.6 ์„ค์น˜์—์„œ ์ด์ „ ์ž‘์—…์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.
      kubectl -n istio-system delete job istio-init-crd-10-1.4.6
      kubectl -n istio-system delete job istio-init-crd-11-1.4.6
      kubectl -n istio-system delete job istio-init-crd-14-1.4.6

    ์ˆ˜๊ณ ํ•˜์…จ์Šต๋‹ˆ๋‹ค. Apigee Hybrid ๋ฒ„์ „ 1.3.6์œผ๋กœ ์„ฑ๊ณต์ ์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.