Cassandra ๅ‚™ไปฝ่ˆ‡ๅพฉๅŽŸ

ๆœฌ็ฏ€ๅฐ‡่ชชๆ˜Žๅฆ‚ไฝ•็‚บๅœจ Apigee ๆททๅˆๅผๅŸท่กŒ้šŽๆฎตๅนณ้ขไธญๅฎ‰่ฃ็š„ Apache Cassandra ่ณ‡ๆ–™ๅบซ็’ฐ๏ผŒ่จญๅฎš่ณ‡ๆ–™ๅ‚™ไปฝๅ’ŒๅพฉๅŽŸๅŠŸ่ƒฝใ€‚ๅฆ่ซ‹ๅƒ้–ฑ Cassandra ่ณ‡ๆ–™ๅ„ฒๅญ˜ๅบซใ€‚

้—œๆ–ผ Cassandra ๅ‚™ไปฝ็š„ๆณจๆ„ไบ‹้ …

Cassandra ๆ˜ฏ่ค‡่ฃฝ่ณ‡ๆ–™ๅบซ๏ผŒๅฏๅœจๆฏๅ€‹ๅ€ๅŸŸๆˆ–่ณ‡ๆ–™ไธญๅฟƒไธญ่‡ณๅฐ‘่ค‡่ฃฝไธ‰ไปฝ่ณ‡ๆ–™ใ€‚Cassandra ๆœƒไฝฟ็”จไธฒๆต่ค‡่ฃฝๅ’Œ่ฎ€ๅ–ไฟฎๅพฉๅŠŸ่ƒฝ๏ผŒๅœจไปปไฝ•ๆ™‚้–“้ปž็ถญ่ญทๅ„ๅ€‹ๅ€ๅŸŸๆˆ–่ณ‡ๆ–™ไธญๅฟƒ็š„่ณ‡ๆ–™ๅ‰ฏๆœฌใ€‚

ๅœจๆททๅˆๅผๆžถๆง‹ไธญ๏ผŒCassandra ๅ‚™ไปฝ้ ่จญ็‚บใ€Œๆœชๅ•Ÿ็”จใ€ใ€‚ไธ้Ž๏ผŒๅปบ่ญฐๆ‚จๅ•Ÿ็”จ Cassandra ๅ‚™ไปฝๅŠŸ่ƒฝ๏ผŒไปฅ้˜ฒ่ณ‡ๆ–™ๆ„ๅค–ๅˆช้™คใ€‚

็ณป็ตฑๆœƒๅ‚™ไปฝๅ“ชไบ›ๅ…งๅฎน๏ผŸ

ๆœฌไธป้กŒๆ‰€่ฟฐ็š„ๅ‚™ไปฝ่จญๅฎšๆœƒๅ‚™ไปฝไธ‹ๅˆ—ๅฏฆ้ซ”๏ผš

  • Cassandra ็ตๆง‹ๅฎš็พฉ๏ผŒๅŒ…ๆ‹ฌไฝฟ็”จ่€…็ตๆง‹ๅฎš็พฉ (Apigee ้ตๅ€ผ็ฉบ้–“ๅฎš็พฉ)
  • ๆฏๅ€‹็ฏ€้ปž็š„ Cassandra ๅˆ†ๅ€็ฌฆ่จ˜่ณ‡่จŠ
  • Cassandra ่ณ‡ๆ–™็š„ๅฟซ็…ง

ๅ‚™ไปฝ่ณ‡ๆ–™ๅ„ฒๅญ˜ๅœจไฝ•่™•๏ผŸ

ๅ‚™ไปฝ็š„่ณ‡ๆ–™ๆœƒๅ„ฒๅญ˜ๅœจๆ‚จๅฟ…้ ˆๅปบ็ซ‹็š„ Google Cloud Storage ๅ€ผๅ€ไธญใ€‚ๆœฌไธป้กŒๅฐ‡่ชชๆ˜Žๅฆ‚ไฝ•ๅปบ็ซ‹ๅŠ่จญๅฎš bucketใ€‚

ๆŽ’ๅฎš Cassandra ๅ‚™ไปฝๆ™‚้–“

ๅ‚™ไปฝๆœƒๅœจๅŸท่กŒๆ™‚้–“ๅนณ้ขไธญๆŽ’็จ‹็‚บ cron ๅทฅไฝœใ€‚ๅฆ‚่ฆๆŽ’ๅฎš Cassandra ๅ‚™ไปฝไฝœๆฅญๆ™‚้–“๏ผŒ่ซ‹ๆŒ‰็…งไธ‹ๅˆ—ๆญฅ้ฉŸๆ“ไฝœ๏ผš

  1. ๅŸท่กŒไธ‹ๅˆ— create-service-account ๆŒ‡ไปค๏ผŒๅปบ็ซ‹ๅ…ทๆœ‰ๆจ™ๆบ– roles/storage.objectAdmin ่ง’่‰ฒ็š„ Google Cloud ๆœๅ‹™ๅธณๆˆถ (SA)ใ€‚้€™ๅ€‹ SA ่ง’่‰ฒๅฏ่ฎ“ๆ‚จๅฐ‡ๅ‚™ไปฝ่ณ‡ๆ–™ๅฏซๅ…ฅ Cloud Storageใ€‚ๅœจๆททๅˆๅผๅฎ‰่ฃๆ น็›ฎ้Œ„ไธญๅŸท่กŒไธ‹ๅˆ—ๆŒ‡ไปค๏ผš
    ./tools/create-service-account apigee-cassandra OUTPUT_DIR
    ไพ‹ๅฆ‚๏ผš
    ./tools/create-service-account apigee-cassandra ./service-accounts
    ๅฆ‚่ฆ้€ฒไธ€ๆญฅ็žญ่งฃ Google Cloud ๆœๅ‹™ๅธณๆˆถ๏ผŒ่ซ‹ๅƒ้–ฑใ€Œๅปบ็ซ‹ๅŠ็ฎก็†ๆœๅ‹™ๅธณๆˆถใ€ใ€‚
  2. create-service-account ๆŒ‡ไปคๆœƒๅ„ฒๅญ˜ๅซๆœ‰ๆœๅ‹™ๅธณๆˆถ็งๅฏ†้‡‘้‘ฐ็š„ JSON ๆช”ๆกˆใ€‚ๆช”ๆกˆๆœƒๅ„ฒๅญ˜ๅœจๅŸท่กŒๆŒ‡ไปคๆ‰€ๅœจ็š„็›ฎ้Œ„ไธญใ€‚ๆ‚จๅฐ‡ๅœจๅพŒ็บŒๆญฅ้ฉŸไธญ้œ€่ฆ้€™ๅ€‹ๆช”ๆกˆ็š„่ทฏๅพ‘ใ€‚
  3. ๅปบ็ซ‹ Cloud Storage ๅ€ผๅ€ใ€‚ ็‚บๅ€ผๅ€ๆŒ‡ๅฎšๅˆ็†็š„่ณ‡ๆ–™ ไฟ็•™ๆ”ฟ็ญ–ใ€‚Apigee ๅปบ่ญฐ็š„่ณ‡ๆ–™ไฟ็•™ๆ”ฟ็ญ–็‚บ 15 ๅคฉใ€‚
  4. ้–‹ๅ•Ÿ overrides.yaml ๆช”ๆกˆใ€‚
  5. ๆ–ฐๅขžไธ‹ๅˆ— cassandra.backup ๅฑฌๆ€งๅณๅฏๅ•Ÿ็”จๅ‚™ไปฝๅŠŸ่ƒฝใ€‚่ซ‹ๅ‹ฟ็งป้™คไปปไฝ•ๅทฒ่จญๅฎš็š„ๅฑฌๆ€งใ€‚

    ๅƒๆ•ธ

    cassandra:
      ...
    
      backup:
        enabled: true
        serviceAccountPath: SA_JSON_FILE_PATH
        dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH
        schedule: BACKUP_SCHEDULE_CODE
    
      ...
      

    ็ฏ„ไพ‹

    ...
    
    cassandra:
      storage:
        type: gcepd
        capacity: 50Gi
        gcepd:
          replicationType: regional-pd
      sslRootCAPath: "/Users/myhome/ssh/cassandra.crt"
      sslCertPath: "/Users/myhome/ssh/cassandra.crt"
      sslKeyPath: "/Users/myhome/ssh/cassandra.key"
      auth:
        default:
          password: "abc123"
        admin:
          password: "abc234"
        ddl:
          password: "abc345"
        dml:
          password: "abc456"
      nodeSelector:
        key: cloud.google.com/gke-nodepool
        value: apigee-data
      backup:
        enabled: true
        serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json"
        dbStorageBucket: "gs://myname-cassandra-backup"
        schedule: "45 23 * * 6"
    
      ... 
  6. ๅ…ถไธญ๏ผš
    ๅฑฌๆ€ง ่ชชๆ˜Ž
    backup:enabled ๅ‚™ไปฝๅŠŸ่ƒฝ้ ่จญ็‚บๅœ็”จใ€‚ๆ‚จๅฟ…้ ˆๅฐ‡้€™้ …ๅฑฌๆ€ง่จญ็‚บ trueใ€‚
    backup:serviceAccountPath

    SA_JSON_FILE_PATH

    ๆ‚จๅœจๅŸท่กŒ ./tools/create-service-account ๆ™‚ไธ‹่ผ‰็š„ๆœๅ‹™ๅธณๆˆถ JSON ๆช”ๆกˆๅœจๆช”ๆกˆ็ณป็ตฑไธญ็š„่ทฏๅพ‘

    backup:dbStorageBucket

    CLOUD_STORAGE_BUCKET_PATH

    Cloud Storage ๅ€ผๅ€่ทฏๅพ‘็š„ๆ ผๅผ็‚บ gs://BUCKET_NAMEใ€‚ๅฟ…้ ˆไฝฟ็”จ gs://ใ€‚

    backup:schedule

    BACKUP_SCHEDULE_CODE

    ๅ‚™ไปฝไฝœๆฅญ้–‹ๅง‹็š„ๆ™‚้–“๏ผŒไปฅ ๆจ™ๆบ– crontab ่ชžๆณ•ๆŒ‡ๅฎšใ€‚้ ่จญๅ€ผ๏ผš0 2 * * *

  7. ๅฐ‡่จญๅฎš่ฎŠๆ›ดๅฅ—็”จ่‡ณๆ–ฐๅข้›†ใ€‚ไพ‹ๅฆ‚๏ผš
    ./apigeectl apply -f overrides.yaml

้‚„ๅŽŸๅ‚™ไปฝ

้‚„ๅŽŸไฝœๆฅญๆœƒๅพžๅ‚™ไปฝไฝ็ฝฎๅ–ๅพ—่ณ‡ๆ–™๏ผŒไธฆๅฐ‡่ณ‡ๆ–™้‚„ๅŽŸ่‡ณๅ…ทๆœ‰็›ธๅŒ็ฏ€้ปžๆ•ธ้‡็š„ๆ–ฐ Cassandra ๅข้›†ใ€‚ไธๆœƒๅพž่ˆŠ Cassandra ๅข้›†ๆ“ทๅ–ไปปไฝ•่ณ‡ๆ–™ใ€‚

ไธ‹ๆ–น็š„้‚„ๅŽŸๆ“ไฝœ่ชชๆ˜Ž้ฉ็”จๆ–ผไฝฟ็”จ Google Cloud Storage ๅ‚™ไปฝ็š„ๅ–ฎไธ€ๅ€ๅŸŸ้ƒจ็ฝฒไฝœๆฅญใ€‚ๅฆ‚้œ€ๅ…ถไป–้ƒจ็ฝฒๆ–นๅผ๏ผŒ่ซ‹ๅƒ้–ฑไปฅไธ‹่ชชๆ˜Ž๏ผš

ๅฆ‚่ฆ้‚„ๅŽŸ Cassandra ๅ‚™ไปฝ๏ผŒ่ซ‹ๆŒ‰็…งไธ‹ๅˆ—ๆญฅ้ฉŸๆ“ไฝœ๏ผš

  1. ๅœจ็พๆœ‰ Kubernetes ๅข้›†ไธญๅปบ็ซ‹ๆ–ฐๅ‘ฝๅ็ฉบ้–“๏ผŒ็”จๆ–ผ้‚„ๅŽŸๆททๅˆๅผๅŸท่กŒ้šŽๆฎต้ƒจ็ฝฒใ€‚่ซ‹ๅ‹ฟไฝฟ็”จๅŽŸๅง‹ๅ‘ฝๅ็ฉบ้–“ๅ็จฑๅปบ็ซ‹ๆ–ฐๅ‘ฝๅ็ฉบ้–“ใ€‚่ซ‹ๅ‹ฟไฝฟ็”จ่ˆŠๅ‘ฝๅ็ฉบ้–“้‚„ๅŽŸใ€‚
  2. ๅœจๆ นๆททๅˆๅฎ‰่ฃ็›ฎ้Œ„ไธญ๏ผŒๅปบ็ซ‹ๆ–ฐ็š„ overrides-restore.yaml ๆช”ๆกˆใ€‚
  3. ๅฐ‡ๅฎŒๆ•ด็š„ Cassandra ่จญๅฎšๅพžๅŽŸๅง‹ overrides.yaml ๆช”ๆกˆ่ค‡่ฃฝๅˆฐๆ–ฐ็š„ overrides-restore.yaml ๆช”ๆกˆไธญใ€‚่ซ‹ๅƒ่€ƒไปฅไธ‹ๆŒ‡ไปค็คบไพ‹ใ€‚
    cp ./overrides.yaml ./overrides-restore.yaml
  4. ๅœจๆ–ฐ็š„ overrides-restore.yaml ๆช”ๆกˆไธญๆ–ฐๅขžๅ‘ฝๅ็ฉบ้–“ๅ…ƒ็ด ใ€‚่ซ‹ๅ‹ฟไฝฟ็”จๅŽŸๅง‹ๅข้›†ไฝฟ็”จ็š„ๅ‘ฝๅ็ฉบ้–“ใ€‚

    ๅƒๆ•ธ

    namespace: YOUR_RESTORE_NAMESPACE
    cassandra:
      ...
      restore:
        enabled: true
        snapshotTimestamp: TIMESTAMP
        serviceAccountPath: SA_JSON_FILE_PATH
        dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH
             image:
               pullPolicy: Always
      ...

    ็ฏ„ไพ‹

    ...
        namespace: cassandra-restore
        cassandra:
          storage:
            type: gcepd
            capacity: 50Gi
            gcepd:
              replicationType: regional-pd
          sslRootCAPath: "/Users/myhome/ssh/cassandra.crt"
          sslCertPath: "/Users/myhome/ssh/cassandra.crt"
          sslKeyPath: "/Users/myhome/ssh/cassandra.key"
          auth:
            default:
              password: "abc123"
            admin:
              password: "abc234"
            ddl:
              password: "abc345"
            dml:
              password: "abc456"
          nodeSelector:
            key: cloud.google.com/gke-nodepool
            value: apigee-data
    
          restore:
            enabled: true
            snapshotTimestamp: "20210203213003"
            serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json"
            dbStorageBucket: "gs://myname-cassandra-backup"
            image:
              pullPolicy: Always
        ...
    

    ๅ…ถไธญ๏ผš

    ๅฑฌๆ€ง ่ชชๆ˜Ž
    namespace

    YOUR_RESTORE_NAMESPACE

    ๆ‚จๅœจๆญฅ้ฉŸ 1 ็‚บๆ–ฐ Cassandra ๅข้›†ๅปบ็ซ‹็š„ๆ–ฐๅ‘ฝๅ็ฉบ้–“ๅ็จฑใ€‚่ซ‹ๅ‹ฟไฝฟ็”จๅŽŸๅง‹ๅข้›†็š„ๅ‘ฝๅ็ฉบ้–“ใ€‚

    restore:enabled ้‚„ๅŽŸๅŠŸ่ƒฝ้ ่จญ็‚บๅœ็”จใ€‚ๆ‚จๅฟ…้ ˆๅฐ‡้€™้ …ๅฑฌๆ€ง่จญ็‚บ trueใ€‚
    restore:snapshotTimestamp

    TIMESTAMP

    ่ฆ้‚„ๅŽŸ็š„ๅ‚™ไปฝๅฟซ็…งๆ™‚้–“ๆˆณ่จ˜ใ€‚ๅฆ‚่ฆๆŸฅ็œ‹ๅฏไฝฟ็”จ็š„ๆ™‚้–“ๆˆณ่จ˜๏ผŒ่ซ‹ๅ‰ๅพ€ dbStorageBucket๏ผŒๆŸฅ็œ‹ๅ€ผๅ€ไธญ็š„ๆช”ๆกˆใ€‚ๆฏๅ€‹ๆช”ๆกˆๅ็จฑ้ƒฝๅŒ…ๅซๆ™‚้–“ๆˆณ่จ˜ๅ€ผ๏ผŒไพ‹ๅฆ‚๏ผš

    backup_20210203213003_apigee-cassandra-default-0.tgz

    ๅ…ถไธญ 20210203213003 ๆ˜ฏๆ‚จ่ฆ้‚„ๅŽŸ็•ถๆ™‚ๅปบ็ซ‹็š„ๅ‚™ไปฝๆ™‚๏ผŒๆœƒไฝฟ็”จ็š„ snapshotTimestamp ๅ€ผใ€‚

    restore:serviceAccountPath

    SA_JSON_FILE_PATH

    ๆ‚จ็‚บๅ‚™ไปฝไฝœๆฅญๅปบ็ซ‹็š„ๆœๅ‹™ๅธณๆˆถๅœจๆช”ๆกˆ็ณป็ตฑไธญ็š„่ทฏๅพ‘ใ€‚

    restore:dbStorageBucket

    CLOUD_STORAGE_BUCKET_PATH

    ๅ‚™ไปฝ่ณ‡ๆ–™ๅ„ฒๅญ˜ไฝ็ฝฎ็š„ Cloud Storage ๅ€ผๅ€่ทฏๅพ‘๏ผŒๆ ผๅผๅฆ‚ไธ‹๏ผš

    gs://BUCKET_NAME

    ๅฟ…้ ˆไฝฟ็”จ gs://ใ€‚

  5. ๅŸท่กŒไธ‹ๅˆ—ๆŒ‡ไปค๏ผŒ่ฎŠๆ›ด่ˆŠๅ‘ฝๅ็ฉบ้–“ไธญไปปไฝ• Cassandra ็ฏ€้ปž็š„ app ๆจ™็ฑค๏ผš
    kubectl label pods --overwrite --namespace=OLD_NAMESPACE -l app=apigee-cassandra app=apigee-cassandra-old
    
  6. ๅปบ็ซ‹ๆ–ฐ็š„ๆททๅˆๅผๅŸท่กŒ้šŽๆฎต้ƒจ็ฝฒไฝœๆฅญใ€‚้€™ๆœƒๅปบ็ซ‹ๆ–ฐ็š„ Cassandra ๅข้›†๏ผŒไธฆ้–‹ๅง‹ๅฐ‡ๅ‚™ไปฝ่ณ‡ๆ–™้‚„ๅŽŸ่‡ณๅข้›†๏ผš
    ./apigeectl init  -f ../overrides-restore.yaml
    
    ./apigeectl apply  -f ../overrides-restore.yaml
    
  7. ้‚„ๅŽŸไฝœๆฅญๅฎŒๆˆๅพŒ๏ผŒๆต้‡ๅฟ…้ ˆๅˆ‡ๆ›็‚บไฝฟ็”จๆ–ฐๅ‘ฝๅ็ฉบ้–“ไธญ็š„ Cassandra ๅข้›†ใ€‚ๅŸท่กŒไธ‹ๅˆ—ๆŒ‡ไปคไพ†ๅˆ‡ๆ›ๆต้‡๏ผš

    kubectl get rs -n OLD_NAMESPACE # look for the 'apigee-connect' replicaset
    
    kubectl patch rs -n OLD_NAMESPACE APIGEE_CONNECT_RS_NAME -p '{"spec":{"replicas" : 0}}'
    
  8. ๆต้‡ๅˆ‡ๆ›ๅฎŒๆˆๅพŒ๏ผŒๆ‚จๅฏไปฅ็งป้™ค restore ่จญๅฎš๏ผŒไธฆๅฐ‡ backup ่จญๅฎšๆ–ฐๅขž่‡ณ overrides-restore.yaml ๆช”ๆกˆ๏ผŒ่—‰ๆญค้‡ๆ–ฐ่จญๅฎš้‚„ๅŽŸๅข้›†ไธญ็š„ๅ‚™ไปฝใ€‚ๅฐ‡ YOUR_RESTORE_NAMESPACE ๆ›ฟๆ›็‚บๅœจๆญฅ้ฉŸ 1 ไธญๅปบ็ซ‹็š„ๆ–ฐๅ‘ฝๅ็ฉบ้–“ๅ็จฑใ€‚
    namespace: YOUR_RESTORE_NAMESPACE
    cassandra:
      ...
       backup:
        enabled: true
        serviceAccountPath: SA_JSON_FILE_PATH
        dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH
        schedule: BACKUP_SCHEDULE_CODE
      ...

    ็„ถๅพŒไฝฟ็”จไธ‹ๅˆ—ๆŒ‡ไปคๅฅ—็”จ backup ่จญๅฎš๏ผš

    ./apigeectl apply  -f ../overrides-restore.yaml
    

ๆŸฅ็œ‹้‚„ๅŽŸ่จ˜้Œ„

ๆ‚จๅฏไปฅๆชขๆŸฅ้‚„ๅŽŸๅทฅไฝœ่จ˜้Œ„๏ผŒไธฆไฝฟ็”จ grep ๆชขๆŸฅ error๏ผŒ็ขบไฟ้‚„ๅŽŸ่จ˜้Œ„ๆฒ’ๆœ‰้Œฏ่ชคใ€‚

็ขบ่ช้‚„ๅŽŸไฝœๆฅญๅทฒๅฎŒๆˆ

ไฝฟ็”จไธ‹ๅˆ—ๆŒ‡ไปคๆชขๆŸฅๅพฉๅŽŸไฝœๆฅญๆ˜ฏๅฆๅทฒๅฎŒๆˆ๏ผš

kubectl get pods

่ผธๅ‡บ็ตๆžœๆœƒ่ˆ‡ไธ‹ๅˆ—ๅ…งๅฎน็›ธไผผ๏ผš

NAME                           READY     STATUS      RESTARTS   AGE
apigee-cassandra-default-0     1/1       Running     0          1h
apigee-cassandra-default-1     1/1       Running     0          1h
apigee-cassandra-default-2     1/1       Running     0          59m
apigee-cassandra-restore-b4lgf 0/1       Completed   0          51m

ๆŸฅ็œ‹้‚„ๅŽŸ่จ˜้Œ„ๆช”

ไฝฟ็”จไธ‹ๅˆ—ๆŒ‡ไปคๆŸฅ็œ‹้‚„ๅŽŸ่จ˜้Œ„๏ผš

kubectl logs -f apigee-cassandra-restore-b4lgf

่ผธๅ‡บ็ตๆžœๆœƒ่ˆ‡ไธ‹ๅˆ—ๅ…งๅฎน็›ธไผผ๏ผš

Restore Logs:

Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
to download file gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1/backup_20190405011309_schema.tgz
INFO: download successfully extracted the backup files from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
finished downloading schema.cql
to create schema from 10.32.0.28

Warnings :
dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0

dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0


Warnings :
dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0

dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0

INFO: the schema has been restored
starting apigee-cassandra-default-0 in default
starting apigee-cassandra-default-1 in default
starting apigee-cassandra-default-2 in default
84 95 106
waiting on waiting nodes $pid to finish  84
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
INFO: restore downloaded  tarball and extracted the file from  gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
INFO: restore downloaded  tarball and extracted the file from  gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
INFO: restore downloaded  tarball and extracted the file from  gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
INFO  12:02:28 Configuration location: file:/etc/cassandra/cassandra.yaml
โ€ฆ...

INFO  12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed

Summary statistics:
   Connections per host    : 3
   Total files transferred : 2
   Total bytes transferred : 0.378KiB
   Total duration          : 5048 ms
   Average transfer rate   : 0.074KiB/s
   Peak transfer rate      : 0.075KiB/s

progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s)
INFO  12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed
progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s)
INFO  12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed
INFO  12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed
INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully
INFO: Restore 20190405011309 completed
INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully
INFO: Restore 20190405011309 completed
waiting on waiting nodes $pid to finish  106
Restore finished

้ฉ—่ญ‰ๅ‚™ไปฝๅทฅไฝœ

ๆ‚จไนŸๅฏไปฅๅœจๅฎ‰ๆŽ’ๅ‚™ไปฝ Cron ๅทฅไฝœๅพŒ้ฉ—่ญ‰ๅ‚™ไปฝๅทฅไฝœใ€‚ๆŽ’ๅฎš cron ๅทฅไฝœๅพŒ๏ผŒๆ‚จๆ‡‰่ฉฒๆœƒ็œ‹ๅˆฐ้กžไผผไธ‹ๆ–น็š„ๅ…งๅฎน๏ผš

kubectl get pods

่ผธๅ‡บ็ตๆžœๆœƒ่ˆ‡ไธ‹ๅˆ—ๅ…งๅฎน็›ธไผผ๏ผš

NAME                                       READY     STATUS      RESTARTS   AGE
apigee-cassandra-default-0                 1/1       Running     0          2h
apigee-cassandra-default-1                 1/1       Running     0          2h
apigee-cassandra-default-2                 1/1       Running     0          2h
apigee-cassandra-backup-1554515580-pff6s   0/1       Running     0          54s

ๆชขๆŸฅๅ‚™ไปฝ่จ˜้Œ„

ๅ‚™ไปฝๅทฅไฝœ๏ผš

  • ๅปบ็ซ‹ schema.cql ๆช”ๆกˆใ€‚
  • ๅฐ‡ๅ…ถไธŠๅ‚ณ่‡ณๅ„ฒๅญ˜้ซ”ๅ€ผๅ€ใ€‚
  • ๆœƒๅ›žๆ‡‰็ฏ€้ปž๏ผŒๅŒๆ™‚ๅ‚™ไปฝๅŠไธŠๅ‚ณ่ณ‡ๆ–™ใ€‚
  • ็ญ‰ๅพ…ๆ‰€ๆœ‰่ณ‡ๆ–™ไธŠๅ‚ณๅฎŒ็•ขใ€‚
kubectl logs -f apigee-cassandra-backup-1554515580-pff6s

่ผธๅ‡บ็ตๆžœๆœƒ่ˆ‡ไธ‹ๅˆ—ๅ…งๅฎน็›ธไผผ๏ผš

myusername-macbookpro:cassandra-backup-utility myusername$ kubectl logs -f apigee-cassandra-backup-1554577680-f9sc4
starting apigee-cassandra-default-0 in default
starting apigee-cassandra-default-1 in default
starting apigee-cassandra-default-2 in default
35 46 57
waiting on process  35
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false}
Snapshot directory: 20190406190808
INFO: backup created cassandra snapshot 20190406190808
tar: Removing leading `/' from member names
/apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/
/apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/
/apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db
Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false}
Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false}
Snapshot directory: 20190406190808
INFO: backup created cassandra snapshot 20190406190808
tar: Removing leading `/' from member names
/apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/
/apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/
/apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/manifest.json
/apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/
/apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/
/apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/manifest.json
/apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/
/apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/
/apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/manifest.json
/apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/
/apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/
/apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/manifest.json
/apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/
/apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/
/apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/manifest.json
โ€ฆโ€ฆ
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Filter.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-CompressionInfo.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Index.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Statistics.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Data.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Index.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Statistics.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-TOC.txt
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Statistics.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Summary.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Filter.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Summary.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Index.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/manifest.json
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Filter.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Digest.crc32
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Summary.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Data.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-TOC.txt
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/schema.cql
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-CompressionInfo.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Digest.crc32
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-TOC.txt
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Digest.crc32
/apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-CompressionInfo.db
โ€ฆโ€ฆ
/tmp/tokens.txt
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.
INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
INFO: removing cassandra snapshot
INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
INFO: removing cassandra snapshot
Requested clearing snapshot(s) for [all keyspaces]
INFO: Backup 20190406190808 completed
waiting on process  46
Requested clearing snapshot(s) for [all keyspaces]
INFO: Backup 20190406190808 completed
Requested clearing snapshot(s) for [all keyspaces]
waiting on process  57
INFO: Backup 20190406190808 completed
waiting result
to get schema from 10.32.0.28
INFO: /tmp/schema.cql has been generated
Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com]
tar: removing leading '/' from member names
tmp/schema.cql
Copying from <TDIN>...
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.
INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1
finished uploading schema.cql