์ด ์น์ ์์๋ Apigee ํ์ด๋ธ๋ฆฌ๋ ๋ฐํ์ ์์ญ์ ์ค์น๋ Apache Cassandra ๋ฐ์ดํฐ๋ฒ ์ด์ค ๋ง์ ๋ฐ์ดํฐ ๋ฐฑ์ ๊ณผ ๋ณต๊ตฌ๋ฅผ ๊ตฌ์ฑํ๋ ๋ฐฉ๋ฒ์ ์ค๋ช ํฉ๋๋ค. Cassandra Datastore๋ ์ฐธ์กฐํ์ธ์.
Cassandra ๋ฐฑ์ ์ ๋ํด ์์์ผ ํ ์ฌํญ
Cassandra๋ ๊ฐ ๋ฆฌ์ ๋๋ ๋ฐ์ดํฐ ์ผํฐ์ ๋ฐ์ดํฐ ๋ณต์ฌ๋ณธ์ด ์ต์ 3๊ฐ ์ด์ ์๋๋ก ๊ตฌ์ฑ๋ ๋ณต์ ๋ ๋ฐ์ดํฐ๋ฒ ์ด์ค์ ๋๋ค. Cassandra๋ ์คํธ๋ฆฌ๋ฐ ๋ณต์ ๋ฐ ์ฝ๊ธฐ ๋ณต๊ตฌ๋ฅผ ์ฌ์ฉํ์ฌ ํน์ ์์ ์ ๊ฐ ๋ฆฌ์ ๋๋ ๋ฐ์ดํฐ ์ผํฐ์ ๋ฐ์ดํฐ ๋ณต์ ๋ณธ์ ์ ์งํฉ๋๋ค.
ํ์ด๋ธ๋ฆฌ๋์์ Cassandra ๋ฐฑ์ ์ ๊ธฐ๋ณธ์ ์ผ๋ก ์ฌ์ฉ ์ค์ ๋์ง ์์ต๋๋ค. ํ์ง๋ง ์ค์๋ก ๋ฐ์ดํฐ๊ฐ ์ญ์ ๋๋ ๊ฒฝ์ฐ๋ฅผ ๋๋นํ์ฌ Cassandra ๋ฐฑ์ ์ ์ฌ์ฉ ์ค์ ํ๋ ๊ฒ์ด ์ข์ต๋๋ค.
๋ฐฑ์ ๋๋ ํญ๋ชฉ
์ด ์ฃผ์ ์์ ์ค๋ช ํ๋ ๋ฐฑ์ ๊ตฌ์ฑ์์๋ ๋ค์ ํญ๋ชฉ์ ๋ฐฑ์ ํฉ๋๋ค.
- ์ฌ์ฉ์ ์คํค๋ง๋ฅผ ํฌํจํ Cassandra ์คํค๋ง(Apigee ํค์คํ์ด์ค ์ ์)
- ๋ ธ๋๋น Cassandra ํํฐ์ ํ ํฐ ์ ๋ณด
- Cassandra ๋ฐ์ดํฐ์ ์ค๋ ์ท
๋ฐฑ์ ๋ฐ์ดํฐ๊ฐ ์ ์ฅ๋๋ ์์น
๋ฐฑ์ ๋ ๋ฐ์ดํฐ๋ Google Cloud Storage ๋ฒํท์ ์ ์ฅ๋๊ณ ๊ฐ๋ฐ์๊ฐ ์ด ๋ฒํท์ ๋ง๋ค์ด์ผ ํฉ๋๋ค. ์ด ์ฃผ์ ์์๋ ๋ฒํท ๋ง๋ค๊ธฐ์ ๊ตฌ์ฑ์ ์ค๋ช ํฉ๋๋ค.
Cassandra ๋ฐฑ์ ์์ฝ
๋ฐฑ์
์ ๋ฐํ์ ์์ญ์์ cron
์์
์ผ๋ก ์์ฝ๋ฉ๋๋ค. Cassandra ๋ฐฑ์
์ ์์ฝํ๋ ค๋ฉด ๋ค์ ์๋ด๋ฅผ ๋ฐ๋ฅด์ธ์.
- ๋ค์
create-service-account
๋ช ๋ น์ด๋ฅผ ์คํํ์ฌ ํ์คroles/storage.objectAdmin
์ญํ ์ด ์๋ Google Cloud ์๋น์ค ๊ณ์ (SA)์ ๋ง๋ญ๋๋ค. ์ด SA ์ญํ ์ ์ฌ์ฉํ๋ฉด ๋ฐฑ์ ๋ฐ์ดํฐ๋ฅผ Cloud Storage์ ์ธ ์ ์์ต๋๋ค. ํ์ด๋ธ๋ฆฌ๋ ์ค์น ๋ฃจํธ ๋๋ ํฐ๋ฆฌ์์ ๋ค์ ๋ช ๋ น์ด๋ฅผ ์คํํฉ๋๋ค. ์๋ฅผ ๋ค๋ฉด ๋ค์๊ณผ ๊ฐ์ต๋๋ค../tools/create-service-account apigee-cassandra OUTPUT_DIR
Google Cloud ์๋น์ค ๊ณ์ ์ ๋ํ ์์ธํ ๋ด์ฉ์ ์๋น์ค ๊ณ์ ๋ง๋ค๊ธฐ ๋ฐ ๊ด๋ฆฌ๋ฅผ ์ฐธ์กฐํ์ธ์../tools/create-service-account apigee-cassandra ./service-accounts
create-service-account
๋ช ๋ น์ด๋ ์๋น์ค ๊ณ์ ๋น๊ณต๊ฐ ํค๊ฐ ํฌํจ๋ JSON ํ์ผ์ ์ ์ฅํฉ๋๋ค. ํ์ผ์ ๋ช ๋ น์ด๊ฐ ์คํ๋๋ ๋์ผํ ๋๋ ํฐ๋ฆฌ์ ์ ์ฅ๋ฉ๋๋ค. ๋ค์ ๋จ๊ณ์์๋ ์ด ํ์ผ์ ๊ฒฝ๋ก๊ฐ ํ์ํฉ๋๋ค.- Cloud Storage ๋ฒํท ์์ฑ ๋ฒํท์ ์ ์ ํ ๋ฐ์ดํฐ ๋ณด๊ด ์ ์ฑ ์ ์ง์ ํฉ๋๋ค. ๋ฐ์ดํฐ ๋ณด๊ด ์ ์ฑ ์ 15์ผ๋ก ์ค์ ํ๋ ๊ฒ์ด ์ข์ต๋๋ค.
overrides.yaml
ํ์ผ์ ์ฝ๋๋ค.- ๋ค์
cassandra.backup
์์ฑ์ ์ถ๊ฐํ์ฌ ๋ฐฑ์ ์ ์ฌ์ฉ ์ค์ ํฉ๋๋ค. ์ด๋ฏธ ๊ตฌ์ฑ๋ ์์ฑ์ ์ญ์ ํ์ง ๋ง์ธ์.๋งค๊ฐ๋ณ์
cassandra: ... backup: enabled: true serviceAccountPath: SA_JSON_FILE_PATH dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH schedule: BACKUP_SCHEDULE_CODE ...
์
... cassandra: storage: type: gcepd capacity: 50Gi gcepd: replicationType: regional-pd sslRootCAPath: "/Users/myhome/ssh/cassandra.crt" sslCertPath: "/Users/myhome/ssh/cassandra.crt" sslKeyPath: "/Users/myhome/ssh/cassandra.key" auth: default: password: "abc123" admin: password: "abc234" ddl: password: "abc345" dml: password: "abc456" nodeSelector: key: cloud.google.com/gke-nodepool value: apigee-data backup: enabled: true serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json" dbStorageBucket: "gs://myname-cassandra-backup" schedule: "45 23 * * 6" ...
๊ฐ ํญ๋ชฉ์ ์๋ฏธ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค. - ๊ตฌ์ฑ ๋ณ๊ฒฝ์ฌํญ์ ์ ํด๋ฌ์คํฐ์ ์ ์ฉํฉ๋๋ค. ์๋ฅผ ๋ค๋ฉด ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
./apigeectl apply -f overrides.yaml
์์ฑ | ์ค๋ช |
---|---|
backup:enabled |
๋ฐฑ์
์ ๊ธฐ๋ณธ์ ์ผ๋ก ์ฌ์ฉ ์ค์ง๋์ด ์์ต๋๋ค. ์ด ์์ฑ์ true ๋ก ์ค์ ํด์ผ ํฉ๋๋ค. |
backup:serviceAccountPath |
SA_JSON_FILE_PATH
|
backup:dbStorageBucket |
CLOUD_STORAGE_BUCKET_PATH
|
backup:schedule |
BACKUP_SCHEDULE_CODE ๋ฐฑ์
์ด ์์๋๋ ์๊ฐ์ด๋ฉฐ ํ์ค crontab ๋ฌธ๋ฒ์ ์ง์ ๋์ด ์์ต๋๋ค. ๊ธฐ๋ณธ๊ฐ: |
๋ฐฑ์ ๋ณต์
๋ณต์์ ๋ฐฑ์ ์์น์์ ๋ฐ์ดํฐ๋ฅผ ๊ฐ์ ธ์ ๋ ธ๋ ์๊ฐ ๋์ผํ ์ Cassandra ํด๋ฌ์คํฐ์ ๋ณต์ํฉ๋๋ค. ์ด์ Cassandra ํด๋ฌ์คํฐ์์๋ ๋ฐ์ดํฐ๋ฅผ ๊ฐ์ ธ์ค์ง ์์ต๋๋ค.
์๋์ ๋ณต์ ์๋ด๋ ๋ฐฑ์ ์ Google Cloud Storage๋ฅผ ์ฌ์ฉํ๋ ๋จ์ผ ๋ฆฌ์ ๋ฐฐํฌ์ ์ ์ฉ๋ฉ๋๋ค. ๋ค๋ฅธ ๋ฐฐํฌ์ธ ๊ฒฝ์ฐ ๋ค์์ ์ฐธ์กฐํ์ธ์.
- ๋ฐฑ์ ์ Google Cloud Storage๋ฅผ ์ฌ์ฉํ์ง ์๋ ๋จ์ผ ๋ฆฌ์ ๋ฐฐํฌ์ ๊ฒฝ์ฐ Google Cloud ์์ด ๋ฐฑ์ ๋ฐ ๋ณต๊ตฌ๋ฅผ ์ฐธ์กฐํ์ธ์.
- ๋ฉํฐ ๋ฆฌ์ ๋ฐฐํฌ์ ๊ฒฝ์ฐ GKE ๋ฐ GKE On-Prem์ ๋ฉํฐ ๋ฆฌ์ ๋ฐฐํฌ๋ฅผ ์ฐธ์กฐํ์ธ์.
Cassandra ๋ฐฑ์ ์ ๋ณต์ํ๋ ค๋ฉด ๋ค์ ์๋ด๋ฅผ ๋ฐ๋ฅด์ธ์.
- ํ์ด๋ธ๋ฆฌ๋ ๋ฐํ์ ๋ฐฐํฌ๋ฅผ ๋ณต์ํ๋ ๋ฐ ์ฌ์ฉ๋๋ ๊ธฐ์กด Kubernetes ํด๋ฌ์คํฐ ๋ด์ ์ ๋ค์์คํ์ด์ค๋ฅผ ๋ง๋ญ๋๋ค. ์ ๋ค์์คํ์ด์ค์ ์๋ณธ ๋ค์์คํ์ด์ค ์ด๋ฆ์ ์ฌ์ฉํ์ง ๋ง์ธ์. ๋ณต์์ ์ด์ ๋ค์์คํ์ด์ค๋ฅผ ์ฌ์ฉํ์ง ๋ง์ธ์.
- ๋ฃจํธ ํ์ด๋ธ๋ฆฌ๋ ์ค์น ๋๋ ํฐ๋ฆฌ์ ์
overrides-restore.yaml
ํ์ผ์ ๋ง๋ญ๋๋ค. - ์ ์ฒด Cassandra ๊ตฌ์ฑ์ ์๋ณธ
overrides.yaml
ํ์ผ์์ ์overrides-restore.yaml
ํ์ผ๋ก ๋ณต์ฌํฉ๋๋ค. ์์๋ ๋ค์ ๋ช ๋ น์ด๋ฅผ ์ฐธ์กฐํ์ธ์.cp ./overrides.yaml ./overrides-restore.yaml
- ์
overrides-restore.yaml
ํ์ผ์ ๋ค์์คํ์ด์ค ์์๋ฅผ ์ถ๊ฐํฉ๋๋ค. ์๋ณธ ํด๋ฌ์คํฐ์ ์ฌ์ฉํ ๋ค์์คํ์ด์ค์ ๋์ผํ ๋ค์์คํ์ด์ค๋ฅผ ์ฌ์ฉํ์ง ๋ง์ธ์.๋งค๊ฐ๋ณ์
namespace: YOUR_RESTORE_NAMESPACE cassandra: ... restore: enabled: true snapshotTimestamp: TIMESTAMP serviceAccountPath: SA_JSON_FILE_PATH dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH image: pullPolicy: Always ...
์
... namespace: cassandra-restore cassandra: storage: type: gcepd capacity: 50Gi gcepd: replicationType: regional-pd sslRootCAPath: "/Users/myhome/ssh/cassandra.crt" sslCertPath: "/Users/myhome/ssh/cassandra.crt" sslKeyPath: "/Users/myhome/ssh/cassandra.key" auth: default: password: "abc123" admin: password: "abc234" ddl: password: "abc345" dml: password: "abc456" nodeSelector: key: cloud.google.com/gke-nodepool value: apigee-data restore: enabled: true snapshotTimestamp: "20210203213003" serviceAccountPath: "/Users/myhome/.ssh/my_cassandra_backup.json" dbStorageBucket: "gs://myname-cassandra-backup" image: pullPolicy: Always ...
๊ฐ ํญ๋ชฉ์ ์๋ฏธ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
์์ฑ ์ค๋ช namespace
YOUR_RESTORE_NAMESPACE
์ Cassandra ํด๋ฌ์คํฐ์ 1๋จ๊ณ์์ ๋ง๋ ์ ๋ค์์คํ์ด์ค์ ์ด๋ฆ์ ๋๋ค. ์๋ ํด๋ฌ์คํฐ์ ์ฌ์ฉํ ๋ค์์คํ์ด์ค์ ๋์ผํ ๋ค์ ์คํ์ด์ค๋ฅผ ์ฌ์ฉํ์ง ๋ง์ธ์.
restore:enabled
๋ณต์์ ๊ธฐ๋ณธ์ ์ผ๋ก ์ฌ์ฉ ์ค์ง๋์ด ์์ต๋๋ค. ์ด ์์ฑ์ true
๋ก ์ค์ ํด์ผ ํฉ๋๋ค.restore:snapshotTimestamp
TIMESTAMP
๋ณต์ํ ๋ฐฑ์ ์ค๋ ์ท์ ํ์์คํฌํ์ ๋๋ค. ์ฌ์ฉํ ์ ์๋ ํ์์คํฌํ๋ฅผ ํ์ธํ๋ ค๋ฉด
dbStorageBucket
์ผ๋ก ์ด๋ํ์ฌ ๋ฒํท์ ์๋ ํ์ผ์ ํ์ธํฉ๋๋ค. ๊ฐ ํ์ผ ์ด๋ฆ์๋ ๋ค์๊ณผ ๊ฐ์ ํ์์คํฌํ ๊ฐ์ด ํฌํจ๋ฉ๋๋ค.backup_20210203213003_apigee-cassandra-default-0.tgz
์ฌ๊ธฐ์ 20210203213003์ ํด๋น ์์ ์ ์์ฑ๋ ๋ฐฑ์ ์ ๋ณต์ํ๋ ค๋ ๊ฒฝ์ฐ์ ์ฌ์ฉํ๋
snapshotTimestamp
๊ฐ์ ๋๋ค.restore:serviceAccountPath
SA_JSON_FILE_PATH
๋ฐฑ์ ์ฉ์ผ๋ก ๋ง๋ ์๋น์ค ๊ณ์ ์ ํ์ผ ์์คํ ๊ฒฝ๋ก์ ๋๋ค.
restore:dbStorageBucket
CLOUD_STORAGE_BUCKET_PATH
๋ฐฑ์ ๋ฐ์ดํฐ๊ฐ ๋ค์ ํ์์ผ๋ก ์ ์ฅ๋๋ Cloud Storage ๋ฒํท ๊ฒฝ๋ก์ ๋๋ค.
gs://BUCKET_NAME
gs://
๋ ํ์ ํญ๋ชฉ์ ๋๋ค. - ๋ค์ ๋ช
๋ น์ด๋ฅผ ์คํํ์ฌ ์ด์ ๋ค์์คํ์ด์ค์ Cassandra ๋
ธ๋์์
app
๋ผ๋ฒจ์ ๋ณ๊ฒฝํฉ๋๋ค.kubectl label pods --overwrite --namespace=OLD_NAMESPACE -l app=apigee-cassandra app=apigee-cassandra-old
- ์ ํ์ด๋ธ๋ฆฌ๋ ๋ฐํ์ ๋ฐฐํฌ๋ฅผ ์์ฑํฉ๋๋ค. ๊ทธ๋ฌ๋ฉด ์ Cassandra ํด๋ฌ์คํฐ๊ฐ ์์ฑ๋๊ณ ๋ฐฑ์
๋ฐ์ดํฐ๊ฐ ํด๋ฌ์คํฐ๋ก ๋ณต์๋๊ธฐ ์์ํฉ๋๋ค.
./apigeectl init -f ../overrides-restore.yaml
./apigeectl apply -f ../overrides-restore.yaml
-
๋ณต์์ด ์๋ฃ๋๋ฉด ์ ๋ค์์คํ์ด์ค์์ Cassandra ํด๋ฌ์คํฐ๋ฅผ ์ฌ์ฉํ๋๋ก ํธ๋ํฝ์ ์ ํํด์ผ ํฉ๋๋ค. ๋ค์ ๋ช ๋ น์ด๋ฅผ ์คํํ์ฌ ํธ๋ํฝ์ ์ ํํฉ๋๋ค.
kubectl get rs -n OLD_NAMESPACE # look for the 'apigee-connect' replicaset
kubectl patch rs -n OLD_NAMESPACE APIGEE_CONNECT_RS_NAME -p '{"spec":{"replicas" : 0}}'
- ํธ๋ํฝ ์ ํ์ด ์๋ฃ๋๋ฉด
restore
๊ตฌ์ฑ์ ์ญ์ ํ๊ณbackup
๊ตฌ์ฑ์overrides-restore.yaml
ํ์ผ์ ์ถ๊ฐํ์ฌ ๋ณต์๋ ํด๋ฌ์คํฐ์์ ๋ฐฑ์ ์ ๋ค์ ๊ตฌ์ฑํ ์ ์์ต๋๋ค. YOUR_RESTORE_NAMESPACE๋ฅผ 1๋จ๊ณ์์ ๋ง๋ ์ ๋ค์์คํ์ด์ค ์ด๋ฆ์ผ๋ก ๋ฐ๊ฟ๋๋ค.namespace: YOUR_RESTORE_NAMESPACE cassandra: ... backup: enabled: true serviceAccountPath: SA_JSON_FILE_PATH dbStorageBucket: CLOUD_STORAGE_BUCKET_PATH schedule: BACKUP_SCHEDULE_CODE ...
๊ทธ๋ฐ ํ ๋ค์ ๋ช ๋ น์ด๋ฅผ ์ฌ์ฉํ์ฌ
backup
๊ตฌ์ฑ์ ์ ์ฉํฉ๋๋ค../apigeectl apply -f ../overrides-restore.yaml
๋ณต์ ๋ก๊ทธ ๋ณด๊ธฐ
๋ณต์ ์์
๋ก๊ทธ๋ฅผ ํ์ธํ๊ณ grep
์ ์ฌ์ฉํ์ฌ error
๋ฅผ ํ์ธํ์ฌ ๋ณต์ ๋ก๊ทธ์ ์ค๋ฅ๊ฐ ์๋์ง ํ์ธํ ์ ์์ต๋๋ค.
๋ณต์์ด ์๋ฃ๋์๋์ง ํ์ธ
๋ค์ ๋ช ๋ น์ด๋ฅผ ์ฌ์ฉํ์ฌ ๋ณต์ ์์ ์ด ์๋ฃ๋์๋์ง ํ์ธํฉ๋๋ค.
kubectl get pods
์ถ๋ ฅ์ ๋ค์๊ณผ ๋น์ทํฉ๋๋ค.
NAME READY STATUS RESTARTS AGE apigee-cassandra-default-0 1/1 Running 0 1h apigee-cassandra-default-1 1/1 Running 0 1h apigee-cassandra-default-2 1/1 Running 0 59m apigee-cassandra-restore-b4lgf 0/1 Completed 0 51m
๋ณต์ ๋ก๊ทธ ๋ณด๊ธฐ
๋ค์ ๋ช ๋ น์ด๋ฅผ ์ฌ์ฉํ์ฌ ๋ณต์ ๋ก๊ทธ๋ฅผ ๋ด ๋๋ค.
kubectl logs -f apigee-cassandra-restore-b4lgf
์ถ๋ ฅ์ ๋ค์๊ณผ ๋น์ทํฉ๋๋ค.
Restore Logs: Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] to download file gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1/backup_20190405011309_schema.tgz INFO: download successfully extracted the backup files from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 finished downloading schema.cql to create schema from 10.32.0.28 Warnings : dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 Warnings : dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 dclocal_read_repair_chance table option has been deprecated and will be removed in version 4.0 INFO: the schema has been restored starting apigee-cassandra-default-0 in default starting apigee-cassandra-default-1 in default starting apigee-cassandra-default-2 in default 84 95 106 waiting on waiting nodes $pid to finish 84 Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: restore downloaded tarball and extracted the file from gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO 12:02:28 Configuration location: file:/etc/cassandra/cassandra.yaml โฆ... INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed Summary statistics: Connections per host : 3 Total files transferred : 2 Total bytes transferred : 0.378KiB Total duration : 5048 ms Average transfer rate : 0.074KiB/s Peak transfer rate : 0.075KiB/s progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s) INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed progress: [/10.32.1.155]0:1/1 100% 1:1/1 100% [/10.32.0.28]1:1/1 100% 0:1/1 100% [/10.32.3.220]0:1/1 100% 1:1/1 100% total: 100% 0.000KiB/s (avg: 0.074KiB/s) INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed INFO 12:02:41 [Stream #e013ee80-5863-11e9-8458-353e9e3cb7f9] All sessions completed INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully INFO: Restore 20190405011309 completed INFO: ./apigee/data/cassandra/data/ks1/user-9fbae960571411e99652c7b15b2db6cc restored successfully INFO: Restore 20190405011309 completed waiting on waiting nodes $pid to finish 106 Restore finished
๋ฐฑ์ ์์ ํ์ธ
๋ํ ๋ฐฑ์ cronjob์ด ์์ฝ๋๋ฉด ๋ฐฑ์ ์์ ์ ํ์ธํ ์ ์์ต๋๋ค. cronjob์ด ์์ฝ๋๋ฉด ๋ค์๊ณผ ๊ฐ์ ์ถ๋ ฅ์ด ํ์๋ฉ๋๋ค.
kubectl get pods
์ถ๋ ฅ์ ๋ค์๊ณผ ๋น์ทํฉ๋๋ค.
NAME READY STATUS RESTARTS AGE apigee-cassandra-default-0 1/1 Running 0 2h apigee-cassandra-default-1 1/1 Running 0 2h apigee-cassandra-default-2 1/1 Running 0 2h apigee-cassandra-backup-1554515580-pff6s 0/1 Running 0 54s
๋ฐฑ์ ๋ก๊ทธ ํ์ธ
๋ฐฑ์ ์์ ์ ๋ค์๊ณผ ๊ฐ์ต๋๋ค.
schema.cql
ํ์ผ์ ๋ง๋ญ๋๋ค.- ์คํ ๋ฆฌ์ง ๋ฒํท์ ์ ๋ก๋ํฉ๋๋ค.
- ๋ ธ๋๋ฅผ ์์ฝํ์ฌ ๋ฐ์ดํฐ๋ฅผ ๋ฐฑ์ ํ๊ณ ๋์์ ์ ๋ก๋ํฉ๋๋ค.
- ๋ชจ๋ ๋ฐ์ดํฐ๊ฐ ์ ๋ก๋๋ ๋๊น์ง ๊ธฐ๋ค๋ฆฝ๋๋ค.
kubectl logs -f apigee-cassandra-backup-1554515580-pff6s
์ถ๋ ฅ์ ๋ค์๊ณผ ๋น์ทํฉ๋๋ค.
myusername-macbookpro:cassandra-backup-utility myusername$ kubectl logs -f apigee-cassandra-backup-1554577680-f9sc4 starting apigee-cassandra-default-0 in default starting apigee-cassandra-default-1 in default starting apigee-cassandra-default-2 in default 35 46 57 waiting on process 35 Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Snapshot directory: 20190406190808 INFO: backup created cassandra snapshot 20190406190808 tar: Removing leading `/' from member names /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/ /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/ /apigee/data/cassandra/data/ks1/mytest3-37bc2df0587811e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Requested creating snapshot(s) for [all keyspaces] with snapshot name [20190406190808] and options {skipFlush=false} Snapshot directory: 20190406190808 INFO: backup created cassandra snapshot 20190406190808 tar: Removing leading `/' from member names /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/ /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/ /apigee/data/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/ /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/ /apigee/data/cassandra/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/ /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/ /apigee/data/cassandra/data/system/range_xfers-55d764384e553f8b9f6e676d4af3976d/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/ /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/ /apigee/data/cassandra/data/system/peer_events-59dfeaea8db2334191ef109974d81484/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/ /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/ /apigee/data/cassandra/data/system/built_views-4b3c50a9ea873d7691016dbc9c38494a/snapshots/20190406190808/manifest.json โฆโฆ /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/ /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/ /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-CompressionInfo.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Statistics.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Index.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/manifest.json /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Filter.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-2-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Summary.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/schema.cql /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-CompressionInfo.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-TOC.txt /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-Data.db /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-3-big-Digest.crc32 /apigee/data/cassandra/data/ks2/user-d6d39d70586311e98e8d875b0ed64754/snapshots/20190406190808/mc-1-big-CompressionInfo.db โฆโฆ /tmp/tokens.txt / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: removing cassandra snapshot INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 INFO: removing cassandra snapshot Requested clearing snapshot(s) for [all keyspaces] INFO: Backup 20190406190808 completed waiting on process 46 Requested clearing snapshot(s) for [all keyspaces] INFO: Backup 20190406190808 completed Requested clearing snapshot(s) for [all keyspaces] waiting on process 57 INFO: Backup 20190406190808 completed waiting result to get schema from 10.32.0.28 INFO: /tmp/schema.cql has been generated Activated service account credentials for: [apigee-cassandra-backup-svc@gce-myusername.iam.gserviceaccount.com] tar: removing leading '/' from member names tmp/schema.cql Copying from <TDIN>... / [1 files][ 0.0 B/ 0.0 B] Operation completed over 1 objects. INFO: backup created tarball and transferred the file to gs://gce-myusername-apigee-cassandra-backup/apigeecluster/dc-1 finished uploading schema.cql