added new features backup, slack notification, named indices - #16
Conversation
krallin
left a comment
krallin
left a comment
There was a problem hiding this comment.
Thanks for your time. I'm not sure this is quite ready to be merged in.
If you're planning the changes I requested, let me know, and I can provide further guidance.
Thanks!
| REPOSITORY_NAME=${REPOSITORY_NAME:-logstash_snapshots} | ||
| WAIT_SECONDS=${WAIT_SECONDS:-1800} | ||
| MAX_DAYS_TO_KEEP=${MAX_DAYS_TO_KEEP:-30} | ||
| MAX_DAYS_TO_KEEP=${MAX_DAYS_TO_KEEP:-60} |
There was a problem hiding this comment.
Did you meant to commit this?
|
|
||
| curl --fail -w "\n" -sS -XPUT ${SNAPSHOT_URL} -d "{ | ||
| \"indices\": \"${INDEX_NAME}\", | ||
| \"ignore_unavailable\": true, |
There was a problem hiding this comment.
Can you explain this change?
| for index in $(curl -sS ${DATABASE_URL}/_cat/indices | grep $index_nm- | sed $SUBSTITUTION | sort); do | ||
|
|
||
| # debug information | ||
| echo "$index" |
There was a problem hiding this comment.
Did you mean to commit this?
| * `S3_BUCKET_BASE_PATH`: Destination path within bucket (Optional) | ||
| * `S3_ACCESS_KEY_ID`: The access key you generated in step 3. | ||
| * `S3_SECRET_ACCESS_KEY`: The secret key you generated in step 3. | ||
| * `INDEX_ARRAY`: array of indices, INDEX_ARRAY="logstash nginx", separated with white space |
There was a problem hiding this comment.
This should explain how these indice names should be formatted. logstash indices have a format we can reasonably rely on to extract the date, but that's not the case with arbitrary indices.
| : ${S3_ACCESS_KEY_ID:?"Error: S3_ACCESS_KEY_ID environment variable not set"} | ||
| : ${S3_SECRET_ACCESS_KEY:?"Error: S3_SECRET_ACCESS_KEY environment variable not set"} | ||
| # list of index patterns to backup, should be separated with white space 'logstash filebeat' | ||
| INDEX_ARRAY=${INDEX_ARRAY:-logstash} |
There was a problem hiding this comment.
Why put this one here when all other default variables are below?
| local SNAPSHOT_URL=${REPOSITORY_URL}/${INDEX_NAME} | ||
| local INDEX_URL=${DATABASE_URL}/${INDEX_NAME} | ||
|
|
||
| grep -q SUCCESS <(curl -sS ${SNAPSHOT_URL}) |
There was a problem hiding this comment.
There's some major code duplication going on over here with archive_index - this would need to be fixed.
|
|
||
| # debug information | ||
| echo "$index" | ||
| CURR_DATE=`echo $index | sed -re $SUBSTITUTION_date` |
There was a problem hiding this comment.
Can we use $() for consistency with everything else in this file?
| let fl++ | ||
| fi | ||
| # if less then cuttoff date, just backup it to s3 without deletion | ||
| else |
There was a problem hiding this comment.
All the logic above just checks for "any snapshot exists" to mean "this is safe to delete" (because that's how this has worked so far). However, if you're now snapshotting indices regardless of their creation date, you might end up creating a snapshot for today, which will be incomplete.
Unfortunately, since that means a snapshot exists, it seems that no final snapshot will be created before finally deleting the index.
Am I missing something here?
| done | ||
|
|
||
| echo -e "results of archiving are -> index_name -> $B $index_nm $NB backuped -> $G $bk $N, archived -> $ar $R, failed -> $fl $N" | ||
| # formating array with message, that ll be send to slack |
| # if slack webhook set, send notify | ||
| if [[ -z "$SLACK_HOOK" ]]; then | ||
|
|
||
| echo "slack hook is not defined" |
There was a problem hiding this comment.
We don't need to remind users that they haven't enabled this feature every time the job runs. If $SLACK_HOOK is set, then notify, if not, then skip it altogether.
| fi | ||
| fi | ||
|
|
||
| # itterate array of indices |
| let ar++ | ||
| else | ||
| echo "$(now): ${index} archival failed." | ||
| let fl++ |
3 participants
Backup: now indices before cutoff date are backuped to S3 without deletion.
Named indices: you can set array of indices - INDEX_ARRAY="logstash filebeat etc etc", backup script ll iterate it.
Slack notification: results of backup ll send to slack, if slack hook set.