Skip to content

feat: date-based Elasticsearch index names via strftime (#27) - #54

Merged
y-ken merged 1 commit into
masterfrom
feature/date-based-index-name
Jun 16, 2026
Merged

feat: date-based Elasticsearch index names via strftime (#27)#54
y-ken merged 1 commit into
masterfrom
feature/date-based-index-name

Conversation

@y-ken

@y-ken y-ken commented Jun 16, 2026

Copy link
Copy Markdown
Owner

Summary

Implements #27: support Logstash-style date-based index names (e.g. myindex-20180831).

How it works

mysql_replicator_elasticsearch resolves the target index from the tag's
index_name segment. This PR expands strftime tokens in that segment using
the record's event time:

def resolve_index_name(index_name, time)
  return index_name unless index_name && index_name.include?('%')
  Time.at(time.to_i).strftime(index_name)
rescue => e
  log.warn "...failed to expand index name '#{index_name}': #{e.message}"
  index_name
end

Users opt in by putting the tokens in the index-name part of the input plugin's tag:

tag myindex-%Y%m%d.mytype.${event}.${primary_key}

Index names without a % are returned unchanged, so existing configurations
are unaffected
(fully backward compatible).

Notes

  • Uses the record's event time (deterministic, matches when the row was emitted).
  • README documents the feature and the delete-vs-rotation caveat (date-rotated
    indices suit insert-only data, since a delete targets the index for the
    delete's own time).

Tests

  • test_expands_strftime_tokens_in_index_name — a %Y%m%d tag produces a dated index.
  • test_index_name_without_token_is_unchanged — pass-through for plain names.

Verified locally on Ruby 3.4 (18 tests, 0 failures).

Closes #27

🤖 Generated with Claude Code

`mysql_replicator_elasticsearch` resolves the index name from the tag. Now,
if that index-name segment contains strftime tokens (e.g. %Y%m%d), they are
expanded using the record's event time, enabling Logstash-style dated
indices like `myindex-20180831`.

Users opt in simply by putting the tokens in the index-name part of the
input plugin's `tag` (e.g. `tag myindex-%Y%m%d.mytype.${event}.${primary_key}`).
Index names containing no `%` are returned unchanged, so existing configs are
unaffected.

Add a unit test for the expansion (and an explicit pass-through test) and
document the feature, including the delete-vs-rotation caveat, in the README.

Closes #27

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@y-ken
y-ken merged commit a17f82d into master Jun 16, 2026
9 checks passed
@y-ken
y-ken deleted the feature/date-based-index-name branch June 16, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How to change Index name in elasticsearch like logstash with date

1 participant