Skip to content

tests: failpoint_tests::txn_cleanup_2pc_locks is flaky #516

Open
Open
tests: failpoint_tests::txn_cleanup_2pc_locks is flaky#516

Description

@pingyu
pingyu
opened on Jan 14, 2026

CI link: https://github.com/tikv/client-rust/actions/runs/20982618570/job/60310631494?pr=515

PR: #515

Nextest run ID 1784fdea-b79b-49f5-ae6b-ac103c96cca0 with nextest profile: ci
    Starting 26 tests across 4 binaries (55 tests skipped)
        SLOW [> 60.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>120.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>180.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>240.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>300.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>360.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>420.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>480.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>540.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
 TERMINATING [>600.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
     TIMEOUT [ 601.103s] ( 1/26) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        PASS [   9.742s] ( 2/26) tikv-client::failpoint_tests txn_cleanup_async_commit_locks
        PASS [   1.373s] ( 3/26) tikv-client::failpoint_tests txn_cleanup_locks_batch_size
        PASS [   5.370s] ( 4/26) tikv-client::failpoint_tests txn_cleanup_range_async_commit_locks
        PASS [   6.477s] ( 5/26) tikv-client::failpoint_tests txn_optimistic_heartbeat
        PASS [   1.349s] ( 6/26) tikv-client::integration_tests txn_bank_transfer
        PASS [   0.524s] ( 7/26) tikv-client::integration_tests txn_batch_mutate_optimistic
        PASS [   4.546s] ( 8/26) tikv-client::integration_tests txn_batch_mutate_pessimistic
        PASS [   0.492s] ( 9/26) tikv-client::integration_tests txn_crud
        PASS [   0.522s] (10/26) tikv-client::integration_tests txn_get_for_update
        PASS [   0.026s] (11/26) tikv-client::integration_tests txn_get_timestamp
        PASS [   0.485s] (12/26) tikv-client::integration_tests txn_insert_duplicate_keys
        PASS [   0.503s] (13/26) tikv-client::integration_tests txn_key_exists
        PASS [  13.040s] (14/26) tikv-client::integration_tests txn_lock_keys
        PASS [  13.055s] (15/26) tikv-client::integration_tests txn_lock_keys_error_handle
        PASS [   0.492s] (16/26) tikv-client::integration_tests txn_pessimistic
        PASS [   0.518s] (17/26) tikv-client::integration_tests txn_pessimistic_delete
        PASS [  23.512s] (18/26) tikv-client::integration_tests txn_pessimistic_heartbeat
        PASS [   1.152s] (19/26) tikv-client::integration_tests txn_pessimistic_rollback
        PASS [   1.636s] (20/26) tikv-client::integration_tests txn_read
        PASS [   0.487s] (21/26) tikv-client::integration_tests txn_scan
        PASS [   0.485s] (22/26) tikv-client::integration_tests txn_scan_reverse
        PASS [   0.802s] (23/26) tikv-client::integration_tests txn_scan_reverse_multi_regions
        PASS [   9.626s] (24/26) tikv-client::integration_tests txn_split_batch
        PASS [   0.573s] (25/26) tikv-client::integration_tests txn_unsafe_destroy_range
        PASS [   1.018s] (26/26) tikv-client::integration_tests txn_update_safepoint
────────────
     Summary [ 698.921s] 26 tests run: 25 passed, 1 timed out, 55 skipped
     TIMEOUT [ 601.103s] ( 1/26) tikv-client::failpoint_tests txn_cleanup_2pc_locks
  stdout ───

    running 1 test
    init finish with 105 regions
    test txn_cleanup_2pc_locks has been running for over 60 seconds
  stderr ───
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:37Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:38Z INFO  failpoint_tests::common] splitting regions...
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:39Z INFO  failpoint_tests] test no commit
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  failpoint_tests] test all committed
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:47Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:49Z INFO  tikv_client::pd::timestamp] TSO stream terminated

    (test timed out)

error: test run failed
make: *** [Makefile:32: integration-test-txn] Error 100

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions